Bump json from 20201115 to 20210307#9
Merged
Conversation
Bumps [json](https://github.com/douglascrockford/JSON-java) from 20201115 to 20210307. - [Release notes](https://github.com/douglascrockford/JSON-java/releases) - [Commits](https://github.com/douglascrockford/JSON-java/commits) --- updated-dependencies: - dependency-name: org.json:json dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com>
Merged
lvca
added a commit
that referenced
this pull request
Jan 6, 2026
Fix for issue #3091 (Fix for Edge Index Corruption) ## Solution Summary Implemented a general fix that **prevents RID reuse within the same transaction**, eliminating the root cause of index corruption. ## Root Cause When an edge was deleted and its RID immediately reused within the same transaction with different property values, the index entries from the deleted record were not properly removed before the RID was assigned to a new record. This caused stale index entries pointing to the reused RID. ## The Fix ### 1. Track Deleted RIDs in Transaction Context **File**: `engine/src/main/java/com/arcadedb/database/TransactionContext.java` Added a `Set<RID>` to track all RIDs deleted in the current transaction: ```java private final Set<RID> deletedRecordsInTx = new HashSet<>(); public void addDeletedRecord(final RID rid) { deletedRecordsInTx.add(rid); } public boolean isDeletedInTransaction(final RID rid) { return deletedRecordsInTx.contains(rid); } ``` The set is cleared in the `reset()` method when the transaction commits or rolls back. ### 2. Mark RIDs as Deleted **File**: `engine/src/main/java/com/arcadedb/engine/LocalBucket.java:line 1007` When a record is deleted, add its RID to the transaction's deleted set: ```java // POINTER = 0 MEANS DELETED page.writeUnsignedInt(PAGE_RECORD_TABLE_OFFSET + positionInPage * INT_SERIALIZED_SIZE, 0); // Track deleted RID to prevent reuse within the same transaction database.getTransaction().addDeletedRecord(rid); ``` ### 3. Prevent Reuse During Allocation **File**: `engine/src/main/java/com/arcadedb/engine/LocalBucket.java:line 1588-1597` Modified the `getFreeSpaceInPage()` method to skip RIDs deleted in the current transaction: ```java for (int i = 0; i < pageAnalysis.totalRecordsInPage; i++) { final int recordPositionInPage = getRecordPositionInPage(pageAnalysis.page, i); if (recordPositionInPage == 0) { // Check if this position was deleted in the current transaction final RID potentialRID = new RID(database, file.getFileId(), ((long) pageAnalysis.page.getPageId().getPageNumber()) * maxRecordsInPage + i); if (!database.getTransaction().isDeletedInTransaction(potentialRID)) { // REUSE THE FIRST AVAILABLE POSITION FROM DELETED RECORD (from previous transactions only) if (pageAnalysis.availablePositionIndex == -1) pageAnalysis.availablePositionIndex = i; } } else if (recordPositionInPage > pageAnalysis.lastRecordPositionInPage) pageAnalysis.lastRecordPositionInPage = recordPositionInPage; } ``` ### 4. Supporting Fixes **File**: `engine/src/main/java/com/arcadedb/database/DocumentIndexer.java:line 416` - Moved `unsetDirty()` call to AFTER property extraction for index deletion **File**: `engine/src/main/java/com/arcadedb/database/LocalDatabase.java:line 1031` - Added handling to load full edge when deleting a `LightEdge` (which returns null for all properties) ## Benefits of This Approach 1. **General Solution**: Prevents RID reuse within ANY transaction, not just for edges 2. **Prevents Multiple Issues**: Eliminates a whole class of potential bugs related to RID reuse 3. **Simple and Clean**: Minimal code changes, easy to understand and maintain 4. **No Performance Impact**: HashSet lookup is O(1), and the set is cleared on transaction completion 5. **Cleaner Transaction Semantics**: RIDs allocated in a transaction remain stable until commit ## Test Results The test case `EdgeIndexCorruptionTest` now passes: **Before Fix:** ``` After Transaction #3: - Edge #9:0: from_id=trs_2 (REUSED RID!) - Edge #9:1: from_id=trs_1 Query for from_id='trs_1' returned 2 edges (WRONG - stale index entry for #9:0) ``` **After Fix:** ``` After Transaction #3: - Edge #9:1: from_id=trs_2 - Edge #9:2: from_id=trs_1 (RID #9:0 NOT reused!) Query for from_id='trs_1' returned 1 edge (CORRECT!) ``` ## Files Modified 1. **engine/src/main/java/com/arcadedb/database/TransactionContext.java** - Added `deletedRecordsInTx` Set - Added `addDeletedRecord()` and `isDeletedInTransaction()` methods - Clear deleted set in `reset()` method 2. **engine/src/main/java/com/arcadedb/engine/LocalBucket.java** - Call `addDeletedRecord()` when record is deleted - Check `isDeletedInTransaction()` before reusing RID in `getFreeSpaceInPage()` 3. **engine/src/main/java/com/arcadedb/database/DocumentIndexer.java** - Moved `unsetDirty()` to after property extraction 4. **engine/src/main/java/com/arcadedb/database/LocalDatabase.java** - Added `LightEdge` import - Handle `LightEdge` by loading full edge before index deletion 5. **engine/src/test/java/com/arcadedb/graph/EdgeIndexCorruptionTest.java** - New test case that reproduces and validates the fix ## Backward Compatibility This fix is **fully backward compatible**: - No API changes - No schema changes - No data migration required - Existing databases work without modification ## Performance Considerations - **Memory**: O(N) where N is the number of deletions in a transaction (typically small) - **CPU**: O(1) HashSet operations for add/check - **Cleanup**: Automatic on transaction commit/rollback The performance impact is negligible as: 1. Most transactions don't delete many records 2. HashSet operations are very fast 3. The set is cleared after each transaction ## Future Enhancements This fix could be extended to: 1. Track all RID allocations in the transaction for rollback purposes 2. Provide transaction-level RID allocation statistics 3. Detect and warn about high RID churn within transactions
This was referenced Jan 27, 2026
robfrank
pushed a commit
that referenced
this pull request
Feb 11, 2026
Fix for issue #3091 (Fix for Edge Index Corruption) ## Solution Summary Implemented a general fix that **prevents RID reuse within the same transaction**, eliminating the root cause of index corruption. ## Root Cause When an edge was deleted and its RID immediately reused within the same transaction with different property values, the index entries from the deleted record were not properly removed before the RID was assigned to a new record. This caused stale index entries pointing to the reused RID. ## The Fix ### 1. Track Deleted RIDs in Transaction Context **File**: `engine/src/main/java/com/arcadedb/database/TransactionContext.java` Added a `Set<RID>` to track all RIDs deleted in the current transaction: ```java private final Set<RID> deletedRecordsInTx = new HashSet<>(); public void addDeletedRecord(final RID rid) { deletedRecordsInTx.add(rid); } public boolean isDeletedInTransaction(final RID rid) { return deletedRecordsInTx.contains(rid); } ``` The set is cleared in the `reset()` method when the transaction commits or rolls back. ### 2. Mark RIDs as Deleted **File**: `engine/src/main/java/com/arcadedb/engine/LocalBucket.java:line 1007` When a record is deleted, add its RID to the transaction's deleted set: ```java // POINTER = 0 MEANS DELETED page.writeUnsignedInt(PAGE_RECORD_TABLE_OFFSET + positionInPage * INT_SERIALIZED_SIZE, 0); // Track deleted RID to prevent reuse within the same transaction database.getTransaction().addDeletedRecord(rid); ``` ### 3. Prevent Reuse During Allocation **File**: `engine/src/main/java/com/arcadedb/engine/LocalBucket.java:line 1588-1597` Modified the `getFreeSpaceInPage()` method to skip RIDs deleted in the current transaction: ```java for (int i = 0; i < pageAnalysis.totalRecordsInPage; i++) { final int recordPositionInPage = getRecordPositionInPage(pageAnalysis.page, i); if (recordPositionInPage == 0) { // Check if this position was deleted in the current transaction final RID potentialRID = new RID(database, file.getFileId(), ((long) pageAnalysis.page.getPageId().getPageNumber()) * maxRecordsInPage + i); if (!database.getTransaction().isDeletedInTransaction(potentialRID)) { // REUSE THE FIRST AVAILABLE POSITION FROM DELETED RECORD (from previous transactions only) if (pageAnalysis.availablePositionIndex == -1) pageAnalysis.availablePositionIndex = i; } } else if (recordPositionInPage > pageAnalysis.lastRecordPositionInPage) pageAnalysis.lastRecordPositionInPage = recordPositionInPage; } ``` ### 4. Supporting Fixes **File**: `engine/src/main/java/com/arcadedb/database/DocumentIndexer.java:line 416` - Moved `unsetDirty()` call to AFTER property extraction for index deletion **File**: `engine/src/main/java/com/arcadedb/database/LocalDatabase.java:line 1031` - Added handling to load full edge when deleting a `LightEdge` (which returns null for all properties) ## Benefits of This Approach 1. **General Solution**: Prevents RID reuse within ANY transaction, not just for edges 2. **Prevents Multiple Issues**: Eliminates a whole class of potential bugs related to RID reuse 3. **Simple and Clean**: Minimal code changes, easy to understand and maintain 4. **No Performance Impact**: HashSet lookup is O(1), and the set is cleared on transaction completion 5. **Cleaner Transaction Semantics**: RIDs allocated in a transaction remain stable until commit ## Test Results The test case `EdgeIndexCorruptionTest` now passes: **Before Fix:** ``` After Transaction #3: - Edge #9:0: from_id=trs_2 (REUSED RID!) - Edge #9:1: from_id=trs_1 Query for from_id='trs_1' returned 2 edges (WRONG - stale index entry for #9:0) ``` **After Fix:** ``` After Transaction #3: - Edge #9:1: from_id=trs_2 - Edge #9:2: from_id=trs_1 (RID #9:0 NOT reused!) Query for from_id='trs_1' returned 1 edge (CORRECT!) ``` ## Files Modified 1. **engine/src/main/java/com/arcadedb/database/TransactionContext.java** - Added `deletedRecordsInTx` Set - Added `addDeletedRecord()` and `isDeletedInTransaction()` methods - Clear deleted set in `reset()` method 2. **engine/src/main/java/com/arcadedb/engine/LocalBucket.java** - Call `addDeletedRecord()` when record is deleted - Check `isDeletedInTransaction()` before reusing RID in `getFreeSpaceInPage()` 3. **engine/src/main/java/com/arcadedb/database/DocumentIndexer.java** - Moved `unsetDirty()` to after property extraction 4. **engine/src/main/java/com/arcadedb/database/LocalDatabase.java** - Added `LightEdge` import - Handle `LightEdge` by loading full edge before index deletion 5. **engine/src/test/java/com/arcadedb/graph/EdgeIndexCorruptionTest.java** - New test case that reproduces and validates the fix ## Backward Compatibility This fix is **fully backward compatible**: - No API changes - No schema changes - No data migration required - Existing databases work without modification ## Performance Considerations - **Memory**: O(N) where N is the number of deletions in a transaction (typically small) - **CPU**: O(1) HashSet operations for add/check - **Cleanup**: Automatic on transaction commit/rollback The performance impact is negligible as: 1. Most transactions don't delete many records 2. HashSet operations are very fast 3. The set is cleared after each transaction ## Future Enhancements This fix could be extended to: 1. Track all RID allocations in the transaction for rollback purposes 2. Provide transaction-level RID allocation statistics 3. Detect and warn about high RID churn within transactions (cherry picked from commit a81163a)
lvca
added a commit
that referenced
this pull request
Jun 17, 2026
…nput, MATLAB_COLUMN (#4642) * feat: [#3099] vector quantization ergonomics, asSparse(), RRF array input, MATLAB_COLUMN Follow-up to #3099 addressing the remaining actionable review items. int8 quantization ergonomics (#8/#9/#10/#11): - vectorDequantizeInt8 accepts the result of vectorQuantizeInt8() directly (single-arg form), so min/max no longer have to be unpacked by hand. The (bytes, min, max) form still works. - vectorApproxDistance infers INT8 vs BINARY from the result objects, making the type string optional. Raw byte arrays still require an explicit type. - "ranking preserved" reworded to "preserves the top-k ordering". asSparse() method (#4): the method form of vector.denseToSparse(), consistent with the asString()/asVector() conversion-method family. Delegates to the function so the dense->sparse logic stays in one place. RRF array input (#14): vectorRRFScore also accepts the ranks grouped in a single array/list (plus optional { k }), consistent with vector.multiScore. MATLAB_COLUMN format (#21): semicolon-separated column vector [1.0; 2.0; 3.0] for vector.toString()/asString(). The asVector()/string parser now also splits on semicolons so it round-trips. Adds coverage in SQLFunctionVectorEnhancementsTest. * fix: [#3099] address PR review - approxDistance mixed-type, RRF rank validation, asSparse singleton - vectorApproxDistance.inferType: detect mixed INT8/BINARY result objects and throw a clear "Cannot mix INT8 and BINARY quantization results" error instead of routing to the wrong distance and surfacing a misleading "Expected BinaryQuantizationResult" message. - vectorRRFScore: unify the variadic and array forms so both skip null ranks (item absent from that ranking list) and validate every present rank as a positive integer. The array form no longer accepts fractional ranks, and the variadic form no longer silently truncates them. - SQLMethodAsSparse: reuse a shared stateless SQLFunctionVectorDenseToSparse instance instead of allocating one per call. - SQLFunctionVectorDequantizeInt8.getSyntax(): disambiguate the two overloads. Adds tests for mixed-type inference, null-rank parity, and non-integer rank rejection. * fix: [#3099] address 2nd PR review - approxDistance inference guard, docs, test coverage - vectorApproxDistance.inferType now requires BOTH 2-arg arguments to be quantization result objects; mixing a result object with a raw array (which inference can't do reliably) is rejected with a clear message pointing at the explicit 3-arg form. - Clarify in the class Javadoc that BINARY requires the BinaryQuantizationResult (raw packed bits are not enough), while INT8 accepts a result or a raw byte array. - Tidy getSyntax() of vectorDequantizeInt8 (drop the repeated function name). - Tests: add non-zero distance assertions for both INT8 and BINARY inference, the 3-arg explicit form with result objects, and the result-object/raw-array mismatch rejection. * fix: [#3099] address 3rd PR review - getSyntax clarity, primitive-array RRF, no boxing - vectorDequantizeInt8.getSyntax(): repeat the function name on both alternatives so neither reads as a bare tuple. - vectorRRFScore: iterate primitive rank arrays (int[]/long[]/float[]/double[]) directly instead of boxing them into an Object[], per the engine's GC-awareness policy. Object[]/List still skip nulls. Validation (positive integer) is shared by both forms via rankTerm(). * fix: [#3099] vectorDequantizeInt8 3-arg form accepts the result object too The single-arg form already accepted a QuantizationResult, but the issue's literal example uses the (result, min, max) form: vectorDequantizeInt8(vectorQuantizeInt8([...]), 1.0, 3.0). The 3-arg path now extracts the bytes from a QuantizationResult as well, so that exact query works instead of throwing "Quantized vector must be an array or list, found: QuantizationResult". * fix: [#3099] address 4th PR review - dequantizeInt8 min/max correctness, RRF NaN guard, edge cases - dequantizeInt8 3-arg form with a QuantizationResult now uses the result's own authoritative min/max instead of the explicit scalars, so passing a wrong min/max alongside the result object no longer silently dequantizes with the wrong scale. Removed the now-dead QuantizationResult branch in toByteArray. - RRFScore rankTerm now guards NaN/Infinity up front with a dedicated 'must be finite' message instead of letting them fall through to the misleading 'must be integers' error. - approxDistance INT8 path gives a helpful error naming the expected INT8 inputs when a BinaryQuantizationResult is passed (mixed types in the explicit 3-arg form). - Clarified inferType comment ('each argument must be a recognized result object'). - Documented the intentional separator leniency in VectorUtils string parsing. - Documented SQLMethodAsSparse shared-instance thread-safety; asSparse Javadoc arrow nit. - Tests: result-own-min/max wins, non-finite ranks rejected, explicit-type mixed-type message, and the SQL asSparse() path now asserts the returned SparseVector value (45 tests). * test: fix phase6VectorStatistics type witness (AVG returns Double, not Float) AVG() returns a Double (issue #4518), so result.<Float>getProperty(...) on the AVG-wrapped avg_magnitude/sparsity_pct columns threw a ClassCastException. Use the matching <Double> witness. * fix: [#3099] address 5th PR review - dequantizeInt8 null-scalar ordering bug - BUG: dequantizeInt8(result, null, null) returned null instead of using the result object's embedded min/max, because the scalar null-guard fired before the QuantizationResult check. Moved the QuantizationResult early-exit above the null-guard so embedded min/max always win, even with null/wrong explicit scalars. - getSyntax() compacted to a single form (NAME + '(<result> | <quantized_bytes>, <min>, <max>)'), matching the convention used by approxDistance. - Documented why float[]/double[] rank arrays are safe (rankTerm rejects any non-integer regardless of array type, so a precision-lost float cannot slip through). - Tests: (result, null, null) uses embedded min/max; explicit 'BINARY' with INT8 result objects surfaces a CommandSQLParsingException naming BinaryQuantizationResult (not a raw ClassCastException). Replaced fully-qualified java.util names with imports (47 tests). * fix: [#3099] address 6th PR review - rrfFromArray docs/AssertionError, dequantizeInt8 syntax hint, no-space semicolon test * fix: [#3099] address 7th PR review - dequantizeInt8 syntax/Javadoc, RRF rejects float[]/double[] rank arrays, inferType readability * test: [#4643] update SEARCH_FIELDS_MORE nonExistentRID expectation to function's own "Record not found" message (missing-bucket lookup no longer throws after #4643) * fix: [#3099] address 8th PR review - document Object[] inclusion in RRF isArrayLike, skip unused separator computation for PRETTY format * fix: [#3099] address 9th PR review - dequantizeInt8 getSyntax shows both forms + clarified ignored-scalar contract, RRF short[] note, tighter binary error assertion * fix: [#3099] address 10th PR review - dequantizeInt8 rejects conflicting (non-matching) explicit min/max with a result, 2-arg comment, stateless contract on denseToSparse * docs: [#3099] address 11th PR review - explain why rankTerm uses Math.rint over a (long) cast for the integer check * fix: [#3099] address 12th PR review - tighten+document dequantize tolerance (1e-5), clarify isArrayLike/RRF syntax/denseToSparse contract comments, test annotations
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bumps json from 20201115 to 20210307.
Release notes
Sourced from json's releases.
Commits
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting
@dependabot rebase.Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
@dependabot rebasewill rebase this PR@dependabot recreatewill recreate this PR, overwriting any edits that have been made to it@dependabot mergewill merge this PR after your CI passes on it@dependabot squash and mergewill squash and merge this PR after your CI passes on it@dependabot cancel mergewill cancel a previously requested merge and block automerging@dependabot reopenwill reopen this PR if it is closed@dependabot closewill close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually@dependabot ignore this major versionwill close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)@dependabot ignore this minor versionwill close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)@dependabot ignore this dependencywill close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)