HDDS-14225. [DO NOT MERGE] Upgrade RocksDB from 7.7.3 to 10.10.1#9813
Draft
smengcl wants to merge 12 commits intoapache:masterfrom
Draft
HDDS-14225. [DO NOT MERGE] Upgrade RocksDB from 7.7.3 to 10.10.1#9813smengcl wants to merge 12 commits intoapache:masterfrom
smengcl wants to merge 12 commits intoapache:masterfrom
Conversation
Generated-by: GPT-5.3-Codex
adoroszlai
reviewed
Feb 24, 2026
Contributor
adoroszlai
left a comment
There was a problem hiding this comment.
Thanks @smengcl for working on this.
We should detect OS at runtime, not build time. Single build (on any OS) should create binaries for all supported operating systems.
-
dependencyManagementin root POM should include an entry forrocksdbjniartifact with each supported classifier, as well as the non-classified artifact:<dependency> <groupId>org.rocksdb</groupId> <artifactId>rocksdbjni</artifactId> <version>${rocksdb.version}</version> </dependency> <dependency> <groupId>org.rocksdb</groupId> <artifactId>rocksdbjni</artifactId> <version>${rocksdb.version}</version> <classifier>linux64</classifier> </dependency> <dependency> <groupId>org.rocksdb</groupId> <artifactId>rocksdbjni</artifactId> <version>${rocksdb.version}</version> <classifier>osx</classifier> </dependency> ...
-
hdds-rocks-nativeshould unpack all of these -
other modules should continue depending on platform-independent
rocksdbjni(no classifier)
Thus changes in most pom.xml files are not needed, nor is rocksdbjni.classifier.
This is a workaround for RocksDB 10.4.2 thin-jar packaging. Ensure classifier JNI artifacts are present at runtime/tests and keep dependency analysis stable. Not needed if mvnrepo rocksdbjni provides a fat jar containing native libs for all supported platforms.
…or change RocksDB 9.10.0 changed DB::KeyMayExist behavior semantics to follow its function comment. Ozone snapshot code paths were treating keyMayExist/getIfExist misses as definitive, which could misclassify existing keys in snapshot-related flows under RocksDB >= 9.10.0 and cause test failures. Treat keyMayExist/getIfExist as hints and fall back to point reads before deciding not-found: - RDBTable: verify with get()/get(ByteBuffer) on inconclusive keyMayExist - TypedTable: in codec-buffer isExist path, fallback to full get before returning false - KeyManagerImpl and ReclaimableRenameEntryFilter: use getSkipCache fallback for snapshot rename lookups RocksDB changelog for reference: https://github.com/facebook/rocksdb/releases/tag/v9.10.0 Behavior Changes DB::KeyMayExist() now follows its function comment, which means value parameter can be null, and it will be set only if value_found is passed in.
RocksDB 9.10+ may consume ByteBuffer state during keyMayExist; treat it as a hint and always use duplicated key buffers for keyMayExist/get fallback paths in RDBTable/RocksDatabase. Add a regression unit test for ByteBuffer fallback behavior. Also make replicas-test.sh restore the whole container.db directory (not just one file) to avoid stale RocksDB WAL/MANIFEST artifacts when recovering from backup.
431f2b2 to
c9b83ef
Compare
c9b83ef to
7270ac0
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Generated-by: GPT-5.3-Codex
What changes were proposed in this pull request?
Bump RocksDB version from 7.7.3 to
10.4.210.10.1. While maintaining compatibility and not breaking anything.What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-14225
How was this patch tested?