HDDS-14778. Fix maven-shade-plugin relocation rules to require package separator dot#9893
Open
jasonosullivan34 wants to merge 1 commit intoapache:masterfrom
Open
Conversation
Contributor
|
@jasonosullivan34 Thanks a lot for working on this. The verification script is especially useful, since the problem is hard to reproduce consistently. The approach of the fix looks good. I noticed some failures in MapReduce jobs ( (Failures in |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
The ozone-filesystem-shaded module's maven-shade-plugin contained relocation rules using bare keyword patterns without a trailing dot — specifically io, kotlin, and info. Because the shade plugin does prefix string replacement on JVM constant pool entries, these rules incorrectly matched any string constant starting with those characters — not just genuine package references.
The fix adds a trailing dot to both and for each affected rule, e.g.:
<pattern>io.</pattern> <shadedPattern>${ozone.shaded.prefix}.io.</shadedPattern>This restricts matching to genuine package references (io.grpc, io.netty etc.) and prevents false positive matches against binary protobuf descriptor data, Netty system property names, and Kotlin compiler metadata strings.
What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-14778
How was this patch tested?
Binary analysis of ozone-filesystem-shaded-2.0.0.jar confirms the bug was actively corrupting 13 class files in the released 2.0.0 artifact, including OzoneManagerProtocolProtos and StorageContainerLocationProtocolProtos, causing ExceptionInInitializerError / InvalidProtocolBufferException: Protocol message tag had invalid wire type at class load time.
The bug does not currently manifest in master (2.2.0-SNAPSHOT) due to a coincidental change in how the protobuf codegen version structures descriptor chunks — the root cause is unchanged and remains latent.
A verification script check-shading.py is included that scans all class files in the shaded JAR for corrupted constant pool entries:
python3 check-shading.py@adoroszlai