Skip to content

HDDS-14778. Fix maven-shade-plugin relocation rules to require package separator dot#9893

Open
jasonosullivan34 wants to merge 1 commit intoapache:masterfrom
jasonosullivan34:HDDS-14778-update-relocation-pattern
Open

HDDS-14778. Fix maven-shade-plugin relocation rules to require package separator dot#9893
jasonosullivan34 wants to merge 1 commit intoapache:masterfrom
jasonosullivan34:HDDS-14778-update-relocation-pattern

Conversation

@jasonosullivan34
Copy link

@jasonosullivan34 jasonosullivan34 commented Mar 10, 2026

What changes were proposed in this pull request?

The ozone-filesystem-shaded module's maven-shade-plugin contained relocation rules using bare keyword patterns without a trailing dot — specifically io, kotlin, and info. Because the shade plugin does prefix string replacement on JVM constant pool entries, these rules incorrectly matched any string constant starting with those characters — not just genuine package references.

The fix adds a trailing dot to both and for each affected rule, e.g.:
<pattern>io.</pattern> <shadedPattern>${ozone.shaded.prefix}.io.</shadedPattern>

This restricts matching to genuine package references (io.grpc, io.netty etc.) and prevents false positive matches against binary protobuf descriptor data, Netty system property names, and Kotlin compiler metadata strings.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-14778

How was this patch tested?

Binary analysis of ozone-filesystem-shaded-2.0.0.jar confirms the bug was actively corrupting 13 class files in the released 2.0.0 artifact, including OzoneManagerProtocolProtos and StorageContainerLocationProtocolProtos, causing ExceptionInInitializerError / InvalidProtocolBufferException: Protocol message tag had invalid wire type at class load time.

The bug does not currently manifest in master (2.2.0-SNAPSHOT) due to a coincidental change in how the protobuf codegen version structures descriptor chunks — the root cause is unchanged and remains latent.

A verification script check-shading.py is included that scans all class files in the shaded JAR for corrupted constant pool entries:
python3 check-shading.py

@adoroszlai

@adoroszlai adoroszlai marked this pull request as ready for review March 10, 2026 20:45
@adoroszlai
Copy link
Contributor

@jasonosullivan34 Thanks a lot for working on this. The verification script is especially useful, since the problem is hard to reproduce consistently. The approach of the fix looks good.

I noticed some failures in MapReduce jobs (acceptance (MR)). They appear to be "random", as slightly different ones failed in your fork and in the PR run. We don't see these failures without the patch. Can you please check? YARN application logs are not saved, so you may need to run the test locally (e.g. hadoop-ozone/dist/target/ozone-2.2.0-SNAPSHOT/compose/ozone/test-hadoop.sh). You will need to apply #9906 to fix a bug in that script.

(Failures in s3a and tools appear to be unrelated.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants