Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[536] Exclude xtable-hudi-support-extensions from mvn deploy #570

Closed
wants to merge 1 commit into from

Conversation

vinishjail97
Copy link
Contributor

@vinishjail97 vinishjail97 commented Oct 28, 2024

Important Read

  • Please ensure the GitHub issue is mentioned at the beginning of the PR

What is the purpose of the pull request

We have non ASF compliant dependencies because of the bundling of dependencies (both direct and indirect) in xtable-hudi-support-extensions.

Both these bundled jars have been excluded from the release because there are indirect dependencies which are not ASF compliant in the class path, need to replace them with ASF compliant ones or remove the usage of maven-shade-plugin.

find OpenSource/incubator-xtable -type f -name "*.jar" | grep bundled
/Users/vinishreddy/OpenSource/incubator-xtable/xtable-hudi-support/xtable-hudi-support-extensions/target/xtable-hudi-support-extensions_2.12-0.2.0-SNAPSHOT-bundled.jar
/Users/vinishreddy/OpenSource/incubator-xtable/xtable-utilities/target/xtable-utilities_2.12-0.2.0-SNAPSHOT-bundled.jar

Brief change log

(for example:)

  • Exclude xtable-hudi-support-extensions from ASF release process

Verify this pull request

(Please pick either of the following options)

This pull request is a trivial rework / code cleanup without any test coverage.

@the-other-tim-brown
Copy link
Contributor

This is mainly taking dependencies on Hudi, does that mean Hudi is not compliant?

@vinishjail97
Copy link
Contributor Author

vinishjail97 commented Oct 28, 2024

This is mainly taking dependencies on Hudi, does that mean Hudi is not compliant?

I had assumed the same but there are few non ASF ones as well like org.openjdk.jol, javassist etc in the class path being added by maven-shade-plugin.

[INFO] --- shade:3.6.0:shade (default) @ xtable-hudi-support-extensions_2.12 ---
[INFO] Including org.apache.xtable:xtable-hudi-support-utils:jar:0.2.0-SNAPSHOT in the shaded jar.
[INFO] Including org.apache.logging.log4j:log4j-api:jar:2.22.0 in the shaded jar.
[INFO] Including org.apache.logging.log4j:log4j-1.2-api:jar:2.22.0 in the shaded jar.
[INFO] Including org.apache.xtable:xtable-core_2.12:jar:0.2.0-SNAPSHOT in the shaded jar.
[INFO] Including org.apache.xtable:xtable-api:jar:0.2.0-SNAPSHOT in the shaded jar.
[INFO] Including com.fasterxml.jackson.datatype:jackson-datatype-jsr310:jar:2.17.1 in the shaded jar.
[INFO] Including com.fasterxml.jackson.module:jackson-module-scala_2.12:jar:2.17.1 in the shaded jar.
[INFO] Including com.thoughtworks.paranamer:paranamer:jar:2.8 in the shaded jar.
[INFO] Including com.google.guava:guava:jar:32.1.3-jre in the shaded jar.
[INFO] Including com.google.guava:failureaccess:jar:1.0.1 in the shaded jar.
[INFO] Including com.google.guava:listenablefuture:jar:9999.0-empty-to-avoid-conflict-with-guava in the shaded jar.
[INFO] Including org.checkerframework:checker-qual:jar:3.37.0 in the shaded jar.
[INFO] Including com.google.errorprone:error_prone_annotations:jar:2.21.1 in the shaded jar.
[INFO] Including com.google.j2objc:j2objc-annotations:jar:2.8 in the shaded jar.
[INFO] Including org.scala-lang:scala-library:jar:2.12.20 in the shaded jar.
[INFO] Including org.apache.hudi:hudi-common:jar:0.14.0 in the shaded jar.
[INFO] Including org.openjdk.jol:jol-core:jar:0.16 in the shaded jar.
[INFO] Including org.apache.httpcomponents:fluent-hc:jar:4.4.1 in the shaded jar.
[INFO] Including org.apache.hbase:hbase-client:jar:2.4.9 in the shaded jar.
[INFO] Including org.apache.hbase.thirdparty:hbase-shaded-protobuf:jar:3.5.1 in the shaded jar.
[INFO] Including org.apache.hbase:hbase-common:jar:2.4.9 in the shaded jar.
[INFO] Including org.apache.hbase:hbase-logging:jar:2.4.9 in the shaded jar.
[INFO] Including org.apache.hbase.thirdparty:hbase-shaded-gson:jar:3.5.1 in the shaded jar.
[INFO] Including org.apache.hbase:hbase-hadoop-compat:jar:2.4.9 in the shaded jar.
[INFO] Including org.apache.hbase:hbase-hadoop2-compat:jar:2.4.9 in the shaded jar.
[INFO] Including javax.activation:javax.activation-api:jar:1.2.0 in the shaded jar.
[INFO] Including org.apache.hbase:hbase-protocol-shaded:jar:2.4.9 in the shaded jar.
[INFO] Including org.apache.hbase:hbase-protocol:jar:2.4.9 in the shaded jar.
[INFO] Including org.apache.hbase.thirdparty:hbase-shaded-miscellaneous:jar:3.5.1 in the shaded jar.
[INFO] Including org.apache.hbase.thirdparty:hbase-shaded-netty:jar:3.5.1 in the shaded jar.
[INFO] Including org.apache.htrace:htrace-core4:jar:4.2.0-incubating in the shaded jar.
[INFO] Including org.jruby.jcodings:jcodings:jar:1.0.55 in the shaded jar.
[INFO] Including org.jruby.joni:joni:jar:2.1.31 in the shaded jar.
[INFO] Including org.apache.yetus:audience-annotations:jar:0.5.0 in the shaded jar.
[INFO] Including org.apache.hbase:hbase-server:jar:2.4.9 in the shaded jar.
[INFO] Including org.apache.hbase:hbase-http:jar:2.4.9 in the shaded jar.
[INFO] Including org.apache.hbase.thirdparty:hbase-shaded-jetty:jar:3.5.1 in the shaded jar.
[INFO] Including org.apache.hbase.thirdparty:hbase-shaded-jersey:jar:3.5.1 in the shaded jar.
[INFO] Including javax.ws.rs:javax.ws.rs-api:jar:2.1.1 in the shaded jar.
[INFO] Including org.apache.hbase:hbase-procedure:jar:2.4.9 in the shaded jar.
[INFO] Including org.apache.hbase:hbase-zookeeper:jar:2.4.9 in the shaded jar.
[INFO] Including org.apache.hbase:hbase-replication:jar:2.4.9 in the shaded jar.
[INFO] Including org.apache.hbase:hbase-metrics-api:jar:2.4.9 in the shaded jar.
[INFO] Including org.apache.hbase:hbase-metrics:jar:2.4.9 in the shaded jar.
[INFO] Including org.apache.hbase:hbase-asyncfs:jar:2.4.9 in the shaded jar.
[INFO] Including org.glassfish.web:javax.servlet.jsp:jar:2.3.2 in the shaded jar.
[INFO] Including org.glassfish:javax.el:jar:3.0.0 in the shaded jar.
[INFO] Including javax.servlet.jsp:javax.servlet.jsp-api:jar:2.3.1 in the shaded jar.
[INFO] Including org.jamon:jamon-runtime:jar:2.4.1 in the shaded jar.
[INFO] Including com.lmax:disruptor:jar:3.4.2 in the shaded jar.
[INFO] Including org.apache.iceberg:iceberg-api:jar:1.4.2 in the shaded jar.
[INFO] Including io.delta:delta-core_2.12:jar:2.4.0 in the shaded jar.
[INFO] Including io.delta:delta-storage:jar:2.4.0 in the shaded jar.
[INFO] Including org.antlr:antlr4-runtime:jar:4.9.3 in the shaded jar.
[INFO] Including org.slf4j:slf4j-api:jar:2.0.9 in the shaded jar.
[INFO] Including org.apache.orc:orc-core:jar:1.6.0 in the shaded jar.
[INFO] Including org.apache.orc:orc-shims:jar:1.6.0 in the shaded jar.
[INFO] Including commons-lang:commons-lang:jar:2.6 in the shaded jar.
[INFO] Including io.airlift:aircompressor:jar:0.15 in the shaded jar.
[INFO] Including javax.xml.bind:jaxb-api:jar:2.2.11 in the shaded jar.
[INFO] Including org.jetbrains:annotations:jar:17.0.0 in the shaded jar.
[INFO] Including io.dropwizard.metrics:metrics-core:jar:4.1.1 in the shaded jar.
[INFO] Including org.apache.commons:commons-math3:jar:3.6.1 in the shaded jar.
[INFO] Including org.apache.httpcomponents:httpclient:jar:4.5.13 in the shaded jar.
[INFO] Including org.apache.httpcomponents:httpcore:jar:4.4.13 in the shaded jar.
[INFO] Including commons-logging:commons-logging:jar:1.2 in the shaded jar.
[INFO] Including commons-codec:commons-codec:jar:1.15 in the shaded jar.
[INFO] Including commons-io:commons-io:jar:2.14.0 in the shaded jar.
[INFO] Including org.apache.commons:commons-lang3:jar:3.12.0 in the shaded jar.
[INFO] Including com.google.code.findbugs:jsr305:jar:3.0.2 in the shaded jar.
[INFO] Including com.fasterxml.jackson.core:jackson-annotations:jar:2.17.1 in the shaded jar.
[INFO] Including javax.annotation:javax.annotation-api:jar:1.3.2 in the shaded jar.
[INFO] Including org.apache.iceberg:iceberg-core:jar:1.4.2 in the shaded jar.
[INFO] Including org.apache.iceberg:iceberg-common:jar:1.4.2 in the shaded jar.
[INFO] Including org.apache.iceberg:iceberg-bundled-guava:jar:1.4.2 in the shaded jar.
[INFO] Including com.github.ben-manes.caffeine:caffeine:jar:2.9.3 in the shaded jar.
[INFO] Including org.roaringbitmap:RoaringBitmap:jar:0.9.47 in the shaded jar.
[INFO] Including org.roaringbitmap:shims:jar:0.9.47 in the shaded jar.
[INFO] Including org.apache.iceberg:iceberg-parquet:jar:1.4.2 in the shaded jar.
[INFO] Including org.lz4:lz4-java:jar:1.8.0 in the shaded jar.
[INFO] Including jakarta.ws.rs:jakarta.ws.rs-api:jar:2.1.6 in the shaded jar.
[INFO] Including org.glassfish.hk2.external:jakarta.inject:jar:2.6.1 in the shaded jar.
[INFO] Including jakarta.annotation:jakarta.annotation-api:jar:1.3.5 in the shaded jar.
[INFO] Including jakarta.validation:jakarta.validation-api:jar:2.0.2 in the shaded jar.
[INFO] Including org.javassist:javassist:jar:3.25.0-GA in the shaded jar.
[INFO] Including org.apache.commons:commons-crypto:jar:1.1.0 in the shaded jar.
[INFO] Including org.rocksdb:rocksdbjni:jar:7.9.2 in the shaded jar.
[INFO] Including org.apache.hive:hive-storage-api:jar:2.8.1 in the shaded jar.
[INFO] Including com.google.protobuf:protobuf-java:jar:3.25.5 in the shaded jar.
 

@vinishjail97
Copy link
Contributor Author

Closing this pull request, found a solution.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants