Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HADOOP-19475. Update Boost to 1.86.0 #7444

Open
wants to merge 2 commits into
base: trunk
Choose a base branch
from
Open

Conversation

stoty
Copy link
Contributor

@stoty stoty commented Mar 1, 2025

Description of PR

Update Boost library version to 1.86.0

How was this patch tested?

Test suite on CI

For code changes:

  • Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
  • Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
  • [] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
  • If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 43m 56s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+0 🆗 shellcheck 0m 0s Shellcheck was not available.
+0 🆗 shelldocs 0m 0s Shelldocs was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-1 ❌ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+0 🆗 mvndep 8m 3s Maven dependency ordering for branch
-1 ❌ mvninstall 37m 21s /branch-mvninstall-root.txt root in trunk failed.
-1 ❌ compile 13m 1s /branch-compile-root.txt root in trunk failed.
-1 ❌ mvnsite 5m 7s /branch-mvnsite-root.txt root in trunk failed.
+1 💚 shadedclient 103m 21s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 33s Maven dependency ordering for patch
-1 ❌ mvninstall 35m 38s /patch-mvninstall-root.txt root in the patch failed.
-1 ❌ compile 2m 48s /patch-compile-root.txt root in the patch failed.
-1 ❌ cc 2m 48s /patch-compile-root.txt root in the patch failed.
-1 ❌ javac 2m 48s /patch-compile-root.txt root in the patch failed.
+1 💚 blanks 0m 0s The patch has no blanks issues.
-1 ❌ mvnsite 4m 47s /patch-mvnsite-root.txt root in the patch failed.
+1 💚 shadedclient 43m 58s patch has no errors when building and testing our client artifacts.
_ Other Tests _
-1 ❌ unit 533m 25s /patch-unit-root.txt root in the patch failed.
+1 💚 asflicense 1m 27s The patch does not generate ASF License warnings.
767m 29s
Reason Tests
Failed junit tests hadoop.tools.TestHadoopArchiveLogs
hadoop.util.TestNativeCodeLoader
hadoop.crypto.TestCryptoCodec
Subsystem Report/Notes
Docker ClientAPI=1.48 ServerAPI=1.48 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7444/1/artifact/out/Dockerfile
GITHUB PR #7444
Optional Tests dupname asflicense codespell detsecrets mvnsite unit shellcheck shelldocs compile cc javac
uname Linux a978e95f2477 5.15.0-131-generic #141-Ubuntu SMP Fri Jan 10 21:18:28 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 4efbd14
Default Java Red Hat, Inc.-1.8.0_412-b08
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7444/1/testReport/
Max. process+thread count 3137 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs-native-client . U: .
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7444/1/console
versions git=2.9.5 maven=3.6.3
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@steveloughran
Copy link
Contributor

Afraid you'll have to work out why the build doesn't build.

Other than that though, I'm happy

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 32m 13s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+0 🆗 shellcheck 0m 0s Shellcheck was not available.
+0 🆗 shelldocs 0m 0s Shelldocs was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-1 ❌ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+0 🆗 mvndep 6m 6s Maven dependency ordering for branch
-1 ❌ mvninstall 31m 47s /branch-mvninstall-root.txt root in trunk failed.
-1 ❌ compile 11m 33s /branch-compile-root.txt root in trunk failed.
-1 ❌ mvnsite 4m 31s /branch-mvnsite-root.txt root in trunk failed.
+1 💚 shadedclient 87m 45s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 33s Maven dependency ordering for patch
-1 ❌ mvninstall 30m 13s /patch-mvninstall-root.txt root in the patch failed.
-1 ❌ compile 2m 31s /patch-compile-root.txt root in the patch failed.
-1 ❌ cc 2m 31s /patch-compile-root.txt root in the patch failed.
-1 ❌ javac 2m 31s /patch-compile-root.txt root in the patch failed.
+1 💚 blanks 0m 0s The patch has no blanks issues.
-1 ❌ mvnsite 4m 17s /patch-mvnsite-root.txt root in the patch failed.
+1 💚 shadedclient 37m 12s patch has no errors when building and testing our client artifacts.
_ Other Tests _
-1 ❌ unit 527m 30s /patch-unit-root.txt root in the patch failed.
+1 💚 asflicense 1m 28s The patch does not generate ASF License warnings.
721m 43s
Reason Tests
Failed junit tests hadoop.tools.TestHadoopArchiveLogs
hadoop.crypto.TestCryptoCodec
hadoop.util.TestNativeCodeLoader
Subsystem Report/Notes
Docker ClientAPI=1.48 ServerAPI=1.48 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7444/4/artifact/out/Dockerfile
GITHUB PR #7444
Optional Tests dupname asflicense codespell detsecrets mvnsite unit shellcheck shelldocs compile cc javac
uname Linux 1bbc7800c98a 5.15.0-131-generic #141-Ubuntu SMP Fri Jan 10 21:18:28 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 8888f65
Default Java Red Hat, Inc.-1.8.0_412-b08
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7444/4/testReport/
Max. process+thread count 3149 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs-native-client . U: .
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7444/4/console
versions git=2.9.5 maven=3.6.3
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

Copy link
Contributor

@cnauroth cnauroth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the patch, @stoty .

Line 9 of BUILDING.txt has another mention of the Boost version number in a comment. Can you please update it there too?

The last CI run showed a protoc failure in hadoop-yarn-csi:

[INFO] --- protobuf-maven-plugin:0.5.1:compile-custom (default) @ hadoop-yarn-csi ---
[INFO] Compiling 1 proto file(s) to /home/jenkins/jenkins-agent/workspace/hadoop-multibranch_PR-7444/centos-7/src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/target/generated-sources/protobuf/grpc-java
[ERROR] PROTOC FAILED: /home/jenkins/jenkins-agent/workspace/hadoop-multibranch_PR-7444/centos-7/src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/target/protoc-plugins/protoc-gen-grpc-java-1.69.0-linux-x86_64.exe: /usr/lib64/libstdc++.so.6: version `CXXABI_1.3.8' not found (required by /home/jenkins/jenkins-agent/workspace/hadoop-multibranch_PR-7444/centos-7/src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/target/protoc-plugins/protoc-gen-grpc-java-1.69.0-linux-x86_64.exe)
/home/jenkins/jenkins-agent/workspace/hadoop-multibranch_PR-7444/centos-7/src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/target/protoc-plugins/protoc-gen-grpc-java-1.69.0-linux-x86_64.exe: /usr/lib64/libstdc++.so.6: version `CXXABI_1.3.9' not found (required by /home/jenkins/jenkins-agent/workspace/hadoop-multibranch_PR-7444/centos-7/src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/target/protoc-plugins/protoc-gen-grpc-java-1.69.0-linux-x86_64.exe)
/home/jenkins/jenkins-agent/workspace/hadoop-multibranch_PR-7444/centos-7/src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/target/protoc-plugins/protoc-gen-grpc-java-1.69.0-linux-x86_64.exe: /usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by /home/jenkins/jenkins-agent/workspace/hadoop-multibranch_PR-7444/centos-7/src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/target/protoc-plugins/protoc-gen-grpc-java-1.69.0-linux-x86_64.exe)
--grpc-java_out: protoc-gen-grpc-java: Plugin failed with status code 1.

[ERROR] /home/jenkins/jenkins-agent/workspace/hadoop-multibranch_PR-7444/centos-7/src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/src/main/proto/csi.proto [0:0]: /home/jenkins/jenkins-agent/workspace/hadoop-multibranch_PR-7444/centos-7/src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/target/protoc-plugins/protoc-gen-grpc-java-1.69.0-linux-x86_64.exe: /usr/lib64/libstdc++.so.6: version `CXXABI_1.3.8' not found (required by /home/jenkins/jenkins-agent/workspace/hadoop-multibranch_PR-7444/centos-7/src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/target/protoc-plugins/protoc-gen-grpc-java-1.69.0-linux-x86_64.exe)
/home/jenkins/jenkins-agent/workspace/hadoop-multibranch_PR-7444/centos-7/src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/target/protoc-plugins/protoc-gen-grpc-java-1.69.0-linux-x86_64.exe: /usr/lib64/libstdc++.so.6: version `CXXABI_1.3.9' not found (required by /home/jenkins/jenkins-agent/workspace/hadoop-multibranch_PR-7444/centos-7/src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/target/protoc-plugins/protoc-gen-grpc-java-1.69.0-linux-x86_64.exe)
/home/jenkins/jenkins-agent/workspace/hadoop-multibranch_PR-7444/centos-7/src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/target/protoc-plugins/protoc-gen-grpc-java-1.69.0-linux-x86_64.exe: /usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by /home/jenkins/jenkins-agent/workspace/hadoop-multibranch_PR-7444/centos-7/src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/target/protoc-plugins/protoc-gen-grpc-java-1.69.0-linux-x86_64.exe)
--grpc-java_out: protoc-gen-grpc-java: Plugin failed with status code 1.

I tried your patch locally, and I couldn't reproduce this. Let's see what the next CI run does.

@stoty
Copy link
Contributor Author

stoty commented Mar 8, 2025

Thank you, done.
I agree, I can't see how the boost upgrade would affect compiling protobuf, looks more like some transient error, or pre-existing issue.

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 34s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+0 🆗 shellcheck 0m 0s Shellcheck was not available.
+0 🆗 shelldocs 0m 0s Shelldocs was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-1 ❌ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+0 🆗 mvndep 6m 17s Maven dependency ordering for branch
-1 ❌ mvninstall 32m 33s /branch-mvninstall-root.txt root in trunk failed.
-1 ❌ compile 12m 0s /branch-compile-root.txt root in trunk failed.
-1 ❌ mvnsite 4m 43s /branch-mvnsite-root.txt root in trunk failed.
+1 💚 shadedclient 89m 44s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 33s Maven dependency ordering for patch
-1 ❌ mvninstall 30m 57s /patch-mvninstall-root.txt root in the patch failed.
-1 ❌ compile 2m 43s /patch-compile-root.txt root in the patch failed.
-1 ❌ cc 2m 43s /patch-compile-root.txt root in the patch failed.
-1 ❌ javac 2m 43s /patch-compile-root.txt root in the patch failed.
+1 💚 blanks 0m 0s The patch has no blanks issues.
-1 ❌ mvnsite 4m 31s /patch-mvnsite-root.txt root in the patch failed.
+1 💚 shadedclient 38m 57s patch has no errors when building and testing our client artifacts.
_ Other Tests _
-1 ❌ unit 505m 12s /patch-unit-root.txt root in the patch failed.
+1 💚 asflicense 1m 33s The patch does not generate ASF License warnings.
672m 53s
Reason Tests
Failed junit tests hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesApps
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsCustomResourceTypes
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppCustomResourceTypes
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesCapacitySchedDefaultLabel
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesCapacitySchedDynamicConfig
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesCapacitySchedulerMixedModeAbsoluteAndPercentage
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesConfigurationMutation
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesContainers
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppAttempts
hadoop.yarn.webapp.TestRMWithCSRFFilter
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesCapacitySchedulerMixedModePercentageAndWeightVector
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesCapacitySchedulerMixedModeAbsoluteAndPercentageVector
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesCapacitySchedulerMixedModeAbsoluteAndWeight
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesCapacitySchedulerMixedModeAbsoluteAndPercentageAndWeightVector
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsModification
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesCapacitySched
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesCapacitySchedulerMixedModeAbsoluteAndPercentageAndWeight
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesReservation
hadoop.yarn.server.resourcemanager.webapp.fairscheduler.TestRMWebServicesFairSchedulerCustomResourceTypes
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServiceAppsNodelabel
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesCapacitySchedulerConfigMutation
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesCapacitySchedDynamicConfigWeightModeDQC
hadoop.yarn.server.resourcemanager.webapp.fairscheduler.TestRMWebServicesFairScheduler
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesCapacitySchedLegacyQueueCreation
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesCapacitySchedDynamicConfigWeightMode
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesSchedulerActivities
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesCapacitySchedLegacyQueueCreationAbsoluteMode
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServices
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesCapacitySchedulerMixedMode
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesCapacitySchedDynamicConfigAbsoluteMode
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesCapacitySchedulerMixedModePercentageAndWeight
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesSchedulerActivitiesWithMultiNodesEnabled
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesForCSWithPartitions
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesDelegationTokens
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodeLabels
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesCapacitySchedulerMixedModeAbsoluteAndWeightVector
hadoop.yarn.server.timeline.webapp.TestTimelineWebServices
hadoop.yarn.server.applicationhistoryservice.webapp.TestAHSWebServices
hadoop.yarn.server.nodemanager.webapp.TestNMWebServicesContainers
hadoop.yarn.server.nodemanager.webapp.TestNMWebServicesApps
hadoop.yarn.server.nodemanager.webapp.TestNMWebServices
hadoop.yarn.server.nodemanager.webapp.TestNMWebServicesAuxServices
hadoop.yarn.server.globalpolicygenerator.webapp.TestGPGWebServices
hadoop.yarn.client.cli.TestSchedConfCLI
hadoop.mapreduce.v2.hs.webapp.TestHsWebServicesJobs
hadoop.mapreduce.v2.hs.webapp.TestHsWebServicesJobConf
hadoop.mapreduce.v2.hs.webapp.TestHsWebServicesLogsExtend
hadoop.mapreduce.v2.hs.webapp.TestHsWebServicesAttempts
hadoop.mapreduce.v2.hs.webapp.TestHsWebServicesTasks
hadoop.mapreduce.v2.hs.webapp.TestHsWebServicesJobsQuery
hadoop.mapreduce.v2.hs.webapp.TestHsWebServices
hadoop.mapreduce.v2.hs.webapp.TestHsWebServicesLogs
hadoop.mapreduce.v2.app.webapp.TestAMWebServicesJobConf
hadoop.mapreduce.v2.app.webapp.TestAMWebServicesAttempt
hadoop.mapreduce.v2.app.webapp.TestAMWebServicesAttempts
hadoop.mapreduce.v2.app.webapp.TestAMWebServices
hadoop.mapreduce.v2.app.webapp.TestAMWebServicesTasks
hadoop.mapreduce.v2.app.webapp.TestAMWebServicesJobs
hadoop.resourceestimator.service.TestResourceEstimatorService
hadoop.tools.TestHadoopArchiveLogs
hadoop.crypto.TestCryptoCodec
hadoop.util.TestNativeCodeLoader
Subsystem Report/Notes
Docker ClientAPI=1.48 ServerAPI=1.48 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7444/5/artifact/out/Dockerfile
GITHUB PR #7444
Optional Tests dupname asflicense codespell detsecrets mvnsite unit shellcheck shelldocs compile cc javac
uname Linux 3b2fdf36db48 5.15.0-131-generic #141-Ubuntu SMP Fri Jan 10 21:18:28 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / a626d7c
Default Java Red Hat, Inc.-1.8.0_412-b08
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7444/5/testReport/
Max. process+thread count 3077 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs-native-client . U: .
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7444/5/console
versions git=2.9.5 maven=3.6.3
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@stoty
Copy link
Contributor Author

stoty commented Mar 8, 2025

I get the same C compiler errors in my other patch that only ever touches Java code.

@stoty
Copy link
Contributor Author

stoty commented Mar 8, 2025

If Centos 7 wasn't very EOL, I'd say that its C libraries have been updated, but that's unlikely.

Copy link
Contributor

@cnauroth cnauroth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the update.

The YARN test failures are unrelated, due to YARN-11793, which I've just reverted.

I still don't have an answer on the protoc problem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants