YARN-11794. (3.4) Track gRPC generated code for hadoop-yarn-csi

[pan3793]

Description of PR

An alternative to #7499 to recover the CentOS 7 build.

This is only for branch-3.4, we should consider removing the support of CentOS 7 for trunk.

How was this patch tested?

The verification step requires manually patching HADOOP-19540 and HADOOP-19246 on branch-3.4 first.

Before

$ git checkout branch-3.4
$ git checkout trunk -- start-build-env.sh
$ git checkout trunk -- dev-support/docker/pkg-resolver/install-yasm.sh
$ ./start-build-env.sh centos_7
$ mvn clean install -DskipTests -pl :hadoop-yarn-csi -am
...
[ERROR] PROTOC FAILED: /home/chengpan/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/target/protoc-plugins/protoc-gen-grpc-java-1.69.0-linux-x86_64.exe: /usr/lib64/libstdc++.so.6: version `CXXABI_1.3.8' not found (required by /home/chengpan/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/target/protoc-plugins/protoc-gen-grpc-java-1.69.0-linux-x86_64.exe)
/home/chengpan/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/target/protoc-plugins/protoc-gen-grpc-java-1.69.0-linux-x86_64.exe: /usr/lib64/libstdc++.so.6: version `CXXABI_1.3.9' not found (required by /home/chengpan/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/target/protoc-plugins/protoc-gen-grpc-java-1.69.0-linux-x86_64.exe)
/home/chengpan/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/target/protoc-plugins/protoc-gen-grpc-java-1.69.0-linux-x86_64.exe: /usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by /home/chengpan/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/target/protoc-plugins/protoc-gen-grpc-java-1.69.0-linux-x86_64.exe)
--grpc-java_out: protoc-gen-grpc-java: Plugin failed with status code 1.
...
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  43.582 s
[INFO] Finished at: 2025-07-10T07:54:08Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.xolstice.maven.plugins:protobuf-maven-plugin:0.5.1:compile-custom (default) on project hadoop-yarn-csi: protoc did not exit cleanly. Review output for more information. -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <args> -rf :hadoop-yarn-csi

After

$ git checkout YARN-11794-3.4
$ git checkout trunk -- start-build-env.sh
$ git checkout trunk -- dev-support/docker/pkg-resolver/install-yasm.sh
$ ./start-build-env.sh centos_7
$ mvn clean install -DskipTests -pl :hadoop-yarn-csi -am
...
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  48.896 s
[INFO] Finished at: 2025-07-10T07:42:12Z
[INFO] ------------------------------------------------------------------------

For code changes:

• Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
• Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
• If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
• If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

[pan3793]

@GauthamBanasandra Seems all current Dockerfile - Ubuntu 20.4, Debian 10, CentOS 7/8, have reached EOL, and the oldest CentOS 7 starts to cause building failures, do you have suggestion on that? Should we upgrade them to the supported versions, on both trunk and branch-3.4? Or we must ensure that branch-3.4 supports all those OS even they are EOL?

and there was a previous discussion in mailing list related to this https://lists.apache.org/thread/yj5lx4lmrflyqyfvwmjt74y3jcl65tk0

[hadoop-yetus]

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	7m 7s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	detsecrets	0m 1s		detect-secrets was not available.
+0 🆗	xmllint	0m 1s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ branch-3.4 Compile Tests _
+1 💚	mvninstall	22m 2s		branch-3.4 passed
+1 💚	compile	0m 19s		branch-3.4 passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚	compile	0m 18s		branch-3.4 passed with JDK Private Build-1.8.0_452-8u452-gaus1-0ubuntu120.04-b09
+1 💚	checkstyle	0m 17s		branch-3.4 passed
+1 💚	mvnsite	0m 23s		branch-3.4 passed
+1 💚	javadoc	0m 20s		branch-3.4 passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	0m 17s		branch-3.4 passed with JDK Private Build-1.8.0_452-8u452-gaus1-0ubuntu120.04-b09
+1 💚	spotbugs	0m 43s		branch-3.4 passed
+1 💚	shadedclient	19m 16s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 15s		the patch passed
+1 💚	compile	0m 15s		the patch passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚	javac	0m 15s		the patch passed
+1 💚	compile	0m 14s		the patch passed with JDK Private Build-1.8.0_452-8u452-gaus1-0ubuntu120.04-b09
+1 💚	javac	0m 14s		the patch passed
-1 ❌	blanks	0m 0s	/blanks-eol.txt	The patch has 318 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
+1 💚	checkstyle	0m 9s		the patch passed
+1 💚	mvnsite	0m 17s		the patch passed
+1 💚	javadoc	0m 13s		the patch passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	0m 13s		the patch passed with JDK Private Build-1.8.0_452-8u452-gaus1-0ubuntu120.04-b09
+1 💚	spotbugs	0m 40s		the patch passed
+1 💚	shadedclient	19m 0s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	0m 28s		hadoop-yarn-csi in the patch passed.
+1 💚	asflicense	0m 25s		The patch does not generate ASF License warnings.
		74m 32s

Subsystem	Report/Notes
Docker	ClientAPI=1.51 ServerAPI=1.51 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7790/1/artifact/out/Dockerfile
GITHUB PR	#7790
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint spotbugs checkstyle
uname	Linux 1a5c43a62cb2 5.15.0-143-generic #153-Ubuntu SMP Fri Jun 13 19:10:45 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	branch-3.4 / d29e84b
Default Java	Private Build-1.8.0_452-8u452-gaus1-0ubuntu120.04-b09
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_452-8u452-gaus1-0ubuntu120.04-b09
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7790/1/testReport/
Max. process+thread count	564 (vs. ulimit of 5500)
modules	C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7790/1/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

[GauthamBanasandra]

@pan3793 I think we should move away from the version of the distros that have reached their EOL. We can upgrade to the latest version of that distro.

Also, retire the distro where Hadoop isn't buildable anymore. No use keeping it. Just remove the relevant code in dev-support/Jenkinsfile, with a note.