Skip to content

Conversation

jbonofre
Copy link
Member

@jbonofre jbonofre commented Oct 2, 2025

No description provided.

@jbonofre jbonofre changed the title Upgrade to Apache POM 35 and identify fixes needed to have CI happy MINOR: Upgrade to Apache POM 35 and identify fixes needed to have CI happy Oct 2, 2025
@jbonofre
Copy link
Member Author

jbonofre commented Oct 2, 2025

For the JNI Ubuntu, I see this message from archery:

    × python setup.py egg_info did not run successfully.
    │ exit code: 1
    ╰─> [1 lines of output]
        Python < 3.10 is not supported
        [end of output]

Checking for an update.

@lidavidm
Copy link
Member

lidavidm commented Oct 2, 2025

@jbonofre
Copy link
Member Author

jbonofre commented Oct 2, 2025

Yup agree. It's what I gonna try.

@raulcd
Copy link
Member

raulcd commented Oct 7, 2025

The error with cffi not being available for free-threaded means it's trying to use free-threaded wheels (an python) which is unexpected from what I understand:

        RuntimeError: CFFI does not support the free-threaded build of CPython 3.13. Upgrade to free-threaded 3.14 or newer to use CFFI with the free-threaded build.

This comment has been minimized.

@jbonofre jbonofre added the dependencies Pull requests that update a dependency file label Oct 8, 2025
@github-actions github-actions bot added this to the 18.4.0 milestone Oct 8, 2025
@jbonofre jbonofre mentioned this pull request Oct 10, 2025
@kou
Copy link
Member

kou commented Oct 10, 2025

We need apache/arrow#47749 and

diff --git a/ci/scripts/jni_macos_build.sh b/ci/scripts/jni_macos_build.sh
index f7543b6f..7f927d19 100755
--- a/ci/scripts/jni_macos_build.sh
+++ b/ci/scripts/jni_macos_build.sh
@@ -142,6 +142,7 @@ github_actions_group_begin "Checking shared dependencies for libraries"
 pushd "${dist_dir}"
 archery linking check-dependencies \
   --allow CoreFoundation \
+  --allow Network \
   --allow Security \
   --allow libSystem \
   --allow libarrow_cdata_jni \

for macOS.

@kou
Copy link
Member

kou commented Oct 10, 2025

We need apache/arrow#47746 and

diff --git a/.env b/.env
index d3e1c1d6..a7783537 100644
--- a/.env
+++ b/.env
@@ -40,7 +40,7 @@ ARCH_SHORT=amd64
 
 # Default repository to pull and push images from
 REPO=ghcr.io/apache/arrow-java-dev
-ARROW_REPO=apache/arrow-dev
+ARROW_REPO=ghcr.io/apache/arrow-dev
 
 # The setup attempts to generate coredumps by default, in order to disable the
 # coredump generation set it to 0
@@ -53,5 +53,4 @@ MAVEN=3.9.9
 # Versions for various dependencies used to build artifacts
 # Keep in sync with apache/arrow
 ARROW_REPO_ROOT=./arrow
-PYTHON=3.9
-VCPKG="f7423ee180c4b7f40d43402c2feb3859161ef625" # 2024.06.15 Release
+VCPKG="4334d8b4c8916018600212ab4dd4bbdc343065d1"    # 2025.09.17 Release
diff --git a/ci/docker/vcpkg-jni.dockerfile b/ci/docker/vcpkg-jni.dockerfile
index 55fa35e0..d6bd322a 100644
--- a/ci/docker/vcpkg-jni.dockerfile
+++ b/ci/docker/vcpkg-jni.dockerfile
@@ -18,24 +18,10 @@
 ARG base
 FROM ${base}
 
-# Install the libraries required by Gandiva to run
-# Use enable llvm[enable-rtti] in the vcpkg.json to avoid link problems in Gandiva
-RUN vcpkg install \
-        --clean-after-build \
-        --x-install-root=${VCPKG_ROOT}/installed \
-        --x-manifest-root=/arrow/ci/vcpkg \
-        --x-feature=dev \
-        --x-feature=flight \
-        --x-feature=gcs \
-        --x-feature=json \
-        --x-feature=parquet \
-        --x-feature=gandiva \
-        --x-feature=s3
-
 # Install Java
 # We need Java for JNI headers, but we don't invoke Maven in this build.
 ARG java=11
-RUN yum install -y java-$java-openjdk-devel && yum clean all
+RUN dnf install -y java-$java-openjdk-devel && dnf clean all
 
 # For ci/scripts/{cpp,java}_*.sh
 ENV ARROW_HOME=/tmp/local \
diff --git a/compose.yaml b/compose.yaml
index b125c3c9..f5082a22 100644
--- a/compose.yaml
+++ b/compose.yaml
@@ -99,7 +99,7 @@ services:
       cache_from:
         - ${REPO}:${ARCH}-vcpkg-jni-${VCPKG}
       args:
-        base: ${ARROW_REPO}:${ARCH}-python-${PYTHON}-wheel-manylinux-2014-vcpkg-${VCPKG}
+        base: ${ARROW_REPO}:${ARCH}-cpp-jni-${VCPKG}
     volumes:
       - .:/arrow-java:delegated
       - ${ARROW_REPO_ROOT}:/arrow:delegated

for Linux.

@jbonofre
Copy link
Member Author

I'm fixing the base image here (need to adapt).

@kou
Copy link
Member

kou commented Oct 10, 2025

apache/arrow#47749 was merged and JNI jobs on macOS passed.

@jbonofre
Copy link
Member Author

@kou macos works now thanks. But not Ubuntu yet. I'm investigating.

@kou
Copy link
Member

kou commented Oct 11, 2025

We need apache/arrow#47746 and

It's not merged yet. It's a blocker for Linux.

@jbonofre
Copy link
Member Author

As apache/arrow#47746 has been merged, I did a rebase and trigger a new run. Let's see.

@kou
Copy link
Member

kou commented Oct 14, 2025

Could you try this?

diff --git a/ci/scripts/jni_manylinux_build.sh b/ci/scripts/jni_manylinux_build.sh
index a34ec0f4..d3a873e3 100755
--- a/ci/scripts/jni_manylinux_build.sh
+++ b/ci/scripts/jni_manylinux_build.sh
@@ -53,8 +53,8 @@ if [ "${ARROW_USE_CCACHE}" == "ON" ]; then
 fi
 
 github_actions_group_begin "Building Arrow C++ libraries"
-devtoolset_version="$(rpm -qa "devtoolset-*-gcc" --queryformat '%{VERSION}' | grep -o "^[0-9]*")"
-devtoolset_include_cpp="/opt/rh/devtoolset-${devtoolset_version}/root/usr/include/c++/${devtoolset_version}"
+gcc_toolset_version="$(rpm -qa "gcc-toolset-*-gcc" --queryformat '%{VERSION}' | grep -o "^[0-9]*")"
+gcc_toolset_include_cpp="/opt/rh/gcc-toolset-${gcc_toolset_version}/root/usr/include/c++/${gcc_toolset_version}"
 : "${ARROW_ACERO:=ON}"
 export ARROW_ACERO
 : "${ARROW_BUILD_TESTS:=OFF}"
@@ -76,7 +76,7 @@ export ARROW_ORC
 : "${VCPKG_ROOT:=/opt/vcpkg}"
 : "${VCPKG_FEATURE_FLAGS:=-manifests}"
 : "${VCPKG_TARGET_TRIPLET:=${VCPKG_DEFAULT_TRIPLET:-x64-linux-static-${CMAKE_BUILD_TYPE}}}"
-: "${GANDIVA_CXX_FLAGS:=-isystem;${devtoolset_include_cpp};-isystem;${devtoolset_include_cpp}/x86_64-redhat-linux;-lpthread}"
+: "${GANDIVA_CXX_FLAGS:=-isystem;${gcc_toolset_include_cpp};-isystem;${gcc_toolset_include_cpp}/x86_64-redhat-linux;-lpthread}"
 
 export ARROW_TEST_DATA="${arrow_dir}/testing/data"
 export PARQUET_TEST_DATA="${arrow_dir}/cpp/submodules/parquet-testing/data"

@kou
Copy link
Member

kou commented Oct 15, 2025

Pushed the change to this branch.

@kou
Copy link
Member

kou commented Oct 15, 2025

JNI on Linux is ready.

We need to update windows-2019 image for JNI. windows-2019 image was removed.

@jbonofre
Copy link
Member Author

jbonofre commented Oct 15, 2025

I updated to use windows-2022 (instead of windows-2019).

@jbonofre
Copy link
Member Author

Windows 2022 JNI fails with:

  CMake Error at src/arrow/CMakeLists.txt:580 (target_link_libraries):
    Target "arrow_util_static" links to:
  
      OpenSSL::Crypto
  
    but the target was not found. 

I'm investigating to fix that.

@jbonofre
Copy link
Member Author

@kou do we still to use windows-2019 for JNI ? I tried to update to windows-2022 but JNI fails (I'm investigating, see my previous comment).

@jbonofre
Copy link
Member Author

I see the lib defined in Arrow CPP (set(ARROW_OPENSSL_LIBS OpenSSL::Crypto OpenSSL::SSL)).
But OpenSSL Crypto doesn't seem to be available.

@kou
Copy link
Member

kou commented Oct 17, 2025

We need to use Visual Studio 2022 not 2019 on windows-2022.
I've pushed the change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dependencies Pull requests that update a dependency file

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants