Skip to content

Commit cc4cea4

Browse files
committed
GH-44950: [C++] Bump minimum CMake version to 3.25
Try installing CMake 3.25 manually on Ubuntu Install cmake in linux-apt-lint.dockerfile and update install_cmake.sh to not require platform argument Install CMake 3.25.0 on several Ubuntu jobs and Windows jobs that require the existing version to be bumped Several fixes to install_cmake Some more fixes and upgrade minimal examples to use Ubuntu 24.04 Some more fixes for R Update from CMAKE to CMAKE_VERSION Ensure that versions returned are strings, bump cmake download Try forcing MINGW package to pull a newer CMake Revert trying to pull newer CMake Try to completely remove mingw CMake from CRAN /c/cmake/bin/cmake.exe should be present and in path, try with .exe Try with full path for CMake Try removing manual installation of CMake and installing CMake via pacman before setting up RTools Remove manual Installation of CMake from pacman not present More tries Temporarily try using msys2 GH action to install cmake Remove CMake from PKGBUILD CMake should be in the path Debug new CMake location Try with C:\Program Files\CMake\bin\cmake.exe Just testing at this point Bundle AWSSDK under RTools (#90) Check for mingw Check Windows compiler Try modifying patch due to redefinition Try fixing patch AWS is bundled on Windows R now, do not use MSYS libraries Fix license header on patch + add CMake condition for AWS patch Update configure.win to remove specific AWS CPP libs Update r/configure.win Co-authored-by: Bryce Mecum <[email protected]> Update r/configure.win Test turning ccache back on Update ci/scripts/r_windows_build.sh Co-authored-by: Neal Richardson <[email protected]> Update r/configure.win Co-authored-by: Neal Richardson <[email protected]> Review comments Remove MSYS2_ARG_CONV_EXCL Testing with manually adding missing rtools40 Add correct slash for path Add comment on why MSYS2_ARG_CONV_EXCL is required Use MSYS2_ARG_CONV_EXCL on the same CMake command Update tutorial examples to use a newer Ubuntu Add CMAKE_VERSION to cuda jobs Upgrade minimal Ubuntu to use 24.04 instead of 22.04 Fix minimal build for Ubuntu 24.04 ubuntu-cuda-cpp requires cmake argument Add cmake to docker-compose.yml missing images Add missing cmake to some more images Some more fixing mainly for R binary jobs Set CMAKE to newly installed path Revert "Try modifying patch due to redefinition" This reverts commit b772b23. Reapply "Try modifying patch due to redefinition" This reverts commit 1d71375. Hardcode enum value I tried to find a proper way to include-guard this but everything I tried didn't work. Remove outdated refs to aws libs Since we are bundling these we don't have these to copy Move _WIN32_WINNT def into {c,cxx}flags Ignore warnings-as-errors in cmake
1 parent 59c5738 commit cc4cea4

36 files changed

+516
-59
lines changed

.env

+1
Original file line numberDiff line numberDiff line change
@@ -54,6 +54,7 @@ UBUNTU=22.04
5454

5555
# Default versions for various dependencies
5656
CLANG_TOOLS=14
57+
CMAKE_VERSION=3.25.0
5758
CUDA=11.2.2
5859
DASK=latest
5960
DOTNET=8.0

.github/workflows/cpp.yml

+10
Original file line numberDiff line numberDiff line change
@@ -165,6 +165,11 @@ jobs:
165165
with:
166166
fetch-depth: 0
167167
submodules: recursive
168+
- name: Install CMake 3.25.0
169+
shell: bash
170+
run: |
171+
ci/scripts/install_cmake.sh 3.25.0 /tmp/local/
172+
echo "/tmp/local/bin" >> $GITHUB_PATH
168173
- name: Check CMake presets
169174
run: |
170175
cd cpp
@@ -324,6 +329,11 @@ jobs:
324329
with:
325330
fetch-depth: 0
326331
submodules: recursive
332+
- name: Install CMake 3.25.0
333+
shell: bash
334+
run: |
335+
ci/scripts/install_cmake.sh 3.25.0 /c/cmake
336+
echo "c:\\cmake\\bin" >> $GITHUB_PATH
327337
- name: Download Timezone Database
328338
shell: bash
329339
run: ci/scripts/download_tz_database.sh

.github/workflows/dev.yml

+5
Original file line numberDiff line numberDiff line change
@@ -53,6 +53,11 @@ jobs:
5353
uses: actions/setup-python@0b93645e9fea7318ecaed2b359559ac225c90a2b # v5.3.0
5454
with:
5555
python-version: 3.12
56+
- name: Install CMake 3.25.0
57+
shell: bash
58+
run: |
59+
ci/scripts/install_cmake.sh 3.25.0 /tmp/local/
60+
echo "/tmp/local/bin" >> $GITHUB_PATH
5661
- name: Install pre-commit
5762
run: |
5863
python -m pip install pre-commit

.github/workflows/r.yml

+6
Original file line numberDiff line numberDiff line change
@@ -282,6 +282,12 @@ jobs:
282282
restore-keys: |
283283
r-${{ matrix.config.rtools }}-ccache-mingw-${{ matrix.config.arch }}-${{ hashFiles('cpp/src/**/*.cc','cpp/src/**/*.h)') }}-
284284
r-${{ matrix.config.rtools }}-ccache-mingw-${{ matrix.config.arch }}-
285+
- name: Install CMake via MSYS2
286+
uses: msys2/setup-msys2@v2
287+
with:
288+
msystem: UCRT64
289+
update: true
290+
install: mingw-w64-ucrt-x86_64-cmake
285291
- uses: r-lib/actions/setup-r@v2
286292
with:
287293
# Note: RTools must be 40 here because RTools40 + ucrt is how we build the Arrow C++

ci/docker/centos-7-cpp.dockerfile

+5-2
Original file line numberDiff line numberDiff line change
@@ -37,7 +37,6 @@ RUN \
3737
-e 's/mirror\.centos\.org/vault.centos.org/' \
3838
/etc/yum.repos.d/CentOS-SCLo-scl*.repo && \
3939
yum install -y \
40-
cmake3 \
4140
curl \
4241
devtoolset-8 \
4342
diffutils \
@@ -49,9 +48,13 @@ RUN \
4948
wget \
5049
which
5150

51+
ARG cmake
52+
COPY ci/scripts/install_cmake.sh /arrow/ci/scripts/
53+
RUN /arrow/ci/scripts/install_cmake.sh ${cmake} /usr/local/
54+
5255
COPY ci/scripts/install_sccache.sh /arrow/ci/scripts/
5356
RUN bash /arrow/ci/scripts/install_sccache.sh unknown-linux-musl /usr/local/bin
5457

5558
ENV \
5659
ARROW_R_DEV=TRUE \
57-
CMAKE=/usr/bin/cmake3
60+
CMAKE=/usr/local/bin/cmake

ci/docker/linux-r.dockerfile

+4
Original file line numberDiff line numberDiff line change
@@ -51,6 +51,10 @@ RUN /arrow/ci/scripts/r_docker_configure.sh
5151
COPY ci/scripts/install_sccache.sh /arrow/ci/scripts/
5252
RUN /arrow/ci/scripts/install_sccache.sh unknown-linux-musl /usr/local/bin
5353

54+
ARG cmake
55+
COPY ci/scripts/install_cmake.sh /arrow/ci/scripts/
56+
RUN /arrow/ci/scripts/install_cmake.sh ${cmake} /usr/local/
57+
5458
COPY ci/scripts/r_deps.sh /arrow/ci/scripts/
5559
COPY r/DESCRIPTION /arrow/r/
5660
RUN /arrow/ci/scripts/r_deps.sh /arrow

ci/docker/python-wheel-manylinux.dockerfile

+1-1
Original file line numberDiff line numberDiff line change
@@ -53,7 +53,7 @@ ENV PATH=/opt/python/${CPYTHON_VERSION}-${CPYTHON_VERSION}/bin:${PATH}
5353
# Install CMake
5454
ARG cmake=3.29.2
5555
COPY ci/scripts/install_cmake.sh arrow/ci/scripts/
56-
RUN /arrow/ci/scripts/install_cmake.sh ${arch} linux ${cmake} /usr/local
56+
RUN /arrow/ci/scripts/install_cmake.sh ${cmake} /usr/local
5757

5858
# Install Ninja
5959
ARG ninja=1.10.2

ci/docker/ubuntu-20.04-cpp-minimal.dockerfile

+4-1
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,6 @@ RUN apt-get update -y -q && \
2727
apt-get install -y -q \
2828
build-essential \
2929
ccache \
30-
cmake \
3130
curl \
3231
gdb \
3332
git \
@@ -68,6 +67,10 @@ RUN latest_system_llvm=10 && \
6867
apt-get clean && \
6968
rm -rf /var/lib/apt/lists*
7069

70+
ARG cmake
71+
COPY ci/scripts/install_cmake.sh /arrow/ci/scripts/
72+
RUN /arrow/ci/scripts/install_cmake.sh ${cmake} /usr/local/
73+
7174
COPY ci/scripts/install_minio.sh /arrow/ci/scripts/
7275
RUN /arrow/ci/scripts/install_minio.sh latest /usr/local
7376

ci/docker/ubuntu-20.04-cpp.dockerfile

+4-1
Original file line numberDiff line numberDiff line change
@@ -67,7 +67,6 @@ RUN apt-get update -y -q && \
6767
autoconf \
6868
ca-certificates \
6969
ccache \
70-
cmake \
7170
curl \
7271
g++ \
7372
gcc \
@@ -121,6 +120,10 @@ RUN apt-get update -y -q && \
121120
apt-get clean && \
122121
rm -rf /var/lib/apt/lists*
123122

123+
ARG cmake
124+
COPY ci/scripts/install_cmake.sh /arrow/ci/scripts/
125+
RUN /arrow/ci/scripts/install_cmake.sh ${cmake} /usr/local/
126+
124127
COPY ci/scripts/install_minio.sh /arrow/ci/scripts/
125128
RUN /arrow/ci/scripts/install_minio.sh latest /usr/local
126129

ci/docker/ubuntu-22.04-cpp-minimal.dockerfile

+4
Original file line numberDiff line numberDiff line change
@@ -68,6 +68,10 @@ RUN latest_system_llvm=14 && \
6868
apt-get clean && \
6969
rm -rf /var/lib/apt/lists*
7070

71+
ARG cmake
72+
COPY ci/scripts/install_cmake.sh /arrow/ci/scripts/
73+
RUN /arrow/ci/scripts/install_cmake.sh ${cmake} /usr/local/
74+
7175
COPY ci/scripts/install_minio.sh /arrow/ci/scripts/
7276
RUN /arrow/ci/scripts/install_minio.sh latest /usr/local
7377

ci/docker/ubuntu-22.04-cpp.dockerfile

+4-1
Original file line numberDiff line numberDiff line change
@@ -71,7 +71,6 @@ RUN apt-get update -y -q && \
7171
ceph \
7272
ceph-fuse \
7373
ceph-mds \
74-
cmake \
7574
curl \
7675
gdb \
7776
git \
@@ -168,6 +167,10 @@ RUN if [ "${gcc}" = "" ]; then \
168167
# make sure zlib is cached in the EMSDK folder
169168
RUN source ~/emsdk/emsdk_env.sh && embuilder --pic build zlib
170169

170+
ARG cmake
171+
COPY ci/scripts/install_cmake.sh /arrow/ci/scripts/
172+
RUN /arrow/ci/scripts/install_cmake.sh ${cmake} /usr/local/
173+
171174
COPY ci/scripts/install_minio.sh /arrow/ci/scripts/
172175
RUN /arrow/ci/scripts/install_minio.sh latest /usr/local
173176

ci/rtools/BuildAwsCCommon.patch

+54
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
# Licensed to the Apache Software Foundation (ASF) under one
2+
# or more contributor license agreements. See the NOTICE file
3+
# distributed with this work for additional information
4+
# regarding copyright ownership. The ASF licenses this file
5+
# to you under the Apache License, Version 2.0 (the
6+
# "License"); you may not use this file except in compliance
7+
# with the License. You may obtain a copy of the License at
8+
#
9+
# http://www.apache.org/licenses/LICENSE-2.0
10+
#
11+
# Unless required by applicable law or agreed to in writing,
12+
# software distributed under the License is distributed on an
13+
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
14+
# KIND, either express or implied. See the License for the
15+
# specific language governing permissions and limitations
16+
# under the License.
17+
18+
diff --git a/cmake/AwsCFlags.cmake b/cmake/AwsCFlags.cmake
19+
index b717bca..5aa8ac9 100644
20+
--- a/cmake/AwsCFlags.cmake
21+
+++ b/cmake/AwsCFlags.cmake
22+
@@ -120,6 +120,10 @@ function(aws_set_common_properties target)
23+
list(APPEND AWS_C_FLAGS -Wno-strict-aliasing)
24+
endif()
25+
26+
+ if(CMAKE_C_IMPLICIT_LINK_LIBRARIES MATCHES "mingw32")
27+
+ list(APPEND AWS_C_FLAGS -D__USE_MINGW_ANSI_STDIO=1 -Wno-unused-local-typedefs)
28+
+ endif()
29+
+
30+
# -moutline-atomics generates code for both older load/store exclusive atomics and also
31+
# Arm's Large System Extensions (LSE) which scale substantially better on large core count systems.
32+
#
33+
diff --git a/include/aws/common/byte_order.inl b/include/aws/common/byte_order.inl
34+
index 1204be0..0abd9cb 100644
35+
--- a/include/aws/common/byte_order.inl
36+
+++ b/include/aws/common/byte_order.inl
37+
@@ -13,7 +13,7 @@
38+
# include <stdlib.h>
39+
#else
40+
# include <netinet/in.h>
41+
-#endif /* _MSC_VER */
42+
+#endif /* _WIN32 */
43+
44+
AWS_EXTERN_C_BEGIN
45+
46+
@@ -39,7 +39,7 @@ AWS_STATIC_IMPL uint64_t aws_hton64(uint64_t x) {
47+
uint64_t v;
48+
__asm__("bswap %q0" : "=r"(v) : "0"(x));
49+
return v;
50+
-#elif defined(_MSC_VER)
51+
+#elif defined(_WIN32)
52+
return _byteswap_uint64(x);
53+
#else
54+
uint32_t low = x & UINT32_MAX;

ci/rtools/aws_c_io_ep.patch

+88
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,88 @@
1+
# Licensed to the Apache Software Foundation (ASF) under one
2+
# or more contributor license agreements. See the NOTICE file
3+
# distributed with this work for additional information
4+
# regarding copyright ownership. The ASF licenses this file
5+
# to you under the Apache License, Version 2.0 (the
6+
# "License"); you may not use this file except in compliance
7+
# with the License. You may obtain a copy of the License at
8+
#
9+
# http://www.apache.org/licenses/LICENSE-2.0
10+
#
11+
# Unless required by applicable law or agreed to in writing,
12+
# software distributed under the License is distributed on an
13+
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
14+
# KIND, either express or implied. See the License for the
15+
# specific language governing permissions and limitations
16+
# under the License.
17+
18+
diff --git a/source/windows/host_resolver.c b/source/windows/host_resolver.c
19+
index 59fbb85..ad4a99e 100644
20+
--- a/source/windows/host_resolver.c
21+
+++ b/source/windows/host_resolver.c
22+
@@ -4,6 +4,7 @@
23+
*/
24+
25+
/* don't move this below the Windows.h include!!!!*/
26+
+#define _WIN32_WINNT 0x0601
27+
#include <winsock2.h>
28+
#include <ws2tcpip.h>
29+
30+
diff --git a/source/windows/iocp/iocp_event_loop.c b/source/windows/iocp/iocp_event_loop.c
31+
index 9ccce30..5cbbef7 100644
32+
--- a/source/windows/iocp/iocp_event_loop.c
33+
+++ b/source/windows/iocp/iocp_event_loop.c
34+
@@ -12,6 +12,7 @@
35+
36+
#include <aws/io/logging.h>
37+
38+
+#define _WIN32_WINNT 0x0601
39+
#include <Windows.h>
40+
41+
/* The next set of struct definitions are taken directly from the
42+
diff --git a/source/windows/secure_channel_tls_handler.c b/source/windows/secure_channel_tls_handler.c
43+
index 50caf02..29fe850 100644
44+
--- a/source/windows/secure_channel_tls_handler.c
45+
+++ b/source/windows/secure_channel_tls_handler.c
46+
@@ -2,6 +2,7 @@
47+
* Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
48+
* SPDX-License-Identifier: Apache-2.0.
49+
*/
50+
+#define _WIN32_WINNT 0x0601
51+
#define SECURITY_WIN32
52+
53+
#include <aws/io/tls_channel_handler.h>
54+
@@ -35,6 +36,25 @@
55+
# pragma warning(disable : 4306) /* Identifier is type cast to a larger pointer. */
56+
#endif
57+
58+
+#ifndef SP_PROT_TLS1_0_SERVER
59+
+#define SP_PROT_TLS1_0_SERVER SP_PROT_TLS1_SERVER
60+
+#endif
61+
+#ifndef SP_PROT_TLS1_0_CLIENT
62+
+#define SP_PROT_TLS1_0_CLIENT SP_PROT_TLS1_CLIENT
63+
+#endif
64+
+#ifndef SP_PROT_TLS1_1_SERVER
65+
+#define SP_PROT_TLS1_1_SERVER 0x00000100
66+
+#endif
67+
+#ifndef SP_PROT_TLS1_1_CLIENT
68+
+#define SP_PROT_TLS1_1_CLIENT 0x00000200
69+
+#endif
70+
+#ifndef SCH_USE_STRONG_CRYPTO
71+
+#define SCH_USE_STRONG_CRYPTO 0x00400000
72+
+#endif
73+
+#ifndef SECBUFFER_ALERT
74+
+#define SECBUFFER_ALERT 0x11
75+
+#endif
76+
+
77+
#define KB_1 1024
78+
#define READ_OUT_SIZE (16 * KB_1)
79+
#define READ_IN_SIZE READ_OUT_SIZE
80+
@@ -456,7 +476,7 @@ static int s_fillin_alpn_data(
81+
82+
*extension_length += sizeof(uint32_t) + sizeof(uint16_t);
83+
84+
- *extension_name = SecApplicationProtocolNegotiationExt_ALPN;
85+
+ *extension_name = 2;
86+
/*now add the protocols*/
87+
for (size_t i = 0; i < protocols_count; ++i) {
88+
struct aws_byte_cursor *protocol_ptr = NULL;

0 commit comments

Comments
 (0)