Skip to content

[Php-wasm] Add intl support #2173

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: trunk
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
22 changes: 20 additions & 2 deletions packages/php-wasm/compile/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -235,9 +235,25 @@ libcurl/jspi/dist/root/lib/lib/libcurl.a: base-image libz libopenssl
docker cp $$(docker create playground-php-wasm:libcurl):/root/curl-7.69.1/lib/.libs ./libcurl/jspi/dist/root/lib/lib
docker cp $$(docker create playground-php-wasm:libcurl):/root/curl-7.69.1/include/ ./libcurl/jspi/dist/root/lib

libintl_asyncify: libintl/asyncify/dist/root/lib/lib/libintl.a
libintl/asyncify/dist/root/lib/lib/libintl.a: base-image libz_asyncify
mkdir -p ./libintl/asyncify/dist/root/lib
docker build -f ./libintl/Dockerfile -t playground-php-wasm:libintl .
docker cp $$(docker create playground-php-wasm:libintl):/root/install/lib ./libintl/asyncify/dist/root/lib
docker cp $$(docker create playground-php-wasm:libintl):/root/install/include ./libintl/asyncify/dist/root/lib
docker cp $$(docker create playground-php-wasm:libintl):/root/icu/source/common ./libintl/asyncify/dist/root/lib

libintl_jspi: libintl/jspi/dist/root/lib/lib/libintl.a
libintl/jspi/dist/root/lib/lib/libintl.a: base-image libz_jspi
mkdir -p ./libintl/jspi/dist/root/lib
docker build -f ./libintl/Dockerfile -t playground-php-wasm:libintl . --build-arg JSPI=1
docker cp $$(docker create playground-php-wasm:libintl):/root/install/lib ./libintl/jspi/dist/root/lib
docker cp $$(docker create playground-php-wasm:libintl):/root/install/include ./libintl/jspi/dist/root/lib
docker cp $$(docker create playground-php-wasm:libintl):/root/icu/source/common ./libintl/jspi/dist/root/lib

all: all_jspi all_asyncify
all_jspi: libz_jspi libzip_jspi libpng16_jspi libjpeg_jspi libwebp_jspi libxml2_jspi libopenssl_jspi libsqlite3_jspi libiconv_jspi bison2.7 oniguruma_jspi libcurl_jspi
all_asyncify: libz_asyncify libzip_asyncify libpng16_asyncify libjpeg_asyncify libwebp_asyncify libxml2_asyncify libopenssl_asyncify libsqlite3_asyncify libiconv_asyncify bison2.7 oniguruma_asyncify libcurl_asyncify
all_jspi: libz_jspi libzip_jspi libpng16_jspi libjpeg_jspi libwebp_jspi libxml2_jspi libopenssl_jspi libsqlite3_jspi libiconv_jspi bison2.7 oniguruma_jspi libcurl_jspi libintl_jspi
all_asyncify: libz_asyncify libzip_asyncify libpng16_asyncify libjpeg_asyncify libwebp_asyncify libxml2_asyncify libopenssl_asyncify libsqlite3_asyncify libiconv_asyncify bison2.7 oniguruma_asyncify libcurl_asyncify libintl_asyncify

clean:
rm -rf ./libz/jspi/dist
Expand All @@ -261,3 +277,5 @@ clean:
rm -rf ./bison2.7/dist
rm -rf ./oniguruma/jspi/dist
rm -rf ./oniguruma/asyncify/dist
rm -rf ./libintl/jspi/dist
rm -rf ./libintl/asyncify/dist
18 changes: 13 additions & 5 deletions packages/php-wasm/compile/base-image/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Originally forked from https://github.com/seanmorris/php-wasm
# ubuntu:lunar supports amd64 and arm64 (Apple Silicon) while
# emscripten/emsdk:3.1.24 supports amd64 only.
FROM ubuntu:lunar as emscripten
FROM ubuntu:noble as emscripten
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm curious why do we need a Ubuntu version update?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lunar is EOL (since January 2024) so i wasn't able to just run the dockerfile and install everything, changing ubuntu version helped in that regard.


SHELL ["/bin/bash", "-c"]

Expand Down Expand Up @@ -31,21 +31,29 @@ RUN set -euxo pipefail;\
wget \
unzip \
cmake \
python3
python3 \
libicu-dev \
zlib1g-dev \
gettext \
gcc \
perl \
bzip2 \
dkms \
gcc-multilib

# Install Emscripten from the repository. We'd use the official
# Docker image, but there is no arm64 image available which makes
# the build take forever on Apple Silicon.
#
#
# ---- IMPORTANT ----
# When upgrading the Emscripten version, you will need to rebuild all the WebAssembly libraries
# shipped in this repository before you can rebuild PHP itself. The entire build, including every
# linked lib, must be produced by the same Emscripten version – otherwise you'll run into undefined
# behaviors and indeterministic failures like we did in these two PRs:
#
#
# * https://github.com/WordPress/wordpress-playground/pull/1471
# * https://github.com/WordPress/wordpress-playground/pull/1339
#
#
# WASM libraries to rebuild live here:
# https://github.com/WordPress/wordpress-playground/tree/67d916b5eccfe78e26e9c953598cc1a81f316931/packages/php-wasm/compile
RUN ln -s /usr/bin/python3 /usr/bin/python
Expand Down
8 changes: 8 additions & 0 deletions packages/php-wasm/compile/build.js
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,11 @@ const argParser = yargs(process.argv.slice(2))
choices: ['yes', 'no'],
description: 'Build with mbregex support',
},
WITH_INTL: {
type: 'string',
choices: ['yes', 'no'],
description: 'Build with intl support',
},
WITH_CLI_SAPI: {
type: 'string',
choices: ['yes', 'no'],
Expand Down Expand Up @@ -139,6 +144,7 @@ const platformDefaults = {
WITH_GD: 'yes',
WITH_MBSTRING: 'yes',
WITH_MBREGEX: 'yes',
WITH_INTL: 'yes',
WITH_OPENSSL: 'yes',
WITH_WS_NETWORKING_PROXY: 'yes',
},
Expand Down Expand Up @@ -205,6 +211,8 @@ await asyncSpawn(
'--build-arg',
getArg('WITH_MBREGEX'),
'--build-arg',
getArg('WITH_INTL'),
'--build-arg',
getArg('WITH_CLI_SAPI'),
'--build-arg',
getArg('WITH_OPENSSL'),
Expand Down
Binary file added packages/php-wasm/compile/gettext-0.21.1.tar.gz
Binary file not shown.
94 changes: 94 additions & 0 deletions packages/php-wasm/compile/libintl/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
FROM playground-php-wasm:base

ARG JSPI


ENV PKG_CONFIG_PATH=/root/install/lib/pkgconfig:$PKG_CONFIG_PATH
ENV ICU_CFLAGS="-I/root/lib/include"
ENV CFLAGS="-I/root/icu/source/common -I/root/lib/lib/icu/source/common -fPIC"
ENV CXXFLAGS="-I/root/icu/source/common -I/root/lib/lib/icu/source/common -fPIC"
ENV CPPFLAGS="-I/root/lib/include"
ENV LDFLAGS="-L/root/lib/lib -fPIC"
ENV JSPI_FLAGS=""

RUN set -eux; \
wget https://github.com/unicode-org/icu/releases/download/release-74-2/icu4c-74_2-src.tgz && \
tar -xvf icu4c-74_2-src.tgz && \
rm icu4c-74_2-src.tgz

WORKDIR /root/icu/source

RUN mkdir -p /root/icu-native-build \
&& cd /root/icu-native-build \
&& ../icu/source/configure \
--disable-shared \
--enable-static \
&& make clean \
&& make -j"$(nproc)"

RUN set -eux; \
cd /root/icu/source/config && \
cp mh-linux mh-unknown && \
sed -i 's/^CC *=.*/CC = emcc/' mh-unknown && \
sed -i 's/^CXX *=.*/CXX = em++/' mh-unknown && \
sed -i 's/^AR *=.*/AR = emar/' mh-unknown && \
sed -i 's/^RANLIB *=.*/RANLIB = emranlib/' mh-unknown && \
sed -i 's/^SHARED_FLAGS *=.*/SHARED_FLAGS =/' mh-unknown

RUN set -eux; \
cd /root/icu/source && \
source /root/emsdk/emsdk_env.sh && \
curl -L -o config.sub 'https://git.savannah.gnu.org/gitweb/?p=config.git;a=blob_plain;f=config.sub' && \
curl -L -o config.guess 'https://git.savannah.gnu.org/gitweb/?p=config.git;a=blob_plain;f=config.guess' && \
chmod +x config.sub config.guess && \
sed -i '/-linux-gnu\*)/a \
-emscripten*) os=-emscripten ;;' config.sub && \
mkdir -p /root/install && \
emconfigure ./configure \
--host=wasm32-unknown-emscripten \
--prefix=/root/install \
--disable-shared \
--enable-static \
--disable-extras \
--disable-tests \
--disable-samples \
--with-cross-build=/root/icu-native-build \
--with-data-packaging=files \
CC=emcc CXX=em++ AR=emar RANLIB=emranlib && \
emmake make clean && \
export JSPI_FLAGS=$(if [ "$JSPI" = "1" ]; then echo "-sSUPPORT_LONGJMP=wasm -fwasm-exceptions"; else echo ""; fi) && \
EMCC_SKIP="-lz" EMCC_FLAGS=" -fPIC -sSIDE_MODULE $JSPI_FLAGS" emmake make -j"$(nproc)" && \
mkdir -p /root/install/share/icu/74.2/icudt74l/{brkitr,coll,curr,lang,rbnf,region,translit,unit,zone}

RUN cd /root/icu/source && \
source /root/emsdk/emsdk_env.sh && \
emmake make install

COPY gettext-0.21.1.tar.gz /root

RUN set -eux; \
cd /tmp && \
tar -xvf /root/gettext-0.21.1.tar.gz && \
cd gettext-0.21.1/gettext-runtime && \
source /root/emsdk/emsdk_env.sh && \
emconfigure ./configure \
--host=wasm32-unknown-emscripten \
--disable-shared \
--enable-static \
--disable-java \
--disable-native-java \
--disable-csharp \
--disable-openmp \
--disable-c++ \
--disable-threads \
--without-emacs \
--without-git \
--prefix=/root/lib/ && \
export JSPI_FLAGS=$(if [ "$JSPI" = "1" ]; then echo "-sSUPPORT_LONGJMP=wasm -fwasm-exceptions"; else echo ""; fi) && \
EMCC_SKIP="-lz" EMCC_FLAGS=" -fPIC -sSIDE_MODULE $JSPI_FLAGS" emmake make -j"$(nproc)" V=1

RUN cd /tmp/gettext-0.21.1/gettext-runtime && \
source /root/emsdk/emsdk_env.sh && \
emmake make install

RUN cp /tmp/gettext-0.21.1/gettext-runtime/intl/.libs/libgnuintl.a /root/install/lib/libintl.a
Loading
Loading