Skip to content

Commit fe2ff12

Browse files
nicktatefacebook-github-bot
nicktate
authored and
facebook-github-bot
committed
Docker Testing Environment for Android & JS
Summary: Created a containerized environment to run unit and integration tests for both javascript and android as well as a Jenkinsfile using the new 2.0 Pipeline syntax for integration into a Jenkins CI cluster. Here is a quick summary of the changes: * The android image is built from two separate dockerfiles. There is a base image that handles the heavy lifting of dependencies that are infrequently changed while the secondary image extends the base and allows for much quicker incremental builds on code updates. * The javascript image is simple and is relatively quick to build, therefore there is no base image for any react specific javascript dependencies and it is all packaged in a single docker image. * A new `scripts/docker` has been created including some javascript files and shell scripts to aid in the running of the tests * The instrumentation test runner script can be passed various flags to control which tests run since the entire suite takes a significant amount of time to run synchronously * Jen Closes facebook#11902 Differential Revision: D4609238 Pulled By: ericvicenti fbshipit-source-id: a317f3ac3be898180b009254a9604ca7d579a8b9
1 parent 387ec8c commit fe2ff12

11 files changed

+961
-1
lines changed

‎ContainerShip/Dockerfile.android‎

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
FROM containership/android-base:latest
2+
3+
# set default environment variables
4+
ENV GRADLE_OPTS="-Dorg.gradle.jvmargs=\"-Xmx512m -XX:+HeapDumpOnOutOfMemoryError\""
5+
ENV JAVA_TOOL_OPTIONS="-Dfile.encoding=UTF8"
6+
ENV REACT_NATIVE_MAX_WORKERS=1
7+
8+
# add ReactAndroid directory
9+
ADD .buckconfig /app/.buckconfig
10+
ADD .buckjavaargs /app/.buckjavaargs
11+
ADD ReactAndroid /app/ReactAndroid
12+
ADD ReactCommon /app/ReactCommon
13+
ADD keystores /app/keystores
14+
15+
# set workdir
16+
WORKDIR /app
17+
18+
# run buck fetches
19+
RUN buck fetch ReactAndroid/src/test/java/com/facebook/react/modules
20+
RUN buck fetch ReactAndroid/src/main/java/com/facebook/react
21+
RUN buck fetch ReactAndroid/src/main/java/com/facebook/react/shell
22+
RUN buck fetch ReactAndroid/src/test/...
23+
RUN buck fetch ReactAndroid/src/androidTest/...
24+
25+
# build app
26+
RUN buck build ReactAndroid/src/main/java/com/facebook/react
27+
RUN buck build ReactAndroid/src/main/java/com/facebook/react/shell
28+
29+
ADD gradle /app/gradle
30+
ADD gradlew /app/gradlew
31+
ADD settings.gradle /app/settings.gradle
32+
ADD build.gradle /app/build.gradle
33+
ADD react.gradle /app/react.gradle
34+
35+
# run gradle downloads
36+
RUN ./gradlew :ReactAndroid:downloadBoost :ReactAndroid:downloadDoubleConversion :ReactAndroid:downloadFolly :ReactAndroid:downloadGlog
37+
38+
# compile native libs with Gradle script, we need bridge for unit and integration tests
39+
RUN ./gradlew :ReactAndroid:packageReactNdkLibsForBuck -Pjobs=1 -Pcom.android.build.threadPoolSize=1
40+
41+
# add all react-native code
42+
ADD . /app
43+
WORKDIR /app
44+
45+
# https://github.com/npm/npm/issues/13306
46+
RUN cd $(npm root -g)/npm && npm install fs-extra && sed -i -e s/graceful-fs/fs-extra/ -e s/fs.rename/fs.move/ ./lib/utils/rename.js
47+
48+
# build node dependencies
49+
RUN npm install
50+
RUN npm install github@0.2.4
51+
52+
WORKDIR /app/website
53+
RUN npm install
54+
55+
WORKDIR /app

‎ContainerShip/Dockerfile.android-base‎

Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,78 @@
1+
FROM library/ubuntu:16.04
2+
3+
# set default build arguments
4+
ARG ANDROID_VERSION=25.2.3
5+
ARG BUCK_VERSION=2016.11.11.01
6+
ARG NDK_VERSION=10e
7+
ARG NODE_VERSION=6.2.0
8+
ARG WATCHMAN_VERSION=4.7.0
9+
10+
# set default environment variables
11+
ENV ADB_INSTALL_TIMEOUT=10
12+
ENV PATH=${PATH}:/opt/buck/bin/
13+
ENV ANDROID_HOME=/opt/android
14+
ENV ANDROID_SDK_HOME=${ANDROID_HOME}
15+
ENV PATH=${PATH}:${ANDROID_HOME}/tools:${ANDROID_HOME}/platform-tools
16+
ENV ANDROID_NDK=/opt/ndk/android-ndk-r$NDK_VERSION
17+
ENV PATH=${PATH}:${ANDROID_NDK}
18+
19+
# install system dependencies
20+
RUN apt-get update && apt-get install ant autoconf automake curl g++ gcc git libqt5widgets5 lib32z1 lib32stdc++6 make maven npm openjdk-8* python-dev python3-dev qml-module-qtquick-controls qtdeclarative5-dev unzip -y
21+
22+
# configure npm
23+
RUN npm config set spin=false
24+
RUN npm config set progress=false
25+
26+
# install node
27+
RUN npm install n -g
28+
RUN n $NODE_VERSION
29+
30+
# download buck
31+
RUN git clone https://github.com/facebook/buck.git /opt/buck
32+
WORKDIR /opt/buck
33+
RUN git checkout v$BUCK_VERSION
34+
35+
# build buck
36+
RUN ant
37+
38+
# download watchman
39+
RUN git clone https://github.com/facebook/watchman.git /opt/watchman
40+
WORKDIR /opt/watchman
41+
RUN git checkout v$WATCHMAN_VERSION
42+
43+
# build watchman
44+
RUN ./autogen.sh
45+
RUN ./configure
46+
RUN make
47+
RUN make install
48+
49+
# download and unpack android
50+
RUN mkdir /opt/android
51+
WORKDIR /opt/android
52+
RUN curl --silent https://dl.google.com/android/repository/tools_r$ANDROID_VERSION-linux.zip > android.zip
53+
RUN unzip android.zip
54+
RUN rm android.zip
55+
56+
# download and unpack NDK
57+
RUN mkdir /opt/ndk
58+
WORKDIR /opt/ndk
59+
RUN curl --silent https://dl.google.com/android/repository/android-ndk-r$NDK_VERSION-linux-x86_64.zip > ndk.zip
60+
RUN unzip ndk.zip
61+
62+
# cleanup NDK
63+
RUN rm ndk.zip
64+
65+
# add android SDK tools
66+
# 2 - Android SDK Platform-tools, revision 25.0.3
67+
# 12 - Android SDK Build-tools, revision 23.0.1
68+
# 35 - SDK Platform Android 6.0, API 23, revision 3
69+
# 39 - SDK Platform Android 4.4.2, API 19, revision 4
70+
# 102 - ARM EABI v7a System Image, Android API 19, revision 5
71+
# 103 - Intel x86 Atom System Image, Android API 19, revision 5
72+
# 131 - Google APIs, Android API 23, revision 1
73+
# 166 - Android Support Repository, revision 41
74+
RUN echo "y" | android update sdk -u -a -t 2,12,35,39,102,103,131,166
75+
RUN ln -s /opt/android/platform-tools/adb /usr/bin/adb
76+
77+
# clean up unnecessary directories
78+
RUN rm -rf /opt/android/system-images/android-19/default/x86

‎ContainerShip/Dockerfile.javascript‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
FROM library/node:6.9.2
2+
3+
ENV YARN_VERSION=0.19.1
4+
5+
# install dependencies
6+
RUN apt-get update && apt-get install ocaml libelf-dev -y
7+
RUN npm install yarn@$YARN_VERSION -g
8+
9+
# add code
10+
RUN mkdir /app
11+
ADD . /app
12+
13+
WORKDIR /app
14+
RUN yarn install --ignore-engines
15+
16+
WORKDIR website
17+
RUN yarn install --ignore-engines --ignore-platform
18+
19+
WORKDIR /app

‎ContainerShip/scripts/run-android-ci-instrumentation-tests.js‎

Lines changed: 153 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,153 @@
1+
/**
2+
* Copyright (c) 2015-present, Facebook, Inc.
3+
* All rights reserved.
4+
*
5+
* This source code is licensed under the BSD-style license found in the
6+
* LICENSE file in the root directory of this source tree. An additional grant
7+
* of patent rights can be found in the PATENTS file in the same directory.
8+
*/
9+
10+
'use strict';
11+
12+
/**
13+
* This script runs instrumentation tests one by one with retries
14+
* Instrumentation tests tend to be flaky, so rerunning them individually increases
15+
* chances for success and reduces total average execution time.
16+
*
17+
* We assume that all instrumentation tests are flat in one folder
18+
* Available arguments:
19+
* --path - path to all .java files with tests
20+
* --package - com.facebook.react.tests
21+
* --retries [num] - how many times to retry possible flaky commands: npm install and running tests, default 1
22+
*/
23+
/*eslint-disable no-undef */
24+
25+
const argv = require('yargs').argv;
26+
const async = require('async');
27+
const child_process = require('child_process');
28+
const fs = require('fs');
29+
const path = require('path');
30+
31+
const colors = {
32+
GREEN: '\x1b[32m',
33+
RED: '\x1b[31m',
34+
RESET: '\x1b[0m'
35+
};
36+
37+
const test_opts = {
38+
FILTER: new RegExp(argv.filter || '.*', 'i'),
39+
PACKAGE: argv.package || 'com.facebook.react.tests',
40+
PATH: argv.path || './ReactAndroid/src/androidTest/java/com/facebook/react/tests',
41+
RETRIES: parseInt(argv.retries || 2, 10),
42+
43+
TEST_TIMEOUT: parseInt(argv['test-timeout'] || 1000 * 60 * 10),
44+
45+
OFFSET: argv.offset,
46+
COUNT: argv.count
47+
}
48+
49+
let max_test_class_length = Number.NEGATIVE_INFINITY;
50+
51+
let testClasses = fs.readdirSync(path.resolve(process.cwd(), test_opts.PATH))
52+
.filter((file) => {
53+
return file.endsWith('.java');
54+
}).map((clazz) => {
55+
return path.basename(clazz, '.java');
56+
}).map((clazz) => {
57+
return test_opts.PACKAGE + '.' + clazz;
58+
}).filter((clazz) => {
59+
return test_opts.FILTER.test(clazz);
60+
});
61+
62+
// only process subset of the tests at corresponding offset and count if args provided
63+
if (test_opts.COUNT != null && test_opts.OFFSET != null) {
64+
const testCount = testClasses.length;
65+
const start = test_opts.COUNT * test_opts.OFFSET;
66+
const end = start + test_opts.COUNT;
67+
68+
if (start >= testClasses.length) {
69+
testClasses = [];
70+
} else if (end >= testClasses.length) {
71+
testClasses = testClasses.slice(start);
72+
} else {
73+
testClasses = testClasses.slice(start, end);
74+
}
75+
}
76+
77+
return async.mapSeries(testClasses, (clazz, callback) => {
78+
if(clazz.length > max_test_class_length) {
79+
max_test_class_length = clazz.length;
80+
}
81+
82+
return async.retry(test_opts.RETRIES, (retryCb) => {
83+
const test_process = child_process.spawn('./ContainerShip/scripts/run-instrumentation-tests-via-adb-shell.sh', [test_opts.PACKAGE, clazz], {
84+
stdio: 'inherit'
85+
})
86+
87+
const timeout = setTimeout(() => {
88+
test_process.kill();
89+
}, test_opts.TEST_TIMEOUT);
90+
91+
test_process.on('error', (err) => {
92+
clearTimeout(timeout);
93+
retryCb(err);
94+
});
95+
96+
test_process.on('exit', (code) => {
97+
clearTimeout(timeout);
98+
99+
if(code !== 0) {
100+
return retryCb(new Error(`Process exited with code: ${code}`));
101+
}
102+
103+
return retryCb();
104+
});
105+
}, (err) => {
106+
return callback(null, {
107+
name: clazz,
108+
status: err ? 'failure' : 'success'
109+
});
110+
});
111+
}, (err, results) => {
112+
print_test_suite_results(results);
113+
114+
const failures = results.filter((test) => {
115+
test.status === 'failure';
116+
});
117+
118+
return failures.length === 0 ? process.exit(0) : process.exit(1);
119+
});
120+
121+
function print_test_suite_results(results) {
122+
console.log('\n\nTest Suite Results:\n');
123+
124+
let color;
125+
let failing_suites = 0;
126+
let passing_suites = 0;
127+
128+
function pad_output(num_chars) {
129+
let i = 0;
130+
131+
while(i < num_chars) {
132+
process.stdout.write(' ');
133+
i++;
134+
}
135+
}
136+
results.forEach((test) => {
137+
if(test.status === 'success') {
138+
color = colors.GREEN;
139+
passing_suites++;
140+
} else if(test.status === 'failure') {
141+
color = colors.RED;
142+
failing_suites++;
143+
}
144+
145+
process.stdout.write(color);
146+
process.stdout.write(test.name);
147+
pad_output((max_test_class_length - test.name.length) + 8);
148+
process.stdout.write(test.status);
149+
process.stdout.write(`${colors.RESET}\n`);
150+
});
151+
152+
console.log(`\n${passing_suites} passing, ${failing_suites} failing!`);
153+
}

‎ContainerShip/scripts/run-android-docker-instrumentation-tests.sh‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
#!/bin/bash
2+
3+
# for buck gen
4+
mount -o remount,exec /dev/shm
5+
6+
AVD_UUID=$(cat /dev/urandom | tr -dc 'a-zA-Z0-9' | fold -w 8 | head -n 1)
7+
8+
# create virtual device
9+
echo no | android create avd -n $AVD_UUID -f -t android-19 --abi default/armeabi-v7a
10+
11+
# emulator setup
12+
emulator64-arm -avd $AVD_UUID -no-skin -no-audio -no-window -no-boot-anim &
13+
bootanim=""
14+
until [[ "$bootanim" =~ "stopped" ]]; do
15+
sleep 5
16+
bootanim=$(adb -e shell getprop init.svc.bootanim 2>&1)
17+
echo "boot animation status=$bootanim"
18+
done
19+
20+
set -x
21+
22+
# solve issue with max user watches limit
23+
echo 65536 | tee -a /proc/sys/fs/inotify/max_user_watches
24+
watchman shutdown-server
25+
26+
# integration tests
27+
# build JS bundle for instrumentation tests
28+
node local-cli/cli.js bundle --platform android --dev true --entry-file ReactAndroid/src/androidTest/js/TestBundle.js --bundle-output ReactAndroid/src/androidTest/assets/AndroidTestBundle.js
29+
30+
# build test APK
31+
buck install ReactAndroid/src/androidTest/buck-runner:instrumentation-tests --config build.threads=1
32+
33+
# run installed apk with tests
34+
node ./ContainerShip/scripts/run-android-ci-instrumentation-tests.js $*

‎ContainerShip/scripts/run-android-docker-unit-tests.sh‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
#!/bin/bash
2+
3+
# set default environment variables
4+
UNIT_TESTS_BUILD_THREADS="${UNIT_TESTS_BUILD_THREADS:-1}"
5+
6+
# for buck gen
7+
mount -o remount,exec /dev/shm
8+
9+
set -x
10+
11+
# run unit tests
12+
buck test ReactAndroid/src/test/... --config build.threads=$UNIT_TESTS_BUILD_THREADS

0 commit comments

Comments
 (0)