forked from viamrobotics/rdk
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
RSDK-6865 android ML and streaming libraries (viamrobotics#3889)
Co-authored-by: Eric Daniels <[email protected]>
- Loading branch information
1 parent
2e97f5e
commit 04de2fd
Showing
31 changed files
with
319 additions
and
38 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -85,3 +85,7 @@ etc/FFmpeg | |
|
|
||
| # qemu core dumps | ||
| web/cmd/server/*.core | ||
|
|
||
| # gomobile / android | ||
| *.aar | ||
| *.jar | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,54 @@ | ||
| $(NDK_ROOT): | ||
| # todo: remove this once we are building .aar in CI | ||
| # download ndk (used by server-android) | ||
| cd etc && wget https://dl.google.com/android/repository/android-ndk-r26-linux.zip | ||
| cd etc && unzip android-ndk-r26-linux.zip | ||
|
|
||
| .PHONY: server-android | ||
| server-android: | ||
| GOOS=android GOARCH=arm64 CGO_ENABLED=1 \ | ||
| CC=$(shell realpath $(NDK_ROOT)/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android30-clang) \ | ||
| go build -v \ | ||
| -tags no_cgo \ | ||
| -o bin/viam-server-$(BUILD_CHANNEL)-android-aarch64 \ | ||
| ./web/cmd/server | ||
|
|
||
| UNAME = $(shell uname) | ||
| ifeq ($(UNAME),Linux) | ||
| PLATFORM_NDK_ROOT ?= $(HOME)/Android/Sdk/ndk/26.1.10909125 | ||
| else | ||
| PLATFORM_NDK_ROOT ?= $(HOME)/Android/Sdk/ndk/26.1.10909125 | ||
| endif | ||
| DROID_CC ?= $(PLATFORM_NDK_ROOT)/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android28-clang | ||
| DROID_PREFIX = $(PWD)/etc/android/prefix | ||
|
|
||
| etc/android/prefix/%: | ||
| TARGET_ARCH=$* etc/android/build-x264.sh | ||
|
|
||
| .SECONDARY: etc/android/prefix/aarch64 etc/android/prefix/x86_64 | ||
|
|
||
| droid-rdk.%.aar: etc/android/prefix/aarch64 etc/android/prefix/x86_64 | ||
| # creates a per-platform android library that can be imported by native code | ||
| # we clear CGO_LDFLAGS so this doesn't try (and fail) to link to linuxbrew where present | ||
| $(eval JNI_ARCH := $(if $(filter arm64,$*),arm64-v8a,x86_64)) | ||
| $(eval CPU_ARCH := $(if $(filter arm64,$*),aarch64,x86_64)) | ||
| CGO_LDFLAGS= PKG_CONFIG_PATH=$(DROID_PREFIX)/$(CPU_ARCH)/lib/pkgconfig \ | ||
| gomobile bind -v -target android/$* -androidapi 28 -tags no_cgo \ | ||
| -o $@ ./web/cmd/droid | ||
| rm -rf droidtmp/jni/$(JNI_ARCH) | ||
| mkdir -p droidtmp/jni/$(JNI_ARCH) | ||
| cp -d etc/android/prefix/$(CPU_ARCH)/lib/*.so* droidtmp/jni/$(JNI_ARCH) | ||
| cd droidtmp && zip --symlinks -r ../$@ jni/$(JNI_ARCH) | ||
| cd ./services/mlmodel/tflitecpu/android/ && zip --symlinks -r ../../../../$@ jni/$(JNI_ARCH) | ||
|
|
||
| droid-rdk.aar: droid-rdk.amd64.aar droid-rdk.arm64.aar | ||
| # multi-platform AAR -- twice the size, but portable | ||
| rm -rf droidtmp | ||
| cp droid-rdk.arm64.aar $@.tmp | ||
| unzip droid-rdk.amd64.aar -d droidtmp | ||
| cd droidtmp && zip --symlinks -r ../$@.tmp jni | ||
| mv $@.tmp $@ | ||
|
|
||
| clean-droid: | ||
| # note: this doesn't clean x264 checkout | ||
| rm -rvf droid-rdk*.aar droid-rdk*.jar etc/android/prefix droidtmp |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,2 @@ | ||
| prefix/ | ||
| x264/ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,3 @@ | ||
| ## unversion-soname | ||
|
|
||
| The unversion-soname patch in this folder changes the soname of x264 from `.so.VERSION_NUMBER` format to strict `.so`. We do this because the android toolchain removes the version-suffixed version of the SO, but our go build links against the soname it observes. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,38 @@ | ||
| #!/bin/bash -e | ||
|
|
||
| if [[ -z "${ANDROID_NDK}" ]]; then | ||
| echo "Must provide ANDROID_NDK in environment" 1>&2 | ||
| exit 1 | ||
| fi | ||
|
|
||
| DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )" | ||
| ROOT_DIR="${DIR}/../../" | ||
| source ${ANDROID_NDK}/build/tools/ndk_bin_common.sh | ||
| NDK_TOOLCHAIN=${ANDROID_NDK}/toolchains/llvm/prebuilt/${HOST_TAG} | ||
|
|
||
| # ripped from private sysops repo | ||
| cd ~ && mkdir -p tensorflow/build_arm64-v8a tensorflow/build_x86_64 && cd tensorflow | ||
| curl -L https://github.com/tensorflow/tensorflow/archive/refs/tags/v2.12.0.tar.gz | tar -xz | ||
| patch -p1 -d tensorflow-2.12.0 < ${DIR}/tflite.patch | ||
| function build() { | ||
| local arch=$1 | ||
| pushd ~/tensorflow/build_$arch | ||
| cmake -DCMAKE_TOOLCHAIN_FILE=${ANDROID_NDK}/build/cmake/android.toolchain.cmake \ | ||
| -DANDROID_ABI=$arch ../tensorflow-2.12.0/tensorflow/lite/c | ||
| cmake --build . -j | ||
| ${NDK_TOOLCHAIN}/bin/llvm-strip --strip-unneeded libtensorflowlite_c.so | ||
| local dest=${ROOT_DIR}/services/mlmodel/tflitecpu/android/jni/$arch | ||
| mkdir -p $dest | ||
| cp libtensorflowlite_c.so $dest | ||
| popd | ||
| } | ||
| for arch in arm64-v8a x86_64; do | ||
| build $arch | ||
| done | ||
| if [ $KEEP_TFLITE_SRC != "1" ]; then | ||
| echo "cleaning up source and build" | ||
| rm -rf ~/tensorflow/ | ||
| else | ||
| echo "cleaning up build, keeping source" | ||
| rm -rf ~/tensorflow/build_* | ||
| fi |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,66 @@ | ||
| #!/usr/bin/env bash | ||
| # build a static x264 distro for android | ||
|
|
||
| set -euxo pipefail | ||
|
|
||
| if [ $(uname) = Linux ]; then | ||
| : "${NDK_ROOT:=$HOME/Android/Sdk/ndk/26.1.10909125}" | ||
| HOST_OS=linux | ||
| else | ||
| : "${NDK_ROOT:=$HOME/Library/Android/sdk/ndk/26.1.10909125}" | ||
| HOST_OS=darwin | ||
| fi | ||
|
|
||
| API_LEVEL=28 | ||
| : "${TARGET_ARCH:=aarch64}" | ||
| TOOLCHAIN=$NDK_ROOT/toolchains/llvm/prebuilt/$HOST_OS-x86_64 | ||
| export CC=$TOOLCHAIN/bin/$TARGET_ARCH-linux-android$API_LEVEL-clang | ||
| EXTRAS= | ||
| if [ $TARGET_ARCH = aarch64 ]; then | ||
| EXTRAS="--extra-cflags=-march=armv8-a" | ||
| fi | ||
| # CXX=$TOOLCHAIN/bin/$CC_ARCH-linux-android$API_LEVEL-clang++ | ||
| # AR=$TOOLCHAIN/bin/llvm-ar | ||
| # LD=$CC | ||
| # RANLIB=$TOOLCHAIN/bin/llvm-ranlib | ||
| # STRIP=$TOOLCHAIN/bin/llvm-strip | ||
| # NM=$TOOLCHAIN/bin/llvm-nm | ||
| SYSROOT=$TOOLCHAIN/sysroot | ||
| DIRNAME=$(realpath $(dirname $0)) | ||
| PREFIX=$DIRNAME/prefix/$TARGET_ARCH | ||
| X264_ROOT=$DIRNAME/x264 | ||
|
|
||
| if [ ! -e $X264_ROOT ]; then | ||
| echo checking out x264 | ||
| git clone https://code.videolan.org/videolan/x264.git -b stable $X264_ROOT | ||
| else | ||
| echo using existing x264 | ||
| fi | ||
|
|
||
| cd $X264_ROOT | ||
| # note: patchelf can also change the soname | ||
| if git apply --check ../unversion-soname.patch; then | ||
| git apply ../unversion-soname.patch | ||
| elif git apply --reverse --check ../unversion-soname.patch; then | ||
| echo "not applying patch, already applied" | ||
| else | ||
| # note: we patch the soname because android build resolves the libx264.so -> libx264.so.164 symlink | ||
| # if allowed, this will build successfully and then fail on startup. | ||
| echo "soname patch could not be applied, bailing" | ||
| exit 1 | ||
| fi | ||
| ./configure \ | ||
| --prefix=$PREFIX \ | ||
| --host=$TARGET_ARCH-linux-android \ | ||
| --cross-prefix=$TOOLCHAIN/bin/llvm- \ | ||
| --sysroot=$SYSROOT \ | ||
| $EXTRAS \ | ||
| --enable-shared \ | ||
| --disable-avs \ | ||
| --disable-swscale \ | ||
| --disable-lavf \ | ||
| --disable-ffms \ | ||
| --disable-gpac \ | ||
| --disable-lsmash \ | ||
| && make \ | ||
| && make install |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,35 @@ | ||
| --- a/tensorflow/lite/core/interpreter.cc | ||
| +++ b/tensorflow/lite/core/interpreter.cc | ||
| @@ -445,8 +445,10 @@ | ||
| } | ||
|
|
||
| TfLiteStatus Interpreter::ReportTelemetrySettings(const char* setting_name) { | ||
| +#ifdef TF_LITE_TENSORFLOW_PROFILER | ||
| telemetry::TelemetryReportSettings(context_, setting_name, | ||
| telemetry_data_.get()); | ||
| +#endif //TFLITE_TENSORFLOW_PROFILER | ||
| return kTfLiteOk; | ||
| } | ||
|
|
||
| --- a/tensorflow/lite/core/subgraph.cc | ||
| +++ b/tensorflow/lite/core/subgraph.cc | ||
| @@ -1382,7 +1382,9 @@ | ||
|
|
||
| TfLiteStatus Subgraph::Invoke() { | ||
| auto status = InvokeImpl(); | ||
| +#ifdef TF_LITE_TENSORFLOW_PROFILER | ||
| telemetry::TelemetryReportEvent(&context_, "Invoke", status); | ||
| +#endif //TFLITE_TENSORFLOW_PROFILER | ||
| return status; | ||
| } | ||
| TfLiteStatus Subgraph::InvokeImpl() { | ||
| @@ -1966,7 +1968,9 @@ | ||
|
|
||
| TfLiteStatus Subgraph::ModifyGraphWithDelegate(TfLiteDelegate* delegate) { | ||
| auto status = ModifyGraphWithDelegateImpl(delegate); | ||
| +#ifdef TF_LITE_TENSORFLOW_PROFILER | ||
| telemetry::TelemetryReportEvent(&context_, "ModifyGraphWithDelegate", status); | ||
| +#endif //TFLITE_TENSORFLOW_PROFILER | ||
| return status; | ||
| } | ||
|
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| diff --git a/configure b/configure | ||
| index e242e73c..83b6ec82 100755 | ||
| --- a/configure | ||
| +++ b/configure | ||
| @@ -1591,7 +1591,7 @@ if [ "$shared" = "yes" ]; then | ||
| echo "SOFLAGS=-shared -Wl,-h,\$(SONAME) $SOFLAGS" >> config.mak | ||
| else | ||
| echo "SOSUFFIX=so" >> config.mak | ||
| - echo "SONAME=libx264.so.$API" >> config.mak | ||
| + echo "SONAME=libx264.so" >> config.mak | ||
| echo "SOFLAGS=-shared -Wl,-soname,\$(SONAME) $SOFLAGS" >> config.mak | ||
| fi | ||
| echo 'default: lib-shared' >> config.mak |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| //go:build !no_cgo | ||
|
|
||
| package gostream | ||
|
|
||
| import ( | ||
| // comment for linter. | ||
| _ "github.com/pion/mediadevices/pkg/driver/microphone" | ||
| ) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| //go:build no_cgo | ||
|
|
||
| package web | ||
|
|
||
| import "net/http" | ||
|
|
||
| // stub for missing graphviz | ||
| func (svc *webService) handleVisualizeResourceGraph(w http.ResponseWriter, r *http.Request) { | ||
| w.Write([]byte(`<html><body>Resource graph not supported on this build</body></html>`)) | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,4 +1,4 @@ | ||
| //go:build !no_cgo | ||
| //go:build !no_cgo || android | ||
|
|
||
| package web | ||
|
|
||
|
|
||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,4 +1,4 @@ | ||
| //go:build !no_cgo | ||
| //go:build !no_cgo || android | ||
|
|
||
| package web | ||
|
|
||
|
|
||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.