Skip to content

Add "exit 0" to all Big Data installation commands #742

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -180,7 +180,7 @@ systemctl start granulate-gprofiler.service
```

## Running on Databricks
For Databricks, the same installation instructions as specified in the [running as an executable](#running-as-an-executable) section can be used (make sure to run them in the initialization script of your node).
For Databricks, the same installation instructions as specified in the [running as an executable](#running-as-an-executable) section can be used (make sure to run them in the initialization script of your node). Also, make sure to add `exit 0` after the commands - Databricks will not start the node if the initalization script fails, to avoid it you should add `exit 0` as the last command.

Additionally, 2 more flags need to be added to gProfiler's commandline: `--disable-pidns-check --perf-mode=none`. You can add them right after the `--service-name` argument.

Expand Down
45 changes: 24 additions & 21 deletions deploy/dataproc/gprofiler_initialization_action.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,31 +3,34 @@
# Copyright (c) Granulate. All rights reserved.
# Licensed under the AGPL3 License. See LICENSE.md in the project root for license information.
#
set -euxo pipefail
(
set -euxo pipefail

GPROFILER_TOKEN=$(/usr/share/google/get_metadata_value attributes/gprofiler-token)
readonly GPROFILER_TOKEN
GPROFILER_TOKEN=$(/usr/share/google/get_metadata_value attributes/gprofiler-token)
readonly GPROFILER_TOKEN

GPROFILER_SERVICE=$(/usr/share/google/get_metadata_value attributes/gprofiler-service)
readonly GPROFILER_SERVICE
GPROFILER_SERVICE=$(/usr/share/google/get_metadata_value attributes/gprofiler-service)
readonly GPROFILER_SERVICE

ENABLE_STDOUT=$(/usr/share/google/get_metadata_value attributes/enable-stdout)
readonly ENABLE_STDOUT
ENABLE_STDOUT=$(/usr/share/google/get_metadata_value attributes/enable-stdout)
readonly ENABLE_STDOUT

SPARK_METRICS=$(/usr/share/google/get_metadata_value attributes/spark-metrics || true)
readonly SPARK_METRICS
SPARK_METRICS=$(/usr/share/google/get_metadata_value attributes/spark-metrics || true)
readonly SPARK_METRICS

OUTPUT_REDIRECTION=""
if [ "$ENABLE_STDOUT" != "1" ]; then
OUTPUT_REDIRECTION="> /dev/null 2>&1"
fi
OUTPUT_REDIRECTION=""
if [ "$ENABLE_STDOUT" != "1" ]; then
OUTPUT_REDIRECTION="> /dev/null 2>&1"
fi

flags=""
if [[ "$SPARK_METRICS" == "1" ]]; then
flags="$flags --collect-spark-metrics"
fi
flags=""
if [[ "$SPARK_METRICS" == "1" ]]; then
flags="$flags --collect-spark-metrics"
fi

wget --no-verbose "https://github.com/Granulate/gprofiler/releases/latest/download/gprofiler_$(uname -m)" -O gprofiler
sudo chmod +x gprofiler
sudo sh -c "setsid ./gprofiler -cu --token='$GPROFILER_TOKEN' --service-name='$GPROFILER_SERVICE' $flags $OUTPUT_REDIRECTION &"
echo "gProfiler installed successfully."
wget --no-verbose "https://github.com/Granulate/gprofiler/releases/latest/download/gprofiler_$(uname -m)" -O gprofiler
sudo chmod +x gprofiler
sudo sh -c "setsid ./gprofiler -cu --token='$GPROFILER_TOKEN' --service-name='$GPROFILER_SERVICE' $flags $OUTPUT_REDIRECTION &"
echo "gProfiler installed successfully."
)
exit 0 # ensures we never block the Dataproc node from starting.
19 changes: 11 additions & 8 deletions deploy/emr/gprofiler_action.sh
Original file line number Diff line number Diff line change
@@ -1,12 +1,15 @@
#!/bin/bash

# Re-run as root:
test $EUID = 0 || exec sudo "$0" "$@"
(
# Re-run as root:
test $EUID = 0 || exec sudo "$0" "$@"

version=latest
arch=$(uname -m)
version=latest
arch=$(uname -m)

wget "https://github.com/Granulate/gprofiler/releases/$version/download/gprofiler_$arch" -O gprofiler
chmod +x gprofiler
# Must supply --token=... and --service-name=... arguments when creating cluster
setsid ./gprofiler -cu "$@" >/dev/null 2>&1 &
wget "https://github.com/Granulate/gprofiler/releases/$version/download/gprofiler_$arch" -O gprofiler
chmod +x gprofiler
# Must supply --token=... and --service-name=... arguments when creating cluster
setsid ./gprofiler -cu "$@" >/dev/null 2>&1 &
)
exit 0 # ensures we never block the EMR node from starting.