Skip to content

chore: Migrate gsutil usage to gcloud storage #484

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
bhandarivijay-png wants to merge 1 commit into
base: master
Choose a base branch
from
Open

chore: Migrate gsutil usage to gcloud storage #484

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 2 additions & 3 deletions cloudbuild.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@ steps:
secretEnv:
- AUTOPUSH_DC_API_KEY

- name: 'gcr.io/cloud-builders/gsutil'
args: ['cp', 'tool/target/datacommons-import-tool-0.1-SNAPSHOT-jar-with-dependencies.jar', 'gs://datacommons_public/import_tools/import-tool.jar']
- name: 'gcr.io/cloud-builders/gcloud'
args: ['storage', 'cp', 'tool/target/datacommons-import-tool-0.1-SNAPSHOT-jar-with-dependencies.jar', 'gs://datacommons_public/import_tools/import-tool.jar']

- name: 'gcr.io/cloud-builders/gcloud'
args: ['builds', 'triggers', 'run', 'dc-import-executor', '--branch=master', '--substitutions', '_DOCKER_IMAGE=us-docker.pkg.dev/datcom-ci/gcr.io/dc-import-executor']
Expand All @@ -15,4 +15,3 @@ availableSecrets:
secretManager:
- versionName: projects/datcom-ci/secrets/autopush-dc-api-key/versions/latest
env: AUTOPUSH_DC_API_KEY

4 changes: 2 additions & 2 deletions simple/run_stats.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -245,10 +245,10 @@ function copy_to_gcs {

[[ -z "$gcs_dir" ]] && return

cmd="gsutil -m cp -r $dir $gcs_dir"
cmd="gcloud storage cp --recursive $dir $gcs_dir"
run_cmd $cmd
echo_log "Copied output files to $gcs_dir"
run_cmd gsutil ls -l -r "$gcs_dir"
run_cmd gcloud storage ls --long --recursive "$gcs_dir"
Comment on lines +248 to +251
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot Mar 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

security-high high

This script is vulnerable to command injection because it constructs shell commands using unquoted variables and executes them via the run_cmd function. Specifically, in the modified lines, $dir and $gcs_dir are unquoted when constructing the command string for gcloud storage cp (line 248) and $gcs_dir is passed unquoted to run_cmd for gcloud storage ls (line 251). If an attacker can control the value of $OUTPUT_DIR (which populates $gcs_dir), they can inject arbitrary shell commands. The run_cmd function is also not safe for arguments containing spaces as it reconstructs the command as a string, losing individual argument quoting. To remediate this, you should: 1. Fix the run_cmd function to execute arguments safely by using the "$@" expansion within quotes. 2. Update calls to run_cmd to pass arguments directly and ensure all variables are properly quoted. This will also make the script robust against paths with spaces.

Suggested change
cmd="gcloud storage cp --recursive $dir $gcs_dir"
run_cmd $cmd
echo_log "Copied output files to $gcs_dir"
run_cmd gsutil ls -l -r "$gcs_dir"
run_cmd gcloud storage ls --long --recursive "$gcs_dir"
run_cmd gcloud storage cp --recursive "$dir" "$gcs_dir"
echo_log "Copied output files to $gcs_dir"
run_cmd gcloud storage ls --long --recursive "$gcs_dir"

}

# Return if being sourced
Expand Down