Skip to content

Conversation

@buildkite-first-draft
Copy link

Description

This PR introduces a new buildkite-agent secret batch-get command that allows users to fetch multiple secrets in a single API call, addressing performance concerns for large organizations like Reddit who have many secrets and experience 30-45 second delays when fetching secrets individually.

The solution provides a backward-compatible addition that doesn't change existing functionality while offering significant performance improvements for batch operations.

Context

Linear Issue: PS-1064 - Reddit request for batch secret retrieval functionality to reduce build setup times from 30-45 seconds.

Changes

API Layer:

  • Added GetSecretsRequest struct for batch requests with Keys []string and JobID string
  • Added GetSecretsResponse struct with Secrets []Secret field
  • Added GetSecrets method to API client using key[]=val1&key[]=val2 query format
  • Maintained full backward compatibility with existing GetSecret method

CLI Layer:

  • Added buildkite-agent secret batch-get subcommand
  • Support for multiple keys as command line arguments: batch-get key1 key2 key3
  • Support for reading keys from file: --keys-from-file secrets.txt
  • Output format options: --format env (default) or --format json
  • Automatic redaction of all retrieved secret values (can be disabled with --skip-redaction)
  • Comments and empty lines ignored when reading from file

Testing:

  • Comprehensive test suite for GetSecrets API method
  • Tests for success, authorization, job not found, and partial secrets not found scenarios
  • Mock HTTP server simulating both single and batch secret endpoints

Command Examples:

# Fetch multiple secrets as environment variables
buildkite-agent secret batch-get deploy_key api_token db_password

# Fetch secrets from file
buildkite-agent secret batch-get --keys-from-file secrets.txt

# Get JSON output
buildkite-agent secret batch-get key1 key2 --format json

Testing

  • Tests have run locally (with go test ./...). Buildkite employees may check this if the pipeline has run automatically.
  • Code is formatted (with go fmt ./...)

Note: Local testing was limited due to Go version constraints in development environment. Comprehensive tests included and CI will validate functionality.

Disclosures / Credits

This implementation was created using Claude Code (First Draft agent) to address the Reddit customer request. The solution includes:

  • Full API and CLI implementation written by Claude Code
  • Comprehensive test suite generated by Claude Code
  • Documentation and examples written by Claude Code
  • Approach designed to maintain backward compatibility while providing the requested batch functionality

🤖 Generated with Claude Code

First Draft and others added 5 commits August 29, 2025 02:11
This commit introduces a new `buildkite-agent secret batch-get` command that allows
users to fetch multiple secrets in a single API call, addressing performance
concerns with large numbers of secrets.

Changes:
- Add GetSecrets API method for batch secret retrieval
- Add SecretBatchGetCommand CLI command with support for:
  - Multiple keys as command line arguments
  - Reading keys from file (--keys-from-file)
  - Output formats: env (default) and json
  - Automatic redaction of secret values
- Add comprehensive tests for batch secret functionality
- Maintain backward compatibility with existing secret get command

Resolves: PS-1064

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
Apply gofmt to align constant declarations properly.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
Add GetSecrets method to APIClient interface to match the new batch secret functionality.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
Include the new batch-get command in the configuration completeness test to ensure all CLI flags are properly tested.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
Change Keys field tag from 'arg' to 'arg:*' to properly handle variable arguments in SecretBatchGetConfig, matching the pattern used in other commands like pipeline upload.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant