gke-cost-analyzer

A CLI tool to monitor and analyze costs of GKE workloads (Autopilot and standard), with support for real-time display and BigQuery export.

Features

• Real-time cost monitoring (watch): Interactive TUI with live cost table, team rollup/drill-down, column sorting, and cost aberration detection via EWMA trend tracking
• Standard GKE cost estimation: Per-node proportional attribution for standard GKE workloads based on Compute Engine pricing
• Historical cost analysis (history): Query BigQuery for recorded cost data with an interactive TUI, sparkline trend visualizations, and optional filters by cluster, namespace, and team
• BigQuery recording (record): Periodically writes cost snapshots to BigQuery for historical analysis, with optional Parquet export via --dry-run --output-file
• Automated setup (setup): Creates the BigQuery dataset and table with the correct schema, and migrates existing tables when newer versions add columns
• Utilization metrics: Automatic CPU/memory utilization from GCP Managed Prometheus (or custom --prometheus-url), with efficiency scoring and wasted cost calculation
• Unmatched pod detection (unmatched-pods): Find running pods missing team or workload labels, grouped by base name
• Price caching: Fetches Autopilot and Compute Engine pricing from the Cloud Billing Catalog API and caches locally for 24 hours
• SPOT support: Automatically detects and separately prices Spot pods
• Configurable label hierarchy: Group costs by team, workload, and subtype using custom label names
• Environment auto-detection: --region, the GCP project, and --cluster-name are auto-detected from GCE metadata or kubeconfig context. The inferred project is used for both BigQuery and Managed Prometheus unless overridden with --bigquery-project-id / --prometheus-project-id (or a fully-qualified --table)
• Namespace exclusion: System namespaces (kube-system, gmp-system) are excluded by default via --exclude-namespaces

Installation

Prerequisites

• Go 1.26+
• Access to a GKE cluster (via kubeconfig)
• GCP credentials (for BigQuery features)

Build

go build -o gke-cost-analyzer .

Docker

Pre-built Docker images are published to GitHub Container Registry on each release:

# Pull the latest release
docker pull ghcr.io/samn/gke-cost-analyzer:latest

# Pull a specific version
docker pull ghcr.io/samn/gke-cost-analyzer:v0.6.1

Run with arguments passed directly:

docker run --rm ghcr.io/samn/gke-cost-analyzer version

docker run --rm ghcr.io/samn/gke-cost-analyzer record \
  --region us-central1 \
  --bigquery-project-id my-gcp-project \
  --cluster-name my-cluster

To use commands that require GCP credentials (like history), mount your Application Default Credentials into the container:

docker run --rm \
  --user "$(id -u):$(id -g)" \
  -v "$HOME/.config/gcloud/application_default_credentials.json:/tmp/adc.json:ro" \
  -e GOOGLE_APPLICATION_CREDENTIALS=/tmp/adc.json \
  ghcr.io/samn/gke-cost-analyzer history 7d \
  --bigquery-project-id my-gcp-project

If you haven't already, generate ADC with:

gcloud auth application-default login

Building locally

To build a Docker image from source, first build the binary then build the image:

make build
docker build -t gke-cost-analyzer .

Permissions

Kubernetes RBAC

The tool needs read access to pods and nodes. It does not create, modify, or delete any Kubernetes resources.

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: gke-cost-analyzer
rules:
- apiGroups: [""]
  resources: ["pods", "nodes"]
  verbs: ["list"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: gke-cost-analyzer
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: gke-cost-analyzer
subjects:
- kind: ServiceAccount
  name: gke-cost-analyzer
  namespace: <namespace>

To restrict to specific namespaces, use a Role + RoleBinding per namespace instead of ClusterRole.

GCP IAM

All GCP API calls use Application Default Credentials. On GKE, use Workload Identity to bind the Kubernetes ServiceAccount to a GCP service account with the required roles.

Role	Used by	Purpose
roles/billing.viewer	watch, record	Read Autopilot pricing SKUs from the Cloud Billing Catalog API
roles/bigquery.dataViewer	history	Read cost snapshots from BigQuery
roles/bigquery.dataEditor	record	Stream-insert cost snapshots into BigQuery
roles/bigquery.dataOwner	setup	Create BigQuery datasets and tables; re-run after upgrades to migrate the schema
roles/monitoring.metricReader	watch, record	Query CPU/memory utilization via GCP Managed Prometheus

Notes:

• roles/monitoring.metricReader is only needed when using GCP Managed Prometheus (the default when a project is available — auto-detected or set via --prometheus-project-id). It is not needed with --prometheus-url (custom Prometheus endpoint) or --dry-run.
• roles/bigquery.dataEditor is not needed in --dry-run mode.
• If you pre-create the dataset and table manually, roles/bigquery.dataEditor is sufficient for setup as well.

Usage

Watch costs in real-time

gke-cost-analyzer watch --region us-central1

Options:

• --region (required): GCP region for pricing lookup (auto-detected from environment)
• --mode: Cost calculation mode: autopilot, standard, or all (default: all)
• --interval: Refresh interval (default: 10s)
• --namespace: Filter to a specific namespace (default: all)
• --prometheus-project-id: GCP project ID for Managed Prometheus utilization (defaults to the auto-detected environment project)
• --prometheus-url: Custom Prometheus API base URL (defaults to GCP Managed Prometheus when a project is available)
• --exclude-namespaces: Namespaces to exclude (default: kube-system,gmp-system)
• --trend-threshold: Z-score threshold for cost aberration detection (default: 3.0, 0 to disable)
• --team-label: Pod label for team grouping (default: "team")
• --workload-label: Pod label for workload grouping (default: "app")
• --subtype-label: Pod label for subtype grouping (optional)

Interactive keys: Enter/a expand/collapse teams, g toggle flat/grouped view, e toggle event log, [/] scroll event log, 1-9,0,-,= sort columns, j/k/Up/Down navigate, q quit.

Record costs to BigQuery

First, set up the BigQuery dataset and table:

gke-cost-analyzer setup \
  --bigquery-project-id my-gcp-project \
  --location US

Then start recording:

gke-cost-analyzer record \
  --region us-central1 \
  --bigquery-project-id my-gcp-project \
  --cluster-name my-cluster \
  --interval 5m

Options:

• --region (required): GCP region for pricing lookup (auto-detected from environment)
• --bigquery-project-id: GCP project ID owning the BigQuery dataset (defaults to the auto-detected environment project; overridden by a fully-qualified --table)
• --prometheus-project-id: GCP project ID for Managed Prometheus utilization (defaults to the auto-detected environment project)
• --cluster-name (required): GKE cluster name (auto-detected)
• --dataset: BigQuery dataset name (default: "gke_costs")
• --table: BigQuery table name (default: "cost_snapshots"). Accepts a fully-qualified dataset.table or project.dataset.table, which overrides --dataset / --bigquery-project-id
• --interval: Snapshot interval (default: 5m)
• --mode: Cost calculation mode: autopilot, standard, or all (default: all)
• --dry-run: Log rows to stdout without writing to BigQuery
• --output-file: Append dry-run snapshots to a local Parquet file (requires --dry-run)
• --prometheus-url: Custom Prometheus API base URL (defaults to GCP Managed Prometheus)
• --exclude-namespaces: Namespaces to exclude (default: kube-system,gmp-system)

A BigQuery project is required: set --bigquery-project-id, use a fully-qualified --table (project.dataset.table), or run where the project can be auto-detected. The recorded project_id column is the auto-detected cluster project (the project where the workloads run), so a central BigQuery dataset can attribute costs across many clusters.

View historical costs

Query BigQuery for recorded cost data:

gke-cost-analyzer history 3d --bigquery-project-id my-gcp-project

View costs across all clusters:

gke-cost-analyzer history 1w --bigquery-project-id my-gcp-project --all-clusters

Options:

• --bigquery-project-id: GCP project ID owning the BigQuery dataset (defaults to the auto-detected environment project; overridden by a fully-qualified --table)
• --dataset: BigQuery dataset name (default: "gke_costs")
• --table: BigQuery table name (default: "cost_snapshots"). Accepts a fully-qualified dataset.table or project.dataset.table, which overrides --dataset / --bigquery-project-id
• --cluster-name: Filter by cluster name (defaults to auto-detected cluster)
• --all-clusters: Show costs from all clusters (adds a CLUSTER column)
• --namespace: Filter to a specific namespace
• --team: Filter by team name

Interactive keys: Enter/a expand/collapse teams, g toggle flat/grouped view, 1-9,0,-,= sort columns, j/k/Up/Down navigate, q quit.

Find pods missing labels

gke-cost-analyzer unmatched-pods

Lists running pods that are missing the configured team or workload labels, grouped by base name (with Kubernetes random suffixes stripped) and namespace. Useful for finding workloads that won't appear in cost attribution.

Other commands

# Print version, commit, and build date
gke-cost-analyzer version

Environment auto-detection

--region, the GCP project, and --cluster-name are automatically detected from the GCE metadata server (when running on GKE) or from the current kubeconfig context. Explicit CLI flags always take priority over detected values. The detected project is the default for both BigQuery (--bigquery-project-id) and Managed Prometheus (--prometheus-project-id); set those flags to point either subsystem at a different project.

Example BigQuery queries

-- Total cost by team for today
SELECT team, SUM(total_cost) as cost
FROM `project.gke_costs.cost_snapshots`
WHERE DATE(timestamp) = CURRENT_DATE()
GROUP BY team;

-- Hourly cost breakdown for a workload
SELECT
  TIMESTAMP_TRUNC(timestamp, HOUR) as hour,
  workload,
  SUM(total_cost) as cost
FROM `project.gke_costs.cost_snapshots`
WHERE DATE(timestamp) = CURRENT_DATE()
  AND team = 'my-team'
GROUP BY hour, workload
ORDER BY hour;

-- Daily cost trend for past week
SELECT
  DATE(timestamp) as day,
  team,
  SUM(total_cost) as cost,
  SUM(CASE WHEN is_spot THEN total_cost ELSE 0 END) as spot_cost
FROM `project.gke_costs.cost_snapshots`
WHERE timestamp >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 7 DAY)
GROUP BY day, team
ORDER BY day;

How it works

1. Pricing: Fetches pricing from the Cloud Billing Catalog API. For Autopilot: pod-level CPU/Memory SKUs from the Kubernetes Engine service. For standard GKE: Compute Engine per-vCPU and per-GB instance SKUs by machine family. Prices are cached locally in ~/.cache/gke-cost-analyzer/ for 24 hours.

2. Pod and node data: Lists running pods (and nodes for standard GKE) from the current kubeconfig context, extracting CPU/memory requests, labels, start time, node placement, and Spot detection.

3. Cost calculation:

   • Autopilot: cost_per_hour = resource_requests × unit_price. No historical data is needed — the hourly rate is derived purely from each pod's resource requests and the region's pricing.
   • Standard GKE: Per-node proportional attribution. Each node's cost (based on its machine type) is distributed to pods proportionally by their resource requests on that node.

4. Aggregation: Costs are grouped by the configured label hierarchy (team, workload, subtype), Spot status, and cost mode (autopilot/standard).

Cost semantics: watch vs record

The two modes display cost differently:

Column	watch	record (BigQuery)
$/HR	Hourly rate based on current resource requests × unit price. Always non-zero for pods with requests.	Same formula, stored as cost_per_hour.
Cost	Cumulative cost since each pod started: cost_per_hour × hours_since_pod_start. Pods running for hours show meaningful totals; just-started pods show ~$0.	Cost for the snapshot interval window only: cost_per_hour × interval_hours. This ensures SUM(total_cost) over a time range equals the actual cost for that period.

Why watch shows values immediately: The TUI computes everything from the current Kubernetes state (pod specs, start times) and cached pricing tables. No prior snapshots or historical data are required. The $/HR column appears instantly because it depends only on resource requests and prices; the Cost column uses each pod's StartTime from Kubernetes to calculate accumulated cost.

Development

# Install tools
mise install

# Install pre-commit hooks
prek install

# Run tests
go test ./...

# Run linter
golangci-lint run

# Build
go build ./...

Releasing

Releases are automated via GitHub Actions. Pushing a v-prefixed tag triggers the release workflow, which runs tests, builds a static linux/amd64 binary, compresses it with gzip, creates a GitHub Release with changelog notes, and pushes a Docker image to ghcr.io/samn/gke-cost-analyzer (tagged with the version and latest). Release artifacts are named gke-cost-analyzer-<os>-<arch>.gz.

Steps

1. Update the changelog. Move items from [Unreleased] in CHANGELOG.md into a new version section:

   ## [X.Y.Z] - YYYY-MM-DD
   
   ### Added
   - ...

   Leave an empty ## [Unreleased] header at the top for future changes.

2. Commit the changelog.

   git add CHANGELOG.md
   git commit -m "Release vX.Y.Z"

3. Tag and push.

   git tag vX.Y.Z
   git push origin main vX.Y.Z

4. Verify. The Release action will build the binary and create the GitHub Release. The release body is extracted automatically from the matching CHANGELOG.md section.

Version injection

Version metadata is injected at build time via -ldflags (see Makefile). The version subcommand prints the tag, short commit hash, and build timestamp. During development the version defaults to dev.

Building locally

make build                          # linux/amd64 (default)
make build GOOS=darwin GOARCH=arm64 # macOS Apple Silicon

Binaries are written to dist/.

License

MIT