Skip to content

feat(prometheus): add histogram_Mode toggle for classic, native, and dual ingestion.#2818

Open
jacobhallburns wants to merge 3 commits intoprometheus-operator:mainfrom
jacobhallburns:feature/histogram-implementation
Open

feat(prometheus): add histogram_Mode toggle for classic, native, and dual ingestion.#2818
jacobhallburns wants to merge 3 commits intoprometheus-operator:mainfrom
jacobhallburns:feature/histogram-implementation

Conversation

@jacobhallburns
Copy link
Copy Markdown

@jacobhallburns jacobhallburns commented Apr 2, 2026

Description

This implementation introduces a histogramMode toggle within prometheus.libsonnet, providing the universal control requested in Issue #2784. By moving away from legacy bucket-based histograms at the library level, the solution maintains the integrity of the kube-prometheus manifests as the source of truth.

The implementation has been refactored to follow the maintainer's feedback: intermediate variables have been removed, and the logic is now applied directly within the resource specifications.

Key Features:

  • Operational Modes: Supports classic (default), native (high-resolution sparse histograms), and dual (parallel ingestion).
  • Library-Level Implementation: Uses std.map to ensure consistent histogram settings across all ServiceMonitor endpoints while maintaining a DRY codebase.
  • Migration Safety: The dual mode allows for parallel ingestion of both formats, ensuring backward compatibility for existing dashboards during the transition.

Closes #2784

Type of change

  • CHANGE (fix or feature that would cause existing functionality to not work as expected)
  • FEATURE (non-breaking change which adds functionality)
  • BUGFIX (non-breaking change which fixes an issue)
  • ENHANCEMENT (non-breaking change which improves existing functionality)
  • NONE (if none of the other choices apply. Example, tooling, build system, CI, docs, etc.)

Development & Testing Guide

To provide a reproducible testing environment, I utilized a Kubernetes-in-Docker (Kind) cluster and a specialized toolchain container.

Initial Setup

# Build the toolchain image (contains jsonnet, jb, and gojsontoyaml)
docker build -t kube-prom-builder .

# When building the initial Docker, or after each histogramMode change
docker run --rm -v $(pwd):/app --entrypoint "/bin/bash" kube-prom-builder -c \
"jb install && \
jsonnet -J /app/vendor -J /app/jsonnet -m manifests example.jsonnet | \
xargs -I{} sh -c 'cat {} | gojsontoyaml > {}.yaml'"

# Create the Kind cluster
kind create cluster --name prometheus-test

# Deploy base manifests (Apply CRDs first)
kubectl apply --server-side -f manifests/setup/
kubectl apply -f manifests/

# Monitor the rollout
kubectl rollout status statefulset prometheus-k8s -n monitoring

# Port Forwarding
kubectl port-forward -n monitoring svc/prometheus-k8s 9090:9090

Changelog entry

Add histogramMode toggle to support classic, native, and dual histogram ingestion modes across the stack.

@jacobhallburns jacobhallburns requested a review from a team as a code owner April 2, 2026 04:18
Comment thread jsonnet/kube-prometheus/components/prometheus.libsonnet Outdated
Comment thread jsonnet/kube-prometheus/main.libsonnet Outdated
Comment thread example.jsonnet
Comment thread jsonnet/kube-prometheus/main.libsonnet Outdated
Comment thread jsonnet/kube-prometheus/components/prometheus.libsonnet Outdated
Comment thread jsonnet/kube-prometheus/components/prometheus.libsonnet Outdated
Comment thread jsonnet/kube-prometheus/components/prometheus.libsonnet Outdated
Comment on lines +23 to +24
enableNativeHistograms: self.histogramMode == 'native' || self.histogramMode == 'dual',
enableClassicHistograms: self.histogramMode == 'classic' || self.histogramMode == 'dual',
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do we need these?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed. These were redundant intermediate variables; the logic has been moved directly into the resource specifications to simplify the component's internal API.

Comment on lines +25 to +26
scrapeNativeHistograms: self.histogramMode == 'native' || self.histogramMode == 'dual',
scrapeClassicHistograms: self.histogramMode == 'classic' || self.histogramMode == 'dual',
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we avoid the intermediate variables and move this where it's used directly?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. I have eliminated the intermediate variables and moved the logic directly into the Prometheus spec and ServiceMonitor endpoints.

I used std.map for the ServiceMonitor endpoints to keep the implementation DRY and consistent. I've verified the fix in a local kind cluster—the generated configuration correctly enables scrape_native_histograms across all targets and I've confirmed native samples are being ingested in the TSDB.

…nent

This change addresses reviewer feedback by moving the histogram mode logic directly into the Prometheus spec and ServiceMonitor endpoints. By using std.map for endpoints, the implementation remains DRY while avoiding unnecessary top-level internal variables.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Allow using native histograms for all components

2 participants