fix: count GetMetricData billed metrics per AWS pricing, not per query - #1919
Open
pujitha24 wants to merge 1 commit into
Open
fix: count GetMetricData billed metrics per AWS pricing, not per query#1919pujitha24 wants to merge 1 commit into
pujitha24 wants to merge 1 commit into
Conversation
Motivation: yace_cloudwatch_getmetricdata_metrics_requested_total is documented as "how AWS bills" for the GetMetricData API, but it was implemented as a raw count of MetricDataQuery entries in each API call. YACE emits one MetricDataQuery per requested statistic of a metric, so a metric configured with 5 statistics incremented this counter by 5, even though AWS's GetMetricData pricing batches up to 5 statistics of the same metric (namespace + name + dimensions) into a single billed metric request. As a result this counter could overstate real GetMetricData costs by up to 5x whenever more than one statistic was requested per metric, making it unusable for the cost estimation it exists for. Approach: Added billedGetMetricDataMetricsCount, which groups the CloudwatchData requested in a GetMetricData call by metric identity (namespace, metric name, sorted dimensions) and sums ceil(statistics / 5) per group, matching AWS's documented batching. This only changes the value fed into the CloudwatchGetMetricDataAPIMetricsCounter metric; it does not change any CloudWatch API calls, exported metric data, or scrape behavior. Validation: - go build ./... - go test ./pkg/... (all packages pass, including new Test_billedGetMetricDataMetricsCount cases covering 1, exactly 5, and 6 statistics per metric, and distinct metrics/dimensions) - make lint (golangci-lint, 0 issues) - make build This was validated at the unit-test level against AWS's documented GetMetricData billing rule (up to 5 statistics per metric batched into one billed metric request); it was not validated against a live AWS bill, which this repo's CONTRIBUTE.md does not require for a change of this kind. Report: prometheus-community#1806 Signed-off-by: Pujitha Paladugu <10557236+pujitha24@users.noreply.github.com>
Author
|
This has been sitting for a bit and is still rebased cleanly on master with DCO passing — happy to adjust anything if it would help move review along. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation:
yace_cloudwatch_getmetricdata_metrics_requested_total is documented as
"how AWS bills" for the GetMetricData API, but it was implemented as a
raw count of MetricDataQuery entries in each API call. YACE emits one
MetricDataQuery per requested statistic of a metric, so a metric
configured with 5 statistics incremented this counter by 5, even
though AWS's GetMetricData pricing batches up to 5 statistics of the
same metric (namespace + name + dimensions) into a single billed
metric request. As a result this counter could overstate real
GetMetricData costs by up to 5x whenever more than one statistic was
requested per metric, making it unusable for the cost estimation it
exists for.
Approach:
Added billedGetMetricDataMetricsCount, which groups the CloudwatchData
requested in a GetMetricData call by metric identity (namespace, metric
name, sorted dimensions) and sums ceil(statistics / 5) per group,
matching AWS's documented batching. This only changes the value fed
into the CloudwatchGetMetricDataAPIMetricsCounter metric; it does not
change any CloudWatch API calls, exported metric data, or scrape
behavior.
Validation:
Test_billedGetMetricDataMetricsCount cases covering 1, exactly 5,
and 6 statistics per metric, and distinct metrics/dimensions)
This was validated at the unit-test level against AWS's documented
GetMetricData billing rule (up to 5 statistics per metric batched into
one billed metric request); it was not validated against a live AWS
bill, which this repo's CONTRIBUTE.md does not require for a change of
this kind.
Report: #1806
Signed-off-by: Pujitha Paladugu 10557236+pujitha24@users.noreply.github.com
Fixes #1806