[BUG] metricsgeneration/infinite loop

[bmiguel-teixeira]

Component(s)

processor/metricsgeneration

What happened?

Description

When using destination and metric1 with equal values (same metric) the processors enters an infinite loop.

Steps to Reproduce

In a perfect scenario, when using the scaling operations, I would like to change the selected metric "in-place". However, if we set metric name equal name, the processor enters in infinite loop.
Config shown below.

  metricsgeneration/scale:
    rules:
    - name: linux_memory_free_percent_percent
      type: scale
      metric1: linux_memory_free_percent_percent
      operation: multiply
      scale_by: 100

This happens because we iterate over the metrics array, and if we find a atching metric, we scale it based on the operation and we "ADD IT" to the ongoing metrics array. If the metrics names are equal this causes never ending loop.

Expected Result

Replace/update metric in-place. If thiscould be a desired behaviour for the config above, I can submit a PR.

Actual Result

Infinite loop in processor.

Collector version

0.118.0

Environment information

Environment

N/A

OpenTelemetry Collector configuration

Log output

Additional context

No response

[github-actions]

Pinging code owners:

• processor/metricsgeneration: @Aneurysm9

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[VihasMakwana]

@bmiguel-teixeira
From another perspective, this also happens because new metric's name conflicts with old:

  metricsgeneration/scale:
    rules:
    - name: linux_memory_free_percent_percent # new name conflicts with metric1
      type: scale
      metric1: linux_memory_free_percent_percent
      operation: multiply
      scale_by: 100

We can only allow this if names are different? wdyt?

[crobert-1]

I was able to reproduce, the bug is in the generateScalarMetrics method, as the loop is adding data points to the metric, while checking to see if we've scaled all of the data points in the same slice that's being iterated. Since we're adding a new data point for each existing data point in place, the loop never finishes.

From the processor's description:

"The metrics generation processor (metricsgeneration) can be used to create new metrics using existing metrics following a given rule."

From this, I believe the usage here is invalid. and the metrics transform processor would be better suited to this situation.

That being said, I think we should fail on config validation if the new metric name matches the existing metric name. This will make it clear to users that this is unsupported behavior.

[bmiguel-teixeira]

For this particular use case of "just" scaling the metrics transform processor would indeed work. Since I was already using this processor for other operations (percentage) I figured to just re use it for scaling this particular use case.

Additionally, the use case to scaling it "in-place" was just to avoid having to cleanup the "older/temporary" metric. Im okay with assuming the "new metric" behaviour if we lock it behind a config check to avoid this scenario.