Add remote write protocol 2.0 support

[dimitarvdimitrov]

What should we do?

Prometheus added remote write protocol 2.0 experimental support in v2.54.0 (released on 2024-08-09). We should add the support in Mimir too.

How will we do it (roughly)?

• Add remote write 2.0 support in Mimir distributors
• Backport all (applicable) optimizations we did to remote write 1.0 (un)marshalling
• Compare performance between remote write 1.0 and 2.0
• Allow to enable 2.0 support on a per-tenant basis

Out of the scope of this work:

• Change data format between distributors and ingesters / Kafka (keep using protocol 1.0 format). This should be a follow up deliverable.

Size?

Between Medium (= ~1 month) and Large (= ~3 month).

What will we deliver?

• Add remote write 2.0 experimental support in Mimir, fully merged in Mimir but disabled by default
• Test remote write 2.0 in dev

What are the documentation dependencies?

• No documentation changes to docs until enabled by default

Urgency?

Not urgent yet, but we can't lag too much behind Prometheus

[jmichalek132]

FYI otel collector contrib has lfx mentorship project starting in September to add support in the remote write exporter for remote write 2.0. Tracking issue: open-telemetry/opentelemetry-collector-contrib#33661.