configstream: let clients ingest streamed env-var-sourced settings (#55310) - #55612
Open
rahulkaukuntla wants to merge 1 commit into
Open
configstream: let clients ingest streamed env-var-sourced settings (#55310)#55612rahulkaukuntla wants to merge 1 commit into
rahulkaukuntla wants to merge 1 commit into
Conversation
…55310) Adds `nodetreemodel.DirectBulkSet` to `model.Writer`, taking a whole resolved snapshot (`[]model.DirectSetting`) in one pass. Unlike `Set` it accepts `SourceEnvVar` and skips change notifications, so it exists for config streaming and nothing else should call it. A key absent from the client's schema is warned and stored rather than dropped, matching the YAML loader, so the client mirrors the sender. Incremental updates still go through `Set`, so notifications continue to reach registered receivers. Remote agents (trace-agent, process-agent, security-agent, system-probe) disable their own env-var layer and instead mirror the core agent's resolved config over configstream, keeping each setting's original source. Env vars are set on the core agent's process, so a faithful mirror has to write those settings as `SourceEnvVar` — but `nodetreemodel.Set` refuses to write that layer, since it's normally populated by reading the process's own environment. So today every env-var-sourced setting in a snapshot is dropped: the remote agents silently diverge from the core agent's config, and it was one of the largest sources of configstream error-log noise in staging. The fix is a separate ingestion path that accepts the real source, leaving `Set`'s guardrail in place for direct callers. Ran a real configstream session locally: core agent with `DD_LOG_LEVEL`, `DD_TAGS`, `DD_APM_ENV` and `DD_APM_MAX_MEMORY` set, and a trace-agent started with an empty environment (the consumer disables its own env-var layer, so anything env-sourced it ends up with can only have come from the stream). - 2136 settings applied in ~84ms, zero `"Writing to env var layers is not allowed..."` errors. - All four landed: the trace-agent logs at DEBUG, and `/info` reports `default_env = livetest-env` (from the nested `apm_config.env`) and `max_memory = 300000000`. - Live updates still take effect: `agent config set log_level info` arrived as a `ConfigUpdate` and silenced the trace-agent's DEBUG lines. - `TestDirectBulkSet` pins precedence (a `SourceFile` write loses to an existing `SourceAgentRuntime` value in the same call) and unknown-key storage. Also ran `dda inv linter.go`, `dda inv test` and `dda inv tidy`, all passing. Co-authored-by: rahul.kaukuntla <rahul.kaukuntla@datadoghq.com> (cherry picked from commit c91f57a)
Contributor
|
Backport evaluation: View Slack thread |
rahulkaukuntla
added
qa/done
aiuto
reviewed
Aug 30, 2026
aiuto
left a comment
aiuto
left a comment
Collaborator
There was a problem hiding this comment.
Meta question: Why are we backporting this?
- We are a week away from the code freeze for the next release, so we are very late in the current release cycle.
- There is no bug attached to either this PR or the one it is picked from. So why any urgency?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Adds
nodetreemodel.DirectBulkSettomodel.Writer, taking a whole resolved snapshot ([]model.DirectSetting) in one pass. UnlikeSetit acceptsSourceEnvVarand skips change notifications, so it exists for config streaming and nothing else should call it. A key absent from the client's schema is warned and stored rather than dropped, matching the YAML loader, so the client mirrors the sender. Incremental updates still go throughSet, so notifications continue to reach registered receivers.Remote agents (trace-agent, process-agent, security-agent, system-probe) disable their own env-var layer and instead mirror the core agent's resolved config over configstream, keeping each setting's original source. Env vars are set on the core agent's process, so a faithful mirror has to write those settings as
SourceEnvVar— butnodetreemodel.Setrefuses to write that layer, since it's normally populated by reading the process's own environment.(cherry picked from commit c91f57a)
Motivation
Describe how you validated your changes
Additional Notes