Skip to content

windows service install: no SCM dependency on network readiness, agent can start before the network is usable #19454

Description

@john-behm-bertelsmann

Relevant command

telegraf.exe service install --config telegraf.conf --service-name my-telegraf

No flag exists to attach a Windows service dependency (e.g. on NlaSvc) or to request
delayed auto-start.

System info

Telegraf 1.39.x, Windows 11 Pro. cmd/telegraf/telegraf_windows.go on master installs
the service identically, so the behaviour is unchanged as of writing.

Steps to reproduce

[agent]
  interval       = "10s"
  flush_interval = "10s"
  logfile        = "C:/telegraf-network-race/telegraf.log"

[[inputs.cpu]]

[[outputs.http]]
  # Replace with a reachable HTTP endpoint before installing.
  url         = "https://your-endpoint.example/ingest"
  method      = "POST"
  data_format = "json"
  1. Create C:\telegraf-network-race\repro.conf with the config above, replacing the URL
    with an endpoint that normally works from the host.

  2. In an elevated PowerShell, install the service:

    telegraf.exe --config C:\telegraf-network-race\repro.conf --service-name repro-netrace service install

    service install registers an automatic-start service; do not start it manually.

  3. Reboot the host.

  4. After Windows is back, compare the first write failure with DHCP readiness:

    Get-Content C:\telegraf-network-race\telegraf.log | Select-String "dial tcp|no such host"
    Get-WinEvent -LogName Microsoft-Windows-Dhcp-Client/Operational -MaxEvents 20 |
      Where-Object { $_.Id -eq 50036 }

    On the affected host, the first Telegraf write error predates the DHCP lease event.

  5. Remove the test service when finished:

    telegraf.exe --service-name repro-netrace service uninstall

Expected behavior

The service either does not attempt a network-dependent write until the network is
actually usable, or is not started by the SCM until that point.

Actual behavior

On a normal boot, Telegraf's Windows service is started well before the network stack is
guaranteed to be usable (DHCP lease / DNS not ready yet, especially with a domain join or
NAC in the mix). The very first output write(s) then fail with a plain dial tcp / no such host style error — nothing is wrong with the destination, the local network simply
was not ready yet at the moment the SCM launched the process.

Root cause

cmd/telegraf/telegraf_windows.go, function installService():

svccfg := mgr.Config{
    DisplayName: cfg.displayName,
    Description: "Collects, processes and publishes data using a series of plugins.",
    StartType:   mgr.StartAutomatic,
    ServiceType: windows.SERVICE_WIN32_OWN_PROCESS,
}

mgr.Config (from golang.org/x/sys/windows/svc/mgr) has a Dependencies []string field,
but it is never populated here, and StartType is always plain mgr.StartAutomatic
never delayed-auto-start. Neither telegraf.exe service install nor any documented flag
lets an operator supply a dependency or request delayed start. As a result the SCM is free
to launch telegraf.exe as soon as the generic "automatic" start phase reaches it,
independent of whether the network stack has actually come up.

This is not unique to Telegraf — it is a generic Windows SCM behavior — but Telegraf does
nothing to protect against it either, unlike services that add an explicit dependency on
NlaSvc (Network Location Awareness) for exactly this reason.

Suggested fix

Two independent, non-exclusive options:

  1. Add a --service-dependencies flag (comma- or repeat-flag separated) to
    telegraf.exe service install, threaded into mgr.Config.Dependencies, so operators
    can opt in to e.g. --service-dependencies NlaSvc for hosts where this matters. Minimal
    change: the field already exists on mgr.Config, it is just never set from a flag.
  2. Optionally default Dependencies to ["NlaSvc"] when the loaded configuration contains
    any network output (outputs.http, outputs.influxdb, etc.), skippable via a
    --no-service-dependencies flag for hosts that must not wait on it (e.g. no network at
    all, or an already-verified fast-attach NIC).

Delayed auto-start (SERVICE_CONFIG_DELAYED_AUTO_START_INFO via ChangeServiceConfig2) is
a coarser alternative to a real dependency and was not pursued here since it delays the
service by a fixed, unrelated-to-network amount rather than an actual readiness signal.

Impact

  • Any Telegraf Windows service with a network output can log at least one spurious
    connection-error write failure on every boot, which is indistinguishable from a real
    outage to anything alerting on it (self-monitoring write-error counters, log scraping,
    etc.) without host-side workarounds.
  • No configuration-level workaround exists inside telegraf.conf — this is strictly an
    SCM service-property gap, set once at service install time.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions