Skip to content

Commit 4da4fa9

Browse files
authored
fix(import): store SmokePing history under the configured target's id (#137)
`smoked import smokeping --apply` mints a UUID onto every imported target, but `--history` stored each RRD's rows under the SmokePing path. The hub resolves a target to its stable id, so the backfilled history never showed on the target's graph or in /api/series. `--history` now looks each SmokePing target up by path in the DB config fragment and, when given, the YAML config passed with `--config`, and stores its rows under that target's id. A target that matches no configured target, or more than one (a path defined in both configs), is named on stderr, counted as unresolved in the summary, and not imported; the run exits 3 as for other partial failures. With no `--config` and no targets in the DB config, rows keep their SmokePing path keys (the id of a YAML target without an explicit id) and the run prints a note. config.TargetIdentities lists each host node's path and storage id exactly as Monitors names them, without inheritance or validation, so a DB fragment can be indexed on its own; Monitors now shares its path and id helpers.
1 parent 1e603e6 commit 4da4fa9

7 files changed

Lines changed: 641 additions & 44 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,16 @@ All notable changes to **Heliograph** are recorded here. The format follows
2828
measurement. A target that only gains an alert on reload is still warm-started. On a cold start,
2929
smoked has no record of a target's previous definition, so a target redefined while smoked was
3030
stopped can still be warm-started from its recent old rounds.
31+
- **`smoked import smokeping --history` now stores history where the target's graph reads it.**
32+
`--apply` gives every imported target a new stable id, but `--history` stored each RRD's rows
33+
under the SmokePing path, so the backfilled history never appeared on the configured target's
34+
graph or in `/api/series`. `--history` now looks each target up by path in the DB config and, when
35+
given, the YAML config passed with `--config`, and stores its rows under that target's id. A target
36+
that matches no configured target, or more than one, is named on stderr, counted as `unresolved`
37+
in the summary, and not imported; the run then exits `3`, the same as other partial failures.
38+
With no `--config` and no targets in the DB config, rows are still keyed by SmokePing path, which
39+
matches a YAML target that has no `id`, and the run says so. History already imported under
40+
SmokePing paths is not moved; re-run `--history` to store it under the target ids.
3141

3242
### Security
3343
- **Revoking and re-adding a vantage now retires its earlier certificates.** Agent authorization

‎README.md‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -245,6 +245,33 @@ SMOKE_TEST_DSN='postgres://smoke:smoke@127.0.0.1:5433/smoke?sslmode=disable' \
245245

246246
The schema (one `samples` hypertable) is created automatically on first connect.
247247

248+
### Migrating from SmokePing
249+
250+
`smoked import smokeping <config-dir>` converts a SmokePing `Targets`/`Probes`/`Database` config
251+
into a Heliograph target tree. Import the config first, then the RRD history:
252+
253+
```sh
254+
DSN='postgres://user:pass@host:5432/smoke?sslmode=disable'
255+
smoked import smokeping /etc/smokeping/config --out smokeping.yaml # review the converted tree
256+
smoked import smokeping /etc/smokeping/config --apply --dsn "$DSN" # merge it into the DB config
257+
smoked import smokeping /etc/smokeping/config --report # dry run: which targets have .rrd files
258+
smoked import smokeping /etc/smokeping/config --history --dsn "$DSN" # backfill the RRD history
259+
```
260+
261+
`--history` needs `rrdtool` and a database where downsampling is already on (`smoked -downsample`).
262+
It reads the RRD files from `<config-dir>/../data` or `<config-dir>/data`; pass `--data` to point
263+
elsewhere. Each SmokePing target's history is stored under the id of the configured target at the
264+
same path, so it shows on that target's graph. `--apply` gives every imported target a new id, so
265+
run `--history` after `--apply`. If you put the converted tree in your YAML config instead (for
266+
example as a `conf.d/` file), pass that config with `--config` so explicit `id`s there are honored.
267+
With no `--config` and no targets in the DB config, history is stored under the SmokePing path,
268+
which is the id of a YAML target with no `id`.
269+
270+
`--history` skips a target and names it on stderr when no configured target has its path (renamed
271+
or moved since the config import) or when more than one does. It also skips a target whose ping
272+
count can't be resolved or whose RRD can't be read. The other targets are still imported, and the
273+
run exits `3` if any target was skipped for one of these reasons.
274+
248275
### Federation deployment (reverse proxy)
249276

250277
Remote **vantages** (the `smoke-agent` collector) reach the hub over two independent surfaces

‎ROADMAP.md‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -157,7 +157,10 @@ Phase-6 items all ship in the single **v1.0** line.
157157
`data` subdir); `--history` extracts each matched target's full RRD history via `rrdtool` and
158158
backfills median/loss into `samples` + the hourly/daily aggregates, idempotently. Requires the
159159
continuous aggregates already enabled (`smoked -downsample`) — refuses to import (no rows
160-
written) otherwise, so old history is never left unmaterialized ahead of raw retention
160+
written) otherwise, so old history is never left unmaterialized ahead of raw retention.
161+
Rows are stored under the stable id of the configured target at the same path (DB config,
162+
plus the YAML config given with `--config`); a target with no single match is reported and
163+
skipped (Unreleased)
161164
- ✅ **Native ICMP probe** — a new `Ping` probe kind (`internal/probe/pingprobe`) speaks ICMP
162165
echo itself via `golang.org/x/net/icmp`: datagram socket first (unprivileged, needs the
163166
`net.ipv4.ping_group_range` sysctl), raw-socket fallback (`CAP_NET_RAW`). Ships alongside

0 commit comments

Comments
 (0)