Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
19 changes: 18 additions & 1 deletion .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -43,4 +43,21 @@ VERSION=dev
BUILD_ID=docker

# Config source (file, env, consul, etc.)
CONFIG_SOURCE=file
CONFIG_SOURCE=file

# ============================================
# CNPG REMOTE ACCESS (optional)
# ============================================
#
# Publish the CNPG Postgres port on the docker host.
# - Default bind is loopback for safety (not reachable from LAN).
# - To connect from a workstation, set CNPG_PUBLIC_BIND=0.0.0.0 (or a specific LAN IP)
# and ensure your firewall allows inbound traffic on CNPG_PUBLIC_PORT.
# - These variables must be present in `.env` (or exported) when running `docker compose`.
#
# CNPG_PUBLIC_BIND=127.0.0.1
# CNPG_PUBLIC_PORT=5455
#
# If workstation clients connect by IP with CNPG_SSL_MODE=verify-full, ensure the CNPG
# server certificate includes the docker host IP in its SAN.
# CNPG_CERT_EXTRA_IPS=192.168.2.134
6 changes: 6 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -198,6 +198,12 @@ serviceradar-web_*
*.gz

# Ko build system for Go

# Elixir/Phoenix (web-ng/)
deps/
_build/
.elixir_ls/
erl_crash.dump
.kodata

# Generated by Cargo
Expand Down
2 changes: 2 additions & 0 deletions .tool-versions
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
erlang 28.3
elixir 1.19.4
46 changes: 46 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,8 +34,15 @@ ServiceRadar is a multi-component system made up of Go services (core, sync, reg
- `k8s/demo/` – Demo cluster manifests (faker, core, sync, CNPG, etc.).
- `docker/`, `docker/images/` – Container builds and push targets.
- `web/` – Next.js UI and API routes.
- `web-ng/` – Phoenix (next-gen) UI/API monolith.
- `proto/` – Protobuf definitions and generated Go code.

## Per-Directory Agent Guides

This file applies repo-wide, but subdirectories may include their own `AGENTS.md` with more specific rules; always read and follow the closest one to the code you are editing.

- `web-ng/AGENTS.md` – Phoenix/Elixir/LiveView/Ecto/HEEx guidelines (must follow for any `web-ng/**` changes).

## Build & Test Commands

- General Go lint/test: `make lint`, `make test`.
Expand Down Expand Up @@ -80,6 +87,45 @@ Reference `docs/docs/agents.md` for: faker deployment details, CNPG truncate/res
- Restart the stack: `APP_TAG=sha-<sha> docker compose up -d --force-recreate`.
- Verify: `docker compose ps` (one-shot jobs like cert-generator/config-updater exit once finished; nginx may sit in "health: starting" briefly).

## Web-NG Remote Dev (CNPG)

Use this playbook to run `web-ng/` on a workstation while connecting to the existing CNPG instance running on the docker host (example: `192.168.2.134`).

### 1. Publish CNPG on the docker host

- By default, CNPG is bound to loopback only. To allow LAN access, set these in the docker host `.env` (or export them before running compose):
- `CNPG_PUBLIC_BIND=0.0.0.0` (or a specific LAN interface IP)
- `CNPG_PUBLIC_PORT=5455`

### 2. Ensure CNPG TLS cert supports IP-based clients (verify-full)

- If clients will connect by IP with `CNPG_SSL_MODE=verify-full`, add the host IP to the CNPG server cert SAN:
- `CNPG_CERT_EXTRA_IPS=192.168.2.134`
- Regenerate certs: `CNPG_CERT_EXTRA_IPS=192.168.2.134 docker compose up cert-generator`
- Restart CNPG (and ensure bind env vars are applied): `CNPG_PUBLIC_BIND=0.0.0.0 CNPG_PUBLIC_PORT=5455 docker compose up -d --force-recreate cnpg`

### 3. Copy workstation client certs (keep out of git)

- Determine the cert volume name: `docker volume ls | rg 'cert-data'`
- Copy out these files from the volume to a private directory on your workstation:
- `root.pem`
- `workstation.pem`
- `workstation-key.pem`

### 4. Run Phoenix from your workstation

```bash
cd web-ng
export CNPG_HOST=192.168.2.134
export CNPG_PORT=5455
export CNPG_DATABASE=serviceradar
export CNPG_USERNAME=serviceradar
export CNPG_PASSWORD=serviceradar
export CNPG_SSL_MODE=verify-full
export CNPG_CERT_DIR=/path/to/private/serviceradar-certs
mix phx.server
```

## Edge Onboarding Testing with Docker mTLS Stack

Use this playbook to test edge onboarding functionality (e.g., sysmon checker mTLS bootstrap) against the Docker Compose mTLS stack.
Expand Down
25 changes: 13 additions & 12 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

53 changes: 2 additions & 51 deletions cmd/core/app/app.go
Original file line number Diff line number Diff line change
Expand Up @@ -30,13 +30,8 @@ var (

// Options contains runtime configuration derived from CLI flags.
type Options struct {
ConfigPath string
BackfillEnabled bool
BackfillDryRun bool
BackfillSeedKV bool
BackfillIPs bool
BackfillNamespace string
DisableWatch bool
ConfigPath string
DisableWatch bool
}

// Run boots the core service using the provided options.
Expand Down Expand Up @@ -129,15 +124,6 @@ func Run(ctx context.Context, opts Options) error {
return err
}

if opts.BackfillEnabled {
backfillOpts := core.BackfillOptions{
DryRun: opts.BackfillDryRun,
SeedKVOnly: opts.BackfillSeedKV,
Namespace: opts.BackfillNamespace,
}
return runBackfill(ctx, server, mainLogger, backfillOpts, opts.BackfillIPs)
}

apiOptions := bootstrap.BuildAPIServerOptions(&cfg, mainLogger, spireAdminClient)

requireDeviceRegistry := cfg.Features.RequireDeviceRegistry != nil && *cfg.Features.RequireDeviceRegistry
Expand Down Expand Up @@ -218,38 +204,3 @@ func Run(ctx context.Context, opts Options) error {
},
})
}

func runBackfill(ctx context.Context, server *core.Server, mainLogger logger.Logger, opts core.BackfillOptions, includeIPs bool) error {
startMsg := "Starting identity backfill (Armis/NetBox) ..."
if opts.DryRun {
startMsg = "Starting identity backfill (Armis/NetBox) in DRY-RUN mode ..."
}
mainLogger.Info().Msg(startMsg)

if err := core.BackfillIdentityTombstones(ctx, server.DB, server.IdentityKVClient(), mainLogger, opts); err != nil {
return err
}

if includeIPs {
ipMsg := "Starting IP alias backfill ..."
if opts.DryRun {
ipMsg = "Starting IP alias backfill (DRY-RUN) ..."
} else if opts.SeedKVOnly {
ipMsg = "Starting IP alias backfill (KV seeding only) ..."
}
mainLogger.Info().Msg(ipMsg)

if err := core.BackfillIPAliasTombstones(ctx, server.DB, server.IdentityKVClient(), mainLogger, opts); err != nil {
return err
}
}

completionMsg := "Backfill completed. Exiting."
if opts.DryRun {
completionMsg = "Backfill DRY-RUN completed. Exiting."
} else if opts.SeedKVOnly {
completionMsg = "Backfill KV seeding completed. Exiting."
}
mainLogger.Info().Msg(completionMsg)
return nil
}
38 changes: 5 additions & 33 deletions cmd/core/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -55,48 +55,20 @@ import (
"github.com/carverauto/serviceradar/cmd/core/app"
)

type coreFlags struct {
ConfigPath string
Backfill bool
BackfillDryRun bool
BackfillSeedKV bool
BackfillIPs bool
}

func parseFlags() coreFlags {
configPath := flag.String("config", "/etc/serviceradar/core.json", "Path to core config file")
backfill := flag.Bool("backfill-identities", false, "Run one-time identity backfill (Armis/NetBox) and exit")
backfillDryRun := flag.Bool("backfill-dry-run", false, "If set with --backfill-identities, only log actions without writing")
backfillSeedKV := flag.Bool("seed-kv-only", false, "Seed canonical identity map without emitting tombstones")
backfillIPs := flag.Bool("backfill-ips", true, "Also backfill sweep-only device IDs by IP aliasing into canonical identities")
flag.Parse()

return coreFlags{
ConfigPath: *configPath,
Backfill: *backfill,
BackfillDryRun: *backfillDryRun,
BackfillSeedKV: *backfillSeedKV,
BackfillIPs: *backfillIPs,
}
}

func main() {
if err := run(); err != nil {
log.Fatalf("Fatal error: %v", err)
}
}

func run() error {
opts := parseFlags()
configPath := flag.String("config", "/etc/serviceradar/core.json", "Path to core config file")
flag.Parse()

watchEnabled := parseEnvBool("CONFIG_WATCH_ENABLED", true)
appOptions := app.Options{
ConfigPath: opts.ConfigPath,
BackfillEnabled: opts.Backfill,
BackfillDryRun: opts.BackfillDryRun,
BackfillSeedKV: opts.BackfillSeedKV,
BackfillIPs: opts.BackfillIPs,
BackfillNamespace: "",
DisableWatch: !watchEnabled,
ConfigPath: *configPath,
DisableWatch: !watchEnabled,
}

return app.Run(context.Background(), appOptions)
Expand Down
24 changes: 0 additions & 24 deletions cmd/tools/kv-sweep/BUILD.bazel

This file was deleted.

Loading
Loading