Overview

An implementation of the Job Worker Service: a server (jobworker) that runs arbitrary Linux processes on behalf of remote clients over mTLS gRPC, and a CLI (jobctl) that starts, stops, queries, and streams the output of those jobs. See DESIGN.md for the high level design.

Environment

Project has been built and tested on an arch-based linux distro, with: Kernel version:

→ uname -r
7.0.10-1-cachyos

with openssl version:

→ openssl -v
OpenSSL 3.6.3

Go version:

→ go version
go version go1.26.4 linux/amd64

Make version:

→ make --version
GNU Make 4.4.1
Built for x86_64-pc-linux-gnu

Protobuf version:

→ protoc --version
libprotoc 35.1

Build

Build the jobworker server and jobctl CLI into bin/:

make build       # default target

Builds are reproducible: the Go toolchain is pinned in go.mod, cgo is disabled, and -trimpath removes local build paths, so a clean rebuild produces byte-identical binaries.

Other targets:

make all         # build, vet, test & gen certs (if do not already exist)
make proto       # generate the ignored protobuf code (requires protoc)
make certs       # generate dev certificates when missing
make clean       # remove binaries and generated protobuf code

Generated protobuf code is not committed. The build, test, and vet targets generate it when needed, so a clean checkout requires protoc. The protoc-gen-go/protoc-gen-go-grpc plugin versions are pinned through the tool directives in go.mod, keeping generation reproducible.

Development certificates are generated by scripts/create_certs.sh in certs/ - but will be checked in for convenience - including a dev CA, a server certificate for localhost/127.0.0.1, per-user client certificates (client.pem for ross (valid client), other-client.pem for nid (non-valid client)), and unsigned.pem , which does not chain to the CA and should be rejected by the server.

Static Analysis

make vet

Test

Run the unit and end-to-end tests:

make test

and with the race detector:

make test-race   # go test -race -count 5 ./...

Run

Start the server (defaults shown):

./bin/jobworker --listen localhost:8443 --cert certs/server.pem --key certs/server-key.pem --ca certs/ca.pem

Control jobs with the CLI. start expects -- to separate the job's command and arguments from jobctl's own:

→ ./bin/jobctl start -- /bin/sh -c 'for i in 1 2 3; do echo $i; sleep 1; done'
job <id> started

→ ./bin/jobctl status <id>
state: Running

→ ./bin/jobctl output <id>   # replay from start and tail until exit or Ctrl-C
1
2
3

→ ./bin/jobctl stop <id>
job <id> stopped

Global flags: --server (default localhost:8443), --ca, --cert, --key (defaults certs/ca.pem, certs/client.pem, certs/client-key.pem).

Failures exit with code 1 and print the error to stderr, for example an unknown or non-owned job ID, or a client certificate the server does not trust (--cert certs/unsigned.pem).

Future Improvements

• Pre-allocate capacity of job maps, etc to avoid reallocation/copying as the server usage grows
  • This is a premature optimisation at this stage, but would be good for production readiness.
• Full client operation audit logging / monitoring
• Support for shutdown of child processes sub-processes, via PGID
• Full SPIFFE compliance for URI SAN client identity validation
• Child process resource monitoring/control using cgroups