Lean, secure, automated, zero downtime*, poor man's infra for services running in docker.
Running a home network? Then you may already have a custom setup, probably using docker compose. You might enjoy all the maintenance and tinkering, but you are surely aware of the pitfalls and potential downtime. If you think that is ok, or if you don't want automation, then this stack is probably not for you. Still interested? Then read on...
Table of contents:
- Documentation
- Key concepts
- Apps included
- Prerequisites
- Dev/ops tools
- Howto
- Questions one might have
- Disclaimer
Comprehensive documentation is available in the docs/ directory:
Getting Started:
- Architecture Overview - System architecture and design principles
- Networking - Network topology and configuration
Stacks:
- DNS Stack - DNS honeypot management
- Proxy Stack - Traefik configuration, routing, and TLS
- API Stack - Management API architecture and deployment
Operations:
- Logging - Log management and rotation
- Monitoring - Container security monitoring
- Backups - Backup and disaster recovery
- Deployment - Deployment procedures and best practices
Development:
- Project Structure - Codebase organization
- Configuration - Configuration guide with schemas
- Testing - Testing strategies and practices
Reference:
- CLI Reference - Complete command-line reference
- Environment Variables - All environment variables
- Troubleshooting - Common issues and solutions
For a complete documentation index, see docs/README.md.
The projects/ directory is used for all the infra and workloads it creates and manages, to ensure a predictable and reliable automated workflow. Each project has its own docker-compose.yml and ingress.yml files, while infrastructure configuration lives in projects/itsup.yml and projects/traefik.yml.
This project-based structure provides both flexibility and reliability, and we strive to mirror standard docker compose functionality, which means no concessions are necessary from a docker compose enthusiast's perspective.
itsUP generates and manages proxy/docker-compose.yml which operates Traefik in a zero-downtime configuration:
Architecture:
- Traefik using Linux SO_REUSEPORT for zero-downtime updates (defaults to 1 replica)
- Host networking mode allows multiple instances to bind to same ports simultaneously
- Kernel load-balances incoming connections between scaled instances
- Users can scale manually:
docker compose up -d --scale traefik=N(e.g., N=2 for high availability) - Docker socket access secured via dockerproxy (wollomatic/socket-proxy) on localhost
Capabilities:
- Terminate TLS and forward tcp/udp traffic over an encrypted network to listening endpoints
- Passthrough TLS to endpoints (most people have secure Home Assistant setups already)
- Open host ports if needed to choose a new port (openvpn service does exactly that)
- Zero-downtime configuration updates via rolling deployment
itsUP generates and manages upstream/{project}/docker-compose.yml files to deploy container workloads based on your project configurations in projects/{project}/. Each project has a docker-compose.yml (service definitions) and ingress.yml (routing configuration).
This centralizes and abstracts away the plethora of custom docker compose setups that are mostly uniform in their approach anyway, so controlling their artifacts from a project-based structure makes a lot of sense.
Like with all docker orchestration platforms (even Kubernetes) this is dependent on the containers:
- are healthchecks correctly implemented?
- Are SIGHUP signals respected to shutdown within an acceptable time frame?
- Are the containers stateless?
Smart Change Detection:
itsUP implements smart change detection that only performs rollouts when necessary:
- Compares Docker Compose config hashes (stored in container labels)
- Detects image updates, environment changes, volume changes, etc.
- Skips rollout if nothing changed (instant operation)
- Automatically performs zero-downtime rollout when changes detected
Rollout Process:
When changes are detected, itsUP will rollout via docker rollout:
- Scale to double the current replicas (1→2, 2→4, etc. depending on your scale setting)
- Wait for new containers to be healthy (max 60s with healthcheck, otherwise 10s)
- Kill old containers and wait for drain
- Remove old containers, leaving the same number of new replicas running
Cost: ~15 seconds when changes detected, instant when nothing changed.
What about stateful services?
It is surely possible to deploy stateful services but beware that those might not be good candidates for the docker rollout automation. In order to update those services it is strongly advised to first read the upgrade documentation for the newer version and follow the prescribed steps. More mature databases might have integrated these steps in the runtime, but expect that to be an exception. So, to garner correct results you are on your own and will have to read up on your chosen solutions.
- traefik/traefik: the famous L7 routing proxy that manages letsencrypt certificates
- minio/minio: S3 storage
- nubacuk/docker-openvpn: vpn access to the host running this stack
- traefik/whoami: to demonstrate that headers are correctly passed along
Tools:
Infra:
- Portforwarding of port
80and443to the machine running this stack. This stack MUST overtake whatever routing you now have, but don't worry, as it supports your home assistant setup and forwards any traffic it expects to it (configure your home-assistant project inprojects/home-assistant/) - A wildcard dns domain like
*.itsup.example.comthat points to your home ip. This allows to choose whatever subdomain for your services. You may of course choose and manage any domain in a similar fashion for a public service, but I suggest not going through such trouble for anything private.
The itsup CLI is the main interface for managing your infrastructure. It provides smart change detection and zero-downtime deployments:
Main commands:
# Initialization
itsup init # Initialize installation (clone repos, copy samples, setup git integration)
# Orchestrated Operations
itsup run # Run complete stack (orchestrated: dns→proxy→api→monitor in report-only mode)
# Stack-Specific Operations
# Every stack follows the same pattern: up, down, restart, logs [service]
itsup dns up # Start DNS stack
itsup dns down # Stop DNS stack
itsup dns restart # Restart DNS stack
itsup dns logs # Tail DNS stack logs
itsup proxy up # Start proxy stack
itsup proxy up traefik # Start only Traefik
itsup proxy down # Stop proxy stack
itsup proxy restart # Restart proxy stack
itsup proxy logs # Tail all proxy logs
itsup proxy logs traefik # Tail Traefik logs only
# Configuration & Deployment
itsup apply [project] # Apply configurations with smart zero-downtime updates
itsup validate [project] # Validate project configurations
itsup migrate # Migrate configuration schema to latest version
# Project Service Management
itsup svc <project> <cmd> [service] # Docker compose operations for project services
itsup svc <project> up # Start all services in a project
itsup svc <project> logs -f # Tail project logs
itsup svc <project> exec web sh # Execute commands in containers
# Container Security Monitor
itsup monitor start [--flags] # Start security monitor
itsup monitor stop # Stop monitor
itsup monitor logs # Tail monitor logs
itsup monitor cleanup # Review blacklist
itsup monitor report # Generate threat report
# Options
itsup --version # Show version
itsup -v # Enable DEBUG logging
itsup -vv # Enable TRACE logging (very verbose)Schema Versioning & Migrations:
itsUP tracks configuration schema versions to ensure compatibility between the CLI and your configuration files. When you upgrade itsUP to a newer version that includes schema changes, you'll need to run a migration:
# Check if migration is needed (automatic on most commands)
itsup validate # Will warn if schema is outdated
# Dry-run to see what would change
itsup migrate --dry-run # Preview changes without applying
# List pending migrations
itsup migrate --list # Show which fixers would run
# Run migration
itsup migrate # Upgrade configuration schemaMigration Features:
- Idempotent fixers (safe to run multiple times)
- Git-aware file operations (uses
git mvto preserve history) - Automatic validation after migration
- Version tracked in
projects/itsup.yml(schemaVersionfield)
Smart Output:
itsUP automatically adapts its output based on context:
-
Interactive terminal (TTY): Clean colored output with symbols
âś“ Migration complete! âš Config needs review âś— Failed to rename project -
Pipes/logs/automation: Full structured output with timestamps
2025-11-04 00:11:48.166 INFO lib/migrations.py:97 Migration complete! 2025-11-04 00:11:48.166 WARNING lib/migrations.py:71 Config needs review 2025-11-04 00:11:48.166 ERROR lib/fixers/rename_ingress.py:62 Failed to rename project
Smart Behavior:
applycommand uses config hash comparison - only performs rollouts when changes detected- Tab completion for project names, docker compose commands, and service names
- All commands work from project root (no need to cd into directories)
- Zero-downtime rollouts only happen when actual changes detected
Directory → Command Mapping:
dns/docker-compose.yml→itsup dnsproxy/docker-compose.yml→itsup proxyupstream/project/→itsup svc project
Orchestrated vs Stack Operations:
itsup run= Full orchestrated startup (dns→proxy→api→monitor in report-only mode) in correct dependency orderitsup dns up= Stack-specific operation (just DNS)itsup proxy up= Stack-specific operation (just proxy)- Different semantics:
rundoes everything, stack commands are surgical
Note: itsup run starts the monitor in report-only mode (detection without blocking). For full protection with blocking, use itsup monitor start.
bin/write_artifacts.py: after updating project configurations inprojects/you can run this script to generate new artifacts (or useitsup applywhich does this automatically).bin/requirements-update.sh: You may want to update requirements once in a while ;)
A minimal Makefile focused on development workflow. Run make help to see all available targets:
make install # Install dependencies (calls itsup init)
make test # Run all tests
make lint # Run linter
make format # Format code
make clean # Remove generated artifactsFor runtime operations (run/dns/proxy/svc/monitor), use itsup commands instead. The Makefile is intentionally minimal to avoid command sprawl.
All container DNS traffic is routed through a DNS honeypot (dns-honeypot container) that logs all queries and responses. This is essential for the container security monitoring system.
The DNS honeypot is managed via proxy/docker-compose-dns.yml and runs dnsmasq with query logging enabled. It integrates with the proxy network to intercept all container DNS requests.
Real-time container security monitoring that detects compromised containers by identifying hardcoded IP connections through DNS correlation analysis.
Key Features:
- DNS correlation detection (connections without DNS = malware)
- Real-time Docker events integration
- OpenSnitch cross-reference (optional)
- iptables blocking (optional)
- Automatic IP list management with hot-reload
- Historical analysis with timestamp resumption
Quick Start:
# Start monitor with full protection (blocking enabled)
itsup monitor start --use-opensnitch
# Detection only (no blocking) - also started automatically by "itsup run"
itsup monitor start --report-only --use-opensnitch
# Stop monitor
itsup monitor stop
# View logs
itsup monitor logs
# Generate threat intelligence report
itsup monitor reportNote: itsup run automatically starts the monitor in report-only mode for safe operation during infrastructure startup. To enable active blocking, explicitly run itsup monitor start.
đź“– For complete documentation, see monitor/README.md
This includes:
- Architecture and detection logic
- Configuration options
- False positive handling
- Testing guide
- Performance characteristics
itsUP uses separate git repositories for configuration and secrets management:
Create the repositories:
# Create a private repository for your project configurations
# (on GitHub, GitLab, or your preferred git host)
# Name it something like: itsup-projects
# Create a private repository for your secrets
# Name it something like: itsup-secretsClone itsUP:
git clone https://github.com/Morriz/itsUP.git
cd itsUPWhy separate repositories?
projects/: Contains your service configurations (YAML files). This keeps your infrastructure-as-code separate from the itsUP codebase, allowing you to manage and version your configurations independently.secrets/: Contains encrypted secrets (using SOPS). Keeping secrets in a separate repository improves security and access control.
Both repositories are gitignored in the main itsUP repo and managed as independent git repositories.
The itsup init command will:
- Prompt for git URLs and clone
projects/andsecrets/repositories - Copy sample configuration files (won't overwrite existing files)
- Set up Python virtual environment
- Install dependencies
itsup initThe script will copy sample files to:
.env(environment variables)projects/traefik.yml(base Traefik configuration)secrets/itsup.txt(template for required secrets)
Edit the copied files:
.env: Configure environment variables as neededprojects/traefik.yml: Changedomain_suffixto your domainsecrets/itsup.txt: Fill in ALL required secrets (validation will fail if any are empty)
# Encrypt secrets with SOPS
cd secrets
sops -e itsup.txt > itsup.enc.txt
# Commit to your secrets repository
git add itsup.enc.txt
git commit -m "Initial secrets"
git push
# Commit your project configuration
cd ../projects
git add traefik.yml
git commit -m "Initial configuration"
git push# Apply your configuration with zero-downtime updates
itsup apply# Tail all logs
bin/tail-logs.sh
# Or use make
make logsProject and service configuration uses a project-based structure in the projects/ directory. Each project has its own subdirectory with docker-compose.yml and ingress.yml files.
Create a new directory in projects/ with two files:
projects/whoami/docker-compose.yml:
services:
web:
image: traefik/whoami:latest
restart: unless-stopped
networks:
- proxynet
networks:
proxynet:
external: trueprojects/whoami/ingress.yml:
enabled: true
ingress:
- service: web
domain: whoami.example.com
port: 80
router: httpRun itsup apply whoami to generate artifacts and deploy the service.
Create a project with passthrough: true in the ingress configuration.
projects/home-assistant/ingress.yml:
enabled: true
ingress:
- service: hass
domain: home.example.com
port: 443
router: http
passthrough: trueprojects/home-assistant/docker-compose.yml:
services:
hass:
image: homeassistant/home-assistant:latest
restart: unless-stopped
networks:
- proxynet
ports:
- '8123:8123'
networks:
proxynet:
external: trueIf you also need port 80 for HTTP challenges, add another ingress entry:
ingress:
- service: hass
domain: home.example.com
port: 443
router: http
passthrough: true
- service: hass
domain: home.example.com
port: 80
router: http
path_prefix: /.well-known/acme-challenge/(Port 80 is only allowed for ACME challenges with passthrough.)
Use router: tcp in the ingress configuration.
projects/minio/docker-compose.yml:
services:
app:
image: minio/minio:latest
command: server --console-address ":9001" /data
environment:
MINIO_ROOT_USER: ${MINIO_ROOT_USER}
MINIO_ROOT_PASSWORD: ${MINIO_ROOT_PASSWORD}
volumes:
- /data/minio:/data
restart: unless-stopped
networks:
- proxynet
networks:
proxynet:
external: trueprojects/minio/ingress.yml:
enabled: true
ingress:
- service: app
domain: minio-api.example.com
port: 9000
router: tcp
- service: app
domain: minio-ui.example.com
port: 9001
router: httpprojects/minio/secrets.txt: (optional project-specific secrets)
MINIO_ROOT_USER=admin
MINIO_ROOT_PASSWORD=secret123
For services running on the host (not in Docker), create a minimal ingress-only configuration.
projects/itsup/ingress.yml:
enabled: true
ingress:
- service: api
domain: itsup.example.com
port: 8888
router: http
target: 172.17.0.1 # Docker bridge IP (use host.docker.internal on Docker Desktop)projects/itsup/docker-compose.yml:
# Empty or minimal - no containers needed for host services
services: {}Each project directory follows this pattern:
projects/
└── {project-name}/
├── docker-compose.yml # Standard Docker Compose file
├── ingress.yml # Routing configuration (IngressV2 schema)
└── secrets.txt # (Optional) Project-specific secrets
IngressV2 schema:
enabled: true # Enable/disable routing for this project
ingress:
- service: web # Service name from docker-compose.yml
domain: example.com # Domain for routing
port: 80 # Container port
router: http # Router type: http, tcp, udp
passthrough: false # (Optional) TLS passthrough
path_prefix: / # (Optional) Path-based routing
hostport: 8080 # (Optional) Expose on host portSee samples/example-project/ for a complete working example.
You can enable and configure plugins in projects/traefik.yml. Right now we support the following:
CrowdSec can run as a container via plugin crowdsec-bouncer-traefik-plugin.
Step 1: generate api key
First set enable: true, run bin/write_artifacts.py, and bring up the crowdsec container:
itsup svc traefik up crowdsecNow we can execute the command to get the key:
itsup svc traefik exec crowdsec cscli bouncers add crowdsecBouncerPut the resulting api key in secrets/itsup.txt as CROWDSEC_API_KEY=<your-key>, then reference it in projects/traefik.yml configuration and apply with itsup apply.
Crowdsec is now running and wired up, but does not use any blocklists yet. Those can be managed manually, but preferable is to become part of the community by creating an account with CrowdSec to get access and contribute to the community blocklists, as well as view results in your account's dashboards.
Step 2: connect your instance with the CrowdSec console
After creating an account create a machine instance in the console, and register the enrollment key in your stack:
itsup svc traefik exec crowdsec cscli console enroll ${enrollment key}Step 3: subscribe to 3rd party blocklists
In the security-engines section select the "Blocklists" of your engine and choose some blocklists of interest. Example:
- Free proxies list
- Firehol SSL proxies list
- Firehol cruzit.com list
Step 4: add ip (or cidr) to whitelist
itsup svc traefik exec crowdsec cscli allowlists create me -d "my dev ips"
itsup svc traefik exec crowdsec cscli allowlists add me 123.123.123.0/24The API allows openapi compatible clients to do management on this stack (ChatGPT works wonders).
Generate the spec with api/extract-openapi.py.
All endpoints do auth and expect either:
- an incoming Bearer token
X-API-KEYheaderapikeyquery param
to be set to .env/API_KEY.
Exception: Only github webhook endpoints (check for annotation @app.hooks.register(...) get it from the github_secret header.
Webhooks are used for the following:
- to receive updates to this repo, which will result in a
git pullandbin/apply.pyto update any changes in the code. The provided project withname: itsUPis used for that, so DON'T delete it if you care about automated updates to this repo. - to receive incoming github webhooks (or GET requests to
/update-upstream?project=bla&service=dida) that result in rolling up of a project or specific service only.
One GitHub webhook listening to workflow_jobs is provided, which needs:
- the hook you will register in the github project to end with
/hook?project=bla&service=dida(serviceoptional), and thegithub_secretset to.env/API_KEY.
I mainly use GitHub workflows and created webhooks for my individual projects, so I can just manage all webhooks in one place.
NOTE:
When using crowdsec this webhook is probably not coming in as it exits the Azure cloud (public IP range), which is also host to many malicious actors that spin up ephemeral intrusion tools. To still receive signals from github you can use a vpn setup as the one used in this repo (check .github/workflows/test.yml).
itsUP includes a robust backup system that archives your service configurations and uploads them to S3-compatible storage for safekeeping. The backup functionality is implemented in bin/backup.py.
The backup system:
- Creates a tarball (
itsup.tar.gz) of yourupstreamdirectory, which contains all your service configurations - Excludes any folders specified in the
BACKUP_EXCLUDEenvironment variable - Uploads the tarball to an S3-compatible storage service
- Implements backup rotation, keeping only the 10 most recent backups
- Automatically adds timestamps to backup filenames for versioning
To use the backup functionality, you need to configure the following environment variables in your .env file:
AWS_ACCESS_KEY_ID=your_access_key
AWS_SECRET_ACCESS_KEY=your_secret_key
AWS_S3_HOST=your_s3_host
AWS_S3_REGION=your_s3_region
AWS_S3_BUCKET=your_bucket_name
BACKUP_EXCLUDE=folder1,folder2 # Optional: comma-separated list of folders to exclude from backup
To manually run a backup:
sudo .venv/bin/python bin/backup.pyFor automated backups, you can set up a cron job. For example, to run a backup daily at 2 AM:
0 5 * * * cd /path/to/itsup && .venv/bin/python bin/backup.py
To restore from a backup, you'll need to:
- Download the desired backup from your S3 bucket
- Extract the tarball to restore your configurations:
tar -xzf itsup.tar.gz.{timestamp} -C /path/to/itsup/- Run the following to apply the restored configurations (assuming itsup installed and activated with
source env.sh):
itsup run # to start the proxy stack
itsup apply # to deploy restored servicesitsUP includes automated threat analysis that correlates blacklisted IPs with threat intelligence from AbuseIPDB, performing reverse DNS lookups and WHOIS queries to identify potential threat actors.
To enable threat intelligence lookups, add your AbuseIPDB API key to .env:
ABUSEIPDB_API_KEY=your_api_key_here
Get a free API key at https://www.abuseipdb.com/register (1,000 checks/day on free tier).
make monitor-reportThis generates reports/potential_threat_actors.csv with:
- Network ranges and abuse confidence scores
- Organization details and contact information
- Usage type (Datacenter, Hosting, ISP, etc.)
- Tor exit node detection
- Last reported timestamp
The script is incremental - it only analyzes NEW IPs not already in the report.
Add to root's crontab to run daily at 4 AM:
sudo crontab -eAdd this line:
0 4 * * * cd /path/to/itsup && make monitor-report >> /var/log/threat_analysis.log 2>&1
This setup contains a project called "vpn" which runs an openvpn service that gives ssh access. To bootstrap it:
itsup svc vpn run vpn-openvpn ovpn_genconfig -u udp4://vpn.itsup.example.com
itsup svc vpn run vpn-openvpn ovpn_initpkiSave the signing passphrase you created.
export CLIENTNAME='github'
itsup svc vpn run vpn-openvpn easyrsa build-client-full $CLIENTNAMESave the client passphrase you created as it will be used for OVPN_PASSWORD below.
itsup svc vpn run vpn-openvpn ovpn_getclient $CLIENTNAME combined > .github/workflows/client.ovpnIMPORTANT: Now change udp to udp4 in the remote: ... line to target UDP with IPv4 as docker is still not there.
Test access (expects local openvpn installed):
sudo openvpn .github/workflows/client.ovpn
Now save the $OVPN_USER_KEY from client.ovpn's <key>$OVPN_USER_KEY</key> and remove the <key>...</key>.
Also save the $OVPN_TLS_AUTH_KEY from <tls-auth... section and remove it.
Add the secrets to your github repo
OVPN_USERNAME:githubOVPN_PASSWORD: the client passphraseOVPN_USER_KEYOVPN_TLS_AUTH_KEY
In order for ssh access by github, create a private key and add the pub part to the authorized_keys on the host:
ssh-keygen -t ed25519 -C "[email protected]"
cat ~/.ssh/id_ed25519.pub >> ~/.ssh/authorized_keys
Add the secrets to GitHub:
SERVER_HOST: the hostname of this repo's api serverSERVER_USERNAME: the username that has access to your host's ssh serverSSH_PRIVATE_KEY: the private key of the user
Now we can start the server and expect all to work ok.
If you wish to revoke a cert or do something else, please visit this page: kylemanna/docker-openvpn/blob/master/docs/docker-compose.md
Traefik was chosen for several key reasons:
- Dynamic configuration: Automatically picks up container changes via Docker labels
- Automatic cert management: Manages Let's Encrypt certificates gracefully with built-in ACME support
- Zero-downtime updates: With SO_REUSEPORT, can run multiple instances on same ports
- Kubernetes-like labels: Familiar L7 routing configuration for those from K8s background
- No manual cert rotation: Unlike Nginx which requires cron jobs for certificate renewal
While Nginx can be faster in raw performance (~40%), Traefik's automation and zero-downtime capabilities make it the better choice for this use case.
In the future we might consider expanding this setup to use docker swarm, as it should be easy to do. For now we like to keep it simple.
Don't blame this infra automation tooling for anything going wrong inside your containers!
I suggest you repeat that mantra now and then and question yourself when things go wrong: where lies the problem?
