Add environment-based variable replacement for stacks #128

jleal52 · 2025-10-26T17:23:02Z

Add environment-based variable replacement for stacks

This PR implements environment-specific variable replacement for Docker Swarm stacks, allowing different configurations per environment (dev, staging, prod, etc.) stored directly in the stack's Git repository.

How it works

Environment detection: SwarmCD reads the environment from a Docker Swarm manager node label (default: swarmcd.environment)
Configuration file: Each stack can define an environments_file in stacks.yaml pointing to a YAML file in the repository with environment-specific variables
Variable replacement: Variables in the format ${VAR_NAME} or $VAR_NAME in the compose file are replaced with values from the environments file
Stack filtering: Stacks with an environments_file are only deployed if:
- The node has an environment label set
- The current environment is defined in the environments file

Example

stacks.yaml:

my-app:
  repo: my-repo
  branch: main
  compose_file: docker-compose.yml
  environments_file: environments.yml  # New field

environments.yml (in repository):

environments:
  prod:
    DB_HOST: prod-db.example.com
    API_URL: https://api.example.com
  dev:
    DB_HOST: dev-db.local
    API_URL: https://dev-api.example.com

docker-compose.yml (in repository):

services:
  app:
    environment:
      DATABASE_HOST: ${DB_HOST}
      API_ENDPOINT: ${API_URL}

Benefits

Environment-specific configuration versioned with the stack code
No need for multiple compose files per environment
Automatic stack filtering based on environment
Variables replaced recursively in all compose fields (environment, command, labels, etc.)

Backward compatibility

Fully backward compatible - stacks without environments_file deploy normally
No breaking changes to existing configurations

Files changed

util/config.go: Added EnvironmentsFile field to StackConfig
swarmcd/init.go: Added environment detection from Docker Swarm node labels
swarmcd/stack.go: Implemented variable loading and replacement logic
swarmcd/stack_test.go: Added comprehensive tests for the new functionality
docs/ENVIRONMENT_FILTERING.md: Complete documentation with examples
docs/stacks.yaml: Updated with environments_file example
docs/environments.yaml: Example environments file

Testing

All tests pass successfully (8/8):

Variable replacement in simple values
Variable replacement in nested structures (maps and arrays)
Environment filtering logic
Backward compatibility with stacks without environments file

sanzoghenzo

What a great feature!

It looks OK overall.
Some of the comments come from my personal preference, so they are open for debate 😉

I got two more observations (that can be addressed in future PRs):

docker compose also supports envrionment variable substitution by just specifying an empty item in the environment map (so you can avoid repeating the variable name in the value), should we support this, too?
Should we also support a global environments_file? I'm thinking, for instance, of a DOMAIN variable that could be used across all the stacks

sanzoghenzo · 2025-10-26T18:22:24Z

docs/ENVIRONMENT_FILTERING.md

Personal preference, but this AI generated doc seems too verbose to describe a relatively simple concept.

Please also add a link to this document in the main readme file.

sanzoghenzo · 2025-10-27T08:35:14Z

swarmcd/init.go

+		currentEnvironment = env
+		logger.Info("detected environment from node label", "environment", currentEnvironment, "label", environmentLabel)
+	} else {
+		logger.Warn("environment label not found on manager node, all stacks will be deployed", "label", environmentLabel)


I don't feel this should be a Warning, since this feature is an opt-in

sanzoghenzo · 2025-10-27T08:41:14Z

swarmcd/init.go

+	cli, err := client.NewClientWithOpts(client.FromEnv, client.WithAPIVersionNegotiation())
+	if err != nil {
+		return fmt.Errorf("could not create docker client for environment detection: %w", err)
+	}
+	defer cli.Close()


Is there a reason to not use the dockerCli object here?

sanzoghenzo · 2025-10-27T08:44:24Z

swarmcd/init.go

+
+	// Check if this is a swarm manager
+	if !info.Swarm.ControlAvailable {
+		return fmt.Errorf("this node is not a swarm manager, cannot detect environment")


The node in which SwarmCD runs could be different from the managed node if a user sets a remote url in DOCKER_HOST or uses a socket proxy, so the "this node" wording could cause a bit of confusion.

sanzoghenzo · 2025-10-27T08:53:49Z

swarmcd/init.go

-	err = initDockerCli()
+	err = detectEnvironment()
+	if err != nil {
+		return err


I don't believe an error of detectEnvironment should stop SwarmCD, especially if a user doesn't need this feature.

sanzoghenzo · 2025-10-27T09:10:43Z

swarmcd/stack.go

+	shouldDeploy, err := swarmStack.loadEnvironmentVars()
+	if err != nil {
+		return
+	}
+	if !shouldDeploy {
+		log.Info("stack skipped due to environment filtering")
+		return revision, nil
+	}
+
 	stackBytes, err := swarmStack.readStack()
 	if err != nil {
 		return
 	}

 	if swarmStack.valuesFile != "" {
-		log.Debug("rendering template...")
 		stackBytes, err = swarmStack.renderComposeTemplate(stackBytes)
+		if err != nil {
+			return
+		}
 	}
+
+	stackContents, err := swarmStack.parseStackString([]byte(stackBytes))
 	if err != nil {
 		return
 	}

-	log.Debug("parsing stack content...")
-	stackContents, err := swarmStack.parseStackString([]byte(stackBytes))
+	err = swarmStack.replaceEnvVars(stackContents)
 	if err != nil {
 		return
 	}
-
-	log.Debug("decrypting secrets...")
 	err = swarmStack.decryptSopsFiles(stackContents)
 	if err != nil {
 		return "", fmt.Errorf("failed to decrypt one or more sops files for %s stack: %w", swarmStack.name, err)
 	}

-	log.Debug("rotating configs and secrets...")
 	err = swarmStack.rotateConfigsAndSecrets(stackContents)
 	if err != nil {
 		return
 	}

-	log.Debug("writing stack to file...")
 	err = swarmStack.writeStack(stackContents)
 	if err != nil {
 		return
 	}

-	log.Debug("deploying stack...")
 	err = swarmStack.deployStack()


Is there a reason to remove the debug messages? if you don't want to see them, just raise the log level 😉

sanzoghenzo · 2025-10-27T09:12:45Z

swarmcd/stack.go

+	// If environments file is specified but no environment label is set on the node,
+	// this stack is environment-filtered and should NOT be deployed


the log message explains this already, no need for a comment here

sanzoghenzo · 2025-10-27T09:13:08Z

swarmcd/stack.go

+	envFilePath := path.Join(swarmStack.repo.path, swarmStack.environmentsFile)
+	envFileBytes, err := os.ReadFile(envFilePath)
+	if err != nil {
+		// If file doesn't exist, it's not an error - just skip loading


remove the comment

sanzoghenzo · 2025-10-27T09:15:15Z

swarmcd/stack.go

+		// Current environment is not defined in the environments file
+		// This means this stack should not be deployed in this environment


remove comment

sanzoghenzo · 2025-10-27T09:19:39Z

swarmcd/stack.go

+			"available_environments", getKeys(envConfig.Environments))
+		return false, nil
+	}
+}
+
+func getKeys(m map[string]map[string]string) []string {
+	keys := make([]string, 0, len(m))
+	for k := range m {
+		keys = append(keys, k)
+	}
+	return keys
+}


I'm not a go guru, but it seems that this can be simplified with

Suggested change

"available_environments", getKeys(envConfig.Environments))

return false, nil

}

}

func getKeys(m map[string]map[string]string) []string {

keys := make([]string, 0, len(m))

for k := range m {

keys = append(keys, k)

}

return keys

}

"available_environments", slices.Collect(maps.Keys(envConfig.Environments))

return false, nil

}

}

Environments feature

4a2f946

sanzoghenzo requested changes Oct 27, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add environment-based variable replacement for stacks #128

Add environment-based variable replacement for stacks #128

Uh oh!

jleal52 commented Oct 26, 2025 •

edited

Loading

Uh oh!

sanzoghenzo left a comment

Uh oh!

sanzoghenzo Oct 26, 2025

Uh oh!

sanzoghenzo Oct 27, 2025

Uh oh!

sanzoghenzo Oct 27, 2025

Uh oh!

sanzoghenzo Oct 27, 2025

Uh oh!

sanzoghenzo Oct 27, 2025

Uh oh!

sanzoghenzo Oct 27, 2025

Uh oh!

sanzoghenzo Oct 27, 2025

Uh oh!

sanzoghenzo Oct 27, 2025

Uh oh!

sanzoghenzo Oct 27, 2025

Uh oh!

sanzoghenzo Oct 27, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		// If environments file is specified but no environment label is set on the node,
		// this stack is environment-filtered and should NOT be deployed

		// Current environment is not defined in the environments file
		// This means this stack should not be deployed in this environment

Add environment-based variable replacement for stacks #128

Are you sure you want to change the base?

Add environment-based variable replacement for stacks #128

Uh oh!

Conversation

jleal52 commented Oct 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Add environment-based variable replacement for stacks

How it works

Example

Benefits

Backward compatibility

Files changed

Testing

Uh oh!

sanzoghenzo left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

jleal52 commented Oct 26, 2025 •

edited

Loading