AGENTS.md

Golem Development Guide

Overview

Golem is a distributed computing platform built in Rust. It uses cargo-make for build orchestration.

Backward Compatibility

Golem development currently makes no backward-compatibility guarantees. Until this section is explicitly revised, do not perform backward-compatibility work.

• Make breaking changes directly and update every in-tree producer, consumer, test, fixture, generated artifact, and document to the new contract in the same change.
• Do not add compatibility shims, legacy aliases, deprecated variants, fallback parsing, dual serialization, version negotiation, migration paths, or support for old protocols, schemas, configuration shapes, persisted formats, APIs, or behavior.
• When replacing a contract or representation, remove the old path instead of preserving both.
• Do not expand the scope of a task to accommodate older clients, servers, deployments, or stored data. If a task appears to require backward compatibility, treat it as conflicting with this repository policy rather than implementing it.

Prerequisites

• Rust: Latest stable toolchain via rustup, with wasm32-wasip2 target
• cargo-make: Latest version (cargo install --force cargo-make)
• cargo-test-r: Match the version CI installs, pinned in .github/actions/restore-binaries/action.yml (cargo install --force --locked cargo-test-r@<VERSION>)
• redis-server: Required by worker-executor, service-integration, and CLI integration tests that use spawned test dependencies; not required by every unit test
• docker: Required by tests that use containerized dependencies

Building

cargo make build          # Build all targets in the root Cargo workspace in debug mode
cargo make build-release  # Release build excluding integration-tests and the single-binary golem package
cargo build -p <crate>    # Build a specific root-workspace crate

Note: The SDKs in sdks/ are not part of the main build flow. Load sdk-development when working on the Rust or TypeScript SDKs, golem-scala-development when working on the Scala SDK, and see sdks/moonbit/AGENTS.md when working on the MoonBit SDK.

Note: The user-facing documentation site (deployed at learn.golem.cloud) lives under docs/ and is a Next.js / Nextra project built with Bun — see docs/AGENTS.md. It is built and deployed by a dedicated workflow (.github/workflows/docs.yaml); the main ci.yaml skips runs on docs-only changes.

Two parts of docs/ are auto-generated from in-tree sources of truth and have CI drift checks (unit-tests-and-checks job):

• docs/src/content/next/rest-api/*.mdx ← openapi/golem-service.yaml. Regenerated by cargo make generate-openapi (or cargo make generate-docs-openapi if the YAML is already current). Drift is rejected by cargo make check-openapi.
• docs/src/content/next/how-to-guides/ and docs/src/content/next/how-to-guides.mdx ← golem-skills/skills/. Regenerated by cargo make generate-docs-skills. Drift is rejected by cargo make check-docs-skills.

If you edit either source, commit the regenerated MDX files in the same PR.

Makefile.toml script conventions

Inline scripts in Makefile.toml must be written in duckscript (script_runner = "@duckscript"), not POSIX shell/bash, so the build works on all operating systems including Windows. Do not introduce bash/sh inline scripts for build/asset tasks.

When a task needs the cargo target directory (e.g. to invoke a freshly built binary), do not hardcode target/... or shell out to cargo metadata. Use the CARGO_MAKE_CRATE_TARGET_DIRECTORY environment variable, which cargo-make resolves cross-platform and which honors a redirected target dir (CARGO_TARGET_DIR, cargo config, or a cargo wrapper). Example:

script_runner = "@duckscript"
script = '''
golem = set "${CARGO_MAKE_CRATE_TARGET_DIRECTORY}/debug/golem"
exec --fail-on-error ${golem} build -P release --yes
'''

Wasmtime configuration compatibility

All production engines that compile, load, or inspect components must use the shared constructors in golem_common::wasmtime_config. This includes the component compilation service, worker executor, component metadata extraction, and any future component tools. Do not set Wasmtime features or tunables independently at those call sites; change the shared constructor instead and verify every consumer still uses it.

The component compilation service serializes precompiled .cwasm artifacts that the worker executor deserializes. Wasmtime validates artifact-affecting feature and tunable settings during deserialization. A mismatch makes the executor reject the cached artifact and compile the original component again, causing a severe cold-start regression. When changing the Wasmtime version or configuration, run:

cargo test -p golem-common precompiled_components_are_compatible_across_engines -- --report-time

Minimal Wasmtime engines in unit tests may use a local configuration only when they exercise an isolated host primitive and neither load production components nor produce or consume precompiled artifacts. Tests intended to mirror production component behavior must use the shared constructor.

Testing

Root-workspace Rust tests use test-r. Important: Each test crate or integration-test entry point using test-r must call test_r::enable!(), and test_r::test must be in lexical scope wherever #[test] is used; otherwise those tests will not be registered with test-r. Separate workspaces may use their own test harnesses.

Tests that directly execute external processes belong in the CLI integration test suite. This includes Golem binaries such as golem and golem-cli, as well as tools and compilers such as cargo, rustc, npm, npx, tsc, bun, sbt, scala-cli, and moon.

• Unit tests and worker-executor tests must never spawn external processes.
• Non-CLI integration tests must use golem-test-framework for all process-backed dependencies and must not spawn additional processes directly.
• CLI integration tests may spawn the external processes required by the behavior they exercise.

The unit-test CI job runs prebuilt test binaries and is intentionally kept fast; subprocess execution in unit tests bypasses that design and can make the job time out.

Worker executor tests, integration tests, and CLI integration tests may depend on built test components from test-components/. Generated .wasm artifacts are generally not checked into the repository, so build the specific components needed by the selected tests before running them. The two minimal concurrent-runtime fixtures are intentional checked-in exceptions. Use the modifying-test-components skill for targeted rebuilds, or rebuild-all-test-components when a full rebuild is needed.

Rebuilding a test component with the latest locally built Golem binary may migrate files in that component's source directory, including manifests and embedded skills. Treat migration edits produced by the rebuild as intentional outputs: review them, keep them, and include them in the changeset even when they are unrelated to the original reason for rebuilding. Do not revert or omit them merely to minimize the diff; test components are expected to evolve with current Golem.

Do not run cargo make test — it runs broad root-workspace unit, worker-executor, service, and CLI integration suites and takes a very long time. Start with the smallest test that exercises the changed behavior, then broaden when the impact cannot be isolated:

Scope	Test Command
One Rust test or test module	cargo test -p <crate> -- <test_name> --report-time
One crate's library tests	cargo test -p <crate> --lib -- --report-time
Worker executor test group	cargo make worker-executor-tests-group1, cargo make worker-executor-tests-group2, cargo make worker-executor-tests-group3, cargo make worker-executor-tests-group4, or cargo make worker-executor-tests-misc
One CLI integration area	Use a targeted cargo-test-r filter; see the testing skill
Broad core logic or utilities	cargo make unit-tests
Broad worker executor functionality	cargo make worker-executor-tests
Broad service and CLI integration	cargo make integration-tests
Broad CLI changes	cargo make cli-integration-tests

For broad CLI integration test reruns, use cargo make cli-integration-tests. For faster local runs with dev-release binaries, use cargo make cli-integration-tests-dev-release. To isolate failures, run targeted cargo-test-r filters or lower local concurrency with CLI_TEST_THREADS=1 cargo make cli-integration-tests.

Whenever tests are modified, always run the affected tests to verify they still pass before considering the task complete.

Load the testing skill for detailed guidance on test filtering, debugging failures, test components, and timeouts.

Running Locally

cargo run -p golem  # Build and run the single Golem executable

Skills

Load these skills for guided workflows on complex tasks:

Skill	When to Use
modifying-http-endpoints	Adding or modifying REST API endpoints (covers OpenAPI regeneration, golem-client rebuild, type mappings, docs MDX sync)
adding-dependencies	Adding or updating crate dependencies (covers workspace dependency management, versioning, features)
testing	Running and debugging tests (covers test filtering, debugging failures, test components, timeouts)
debugging-hanging-tests	Diagnosing worker executor or integration tests that hang indefinitely
modifying-test-components	Building or modifying test WASM components, or rebuilding after SDK changes
modifying-wit-interfaces	Adding or modifying WIT interfaces and synchronizing across sub-projects
modifying-cli-manifest-schema	Adding or changing application manifest JSON schema versions and aligning CLI schema references
modifying-cli-output-schema	Adding or changing structured golem-cli output, StructuredOutput types, or the command output JSON schema
modifying-service-configs	Changing service configuration structs, defaults, or adding new config fields
db-migration-scripts	Writing PostgreSQL and SQLite migration scripts under a db/migration/ root
logging	Adding or reviewing tracing statements and following the structured logging conventions
modifying-builtin-plugins	Changing built-in plugin source, committed WASM, descriptors, versions, or provisioning
creating-new-builtin-plugins	Adding a new built-in WASM plugin embedded in and provisioned by the registry service
sdk-development	Working on the Rust, TypeScript, or MoonBit SDKs in sdks/
migrate-ts-decorator-sdk	Porting a TypeScript agent from the removed decorator/BaseAgent API to defineAgent
golem-scala-development	Compile, publish, and test the Golem Scala SDK in sdks/scala/
golem-scala-integration-tests	Running and debugging Scala SDK integration tests
golem-scala-base-image	WIT folder structure and regenerating agent_guest.wasm for the Scala SDK
golem-scala-code-generation	Writing Scala code generators for the Scala SDK
moonbit-agent-guide	Writing, refactoring, and testing MoonBit projects, and using moon tooling
moonbit-refactoring	Refactoring MoonBit code to be idiomatic without regressing test coverage
moonbit-c-binding	Writing MoonBit bindings to C libraries with native FFI
moonbit-code-transform	Source-to-source MoonBit transformations with moonbitlang/parser and formatter
moonbit-proof	Writing proof-carrying MoonBit code with Why3-backed specifications
investigating-executor-performance	Investigating worker-executor performance with OTLP tracing and Jaeger
investigating-benchmark-performance	Profiling Golem benchmarks with OTLP tracing and Jaeger
analysing-ci-failures	Diagnosing a GitHub Actions failure from a run or job URL and reproducing it locally
golem-skill-harness	Developing, testing, and running Golem skill tests with the skill test harness
managing-docs-versions	Cutting a new docs version, backporting fixes to older releases, and managing versioned content under docs/src/content/
pre-pr-checklist	Final checks before submitting a pull request

Before Submitting a PR

Validate the smallest dependency and behavior scope that fully covers the change. Use package-scoped, non-mutating format, lint, build, and test commands by default. Broaden checks when shared contracts, workspace configuration, generated artifacts, or multiple subsystems are affected.

Do not run cargo make fix by default. It mutates the entire root and dev-tools workspaces and does not validate the separately built SDKs. Use scoped auto-fix commands only when needed, and inspect their diff afterward.

Load the pre-pr-checklist skill for the change-scope matrix and escalation rules. Repository-wide CI remains the final broad safety net; local verification must still cover the affected code and behavior before opening a PR.

Code Style

• Follow existing code conventions in the file you're editing
• Do not add unnecessary comments
• Comments must describe the code as it is, for a reader who has no other context. Never reference implementation plans, migration "phases", ticket/issue/PR numbers, or other transient process artifacts in code comments — describe the actual invariant, behavior, or rationale instead.
• Use existing libraries and utilities from the codebase
• Security: Never expose or log secrets/keys

Dependency Management

Dependencies of root Cargo workspace members must be declared centrally in the root Cargo.toml under [workspace.dependencies]. Workspace members must reference them using x = { workspace = true } rather than declaring them directly. Independently built Cargo workspaces under sdks/, test-components/, plugins/, and dev-tools/ manage dependencies in their own workspace manifests.

Project Structure

• golem-worker-executor/ - Worker execution engine
• golem-worker-service/ - Worker management service
• golem-component-compilation-service/ - Component compiler
• golem-shard-manager/ - Distributed shard management
• golem-registry-service/ - Component registry
• golem-common/ - Shared types and utilities
• cli/ - CLI tools (golem-cli, golem)
• sdks/ - Language-specific SDKs (Rust, TypeScript, Scala, MoonBit) - not part of main build flow, see SDK-specific AGENTS.md
• docs/ - User-facing documentation site at learn.golem.cloud (Next.js / Nextra / Bun) - not part of main build flow, see docs/AGENTS.md
• golem-skills/ - Skill definitions and skill testing harness
• integration-tests/ - Integration test suite
• test-components/ - Test WASM components
• website/ - Marketing site for golem.cloud (Astro, deployed to Netlify) - separate from docs/