Skip to content

CONTRIBUTING.md

Latest commit

 

History

History
155 lines (107 loc) · 5.32 KB

CONTRIBUTING.md

File metadata and controls

155 lines (107 loc) · 5.32 KB
Raw

Golem development

To work on Golem you need to install the following tools:

  • Latest stable rust compiler
    • use rustup
    • then run rustup update stable
    • and rustup default stable
  • Install the wasm32-wasip1 target
    • rustup target add wasm32-wasip1
  • Install cargo-make
    • cargo install --force cargo-make
  • The prost crate requires protoc to be installed.

To be able to run all integration tests:

  • Install redis
    • brew install redis on OSX
  • Install docker (for running PostgreSQL container in tests)

To be able to run all services with cargo-make run with a merged log view:

  • Install lnav
    • brew install lnav on OSX
  • Install nginx
    • brew install nginx on OSX

Everything else is managed by cargo-make.

Development workflow

Building

To compile everything use

cargo make build

It is recommended to do a full build before starting working on Golem and opening it with an IDE. During development it is usually enough to recompile only the crate you are working on, for example:

cargo build -p golem-worker-service-base

If cargo runs out of memory

Depending on the number of CPU cores and available memory, building everything can use a lot of memory. If cargo runs out of memory or just uses too much, you can limit the number of parallel jobs by providing a cargo config.toml file, for example:

[build]
jobs = 4

in ~/.cargo/config.toml. For other options and possibilities check the cargo documentation.

Running tests

Tests are using the test-r library.

During development you can run the involved tests in the usual ways: from the IDE, or using cargo test command selecting a specific crate and test module, for example:

cargo test -p golem-worker-executor-base api::promise -- --report-time

Running all unit tests

To run all unit tests use

cargo make unit-tests

Running all worker executor tests

The worker executor tests are testing the Golem Worker Executor standalone without any of the other services. These tests require redis. To run all of them use

cargo make worker-executor-tests

As there are many of these tests, they are organized into groups that are executed in parallel on CI. You can run only a specific group with cargo make, for example:

cargo make worker-executor-tests-group1

Running all integration tests

The integration tests are starting up several Golem services and testing them together. These tests also require docker and redis to be available.

To run all integration tests use

cargo make integration-tests

Running all the CLI tests

The CLI tests are similar to the integration tests but interact with the running services only through the Golem CLI application.

To run all CLI tests use

cargo make cli-tests

Using a debugger

When using a debugger with the tests, make sure to pass the --nocapture option to the test runner, otherwise the debugger will not be usable (when capturing is on, the test framework spawns child processes to run the actual tests).

Updating the REST API

Golem generates OpenAPI specs from the Rust code (using the poem-openapi crate, and the generated OpenAPI yaml file is also stored in the repository and a Rust Client crate is generated from it, used by the CLI app and also published into crates.io.

When changing anything that affects the user facing REST API, this YAML needs to be explicitly regenerated. If this does not happen, the CI process will fail and ask for doing it.

To regenerate the OpenAPI spec use

cargo make generate-openapi

Updating the config files

Service config files are also generated from code similarly to OpenAPI specs. When changing any of the service configuration data types, they have to be regeneraetd otherwise the CI process fails and asks for doing it.

To regenerate these files, use

cargo make generate-configs

Preparing the pull request

Golem CI checks the pull requests with rustfmt and cargo clippy. To make sure these checks pass, before opening a pull request run

cargo make fix

and fix any possible errors and warnings reported by it.

Running Golem locally

There are two ways now to run Golem locally:

Using cargo make run

By running

cargo make run

all the services are going to be built and started as individual native processes, configured to communicate with each other. Beside that, an nginx process is going to be started to unify the various processes' HTTP APIs, and lnav is started to merge their logs.

Using the Single Golem Executable

Golem now contains a single executable crate, called golem, that links all the services into a single binary. This binary is built using a regular cargo make build run and can be executed with

cargo run -p golem