Skip to content

CONTRIBUTING.md

Latest commit

 

History

History
184 lines (132 loc) · 12.5 KB

CONTRIBUTING.md

File metadata and controls

184 lines (132 loc) · 12.5 KB
Raw

Contributing

Thanks for considering a contribution. This repo is intentionally small and easy to audit — please keep it that way.

Ground rules

  • pnpm only. npm install, yarn add, and bun install are blocked by a preinstall hook. Use corepack enable to pick up the pinned pnpm version.
  • Supply chain is non-negotiable. Read .claude/rules/supply-chain.md before adding any dependency. The 7-day cooldown, deny-by-default lifecycle scripts, and SHA-pinned GitHub Actions are not optional.
  • This repo is public. Read .claude/rules/public-repo.md before touching .env, the Dockerfile, or commit messages. A leaked secret must be rotated, never rewritten.
  • Conventional commits. Enforced by commitlint in the commit-msg hook. Examples: feat(server): add Mistral judge provider, fix(server): handle missing api key.
  • No as casts. Banned by a custom Biome plugin (biome-plugins/no-as-cast.grit). The only allowed form is as const (literal-type widening control — not a cast). For everything else: use satisfies, a type guard, or ts-pattern.match.

First-time setup

corepack enable
pnpm install

pnpm install runs lefthook install automatically (via the prepare script), wiring up pre-commit (Biome + tsc + gitleaks) and commit-msg (commitlint) hooks.

Install gitleaks on your machine — the pre-commit hook calls the binary directly to keep CI = local:

# macOS
brew install gitleaks
# Arch
sudo pacman -S gitleaks
# Debian/Ubuntu (no apt package) — download the pinned release:
curl -fsSL https://github.com/gitleaks/gitleaks/releases/download/v8.30.1/gitleaks_8.30.1_linux_x64.tar.gz \
  | tar -xz gitleaks && sudo install -m 0755 gitleaks /usr/local/bin/ && rm gitleaks

Why: see .claude/rules/public-repo.md §4. The pre-commit hook is the only layer that prevents a leak rather than mitigating one — Push Protection and CI both run after the commit exists.

Python SDK (only if you touch sdk/python/)

The pre-commit ruff step calls sdk/python/.venv/bin/ruff directly to match the CI invocation in .github/workflows/test-python-sdk.yml. Bootstrap the venv once:

cd sdk/python
uv venv
uv pip install -e '.[dev]'

Install uv if you don't have it (curl -LsSf https://astral.sh/uv/install.sh | sh or brew install uv). CI installs it via a SHA-pinned setup-uv action; .tool-versions pins Node/Bun/pnpm/Python — not uv.

Daily loop

pnpm dev              # Single Bun container serving SPA + API on :8080 with HMR
pnpm cosmos           # Component workbench on :5050 (renderer on :5051)
pnpm typecheck        # tsc --noEmit
pnpm typecheck:cosmos # tsc -p cosmos --noEmit (the workbench harness)
pnpm check            # biome check (lint + format)
pnpm check:fix        # biome check --write
pnpm test             # bun test --isolate
pnpm test:watch       # bun test --isolate --watch (the TDD loop)
pnpm test:coverage    # bun test --isolate --coverage — same gate CI runs
pnpm docker:smoke     # build the image, run it, wait for the container HEALTHCHECK (probes /healthz) to go healthy

TDD

Every behavior lands red → green → refactor. The failing test goes in first, runs (and fails for the right reason), then the production code makes it green. See .claude/rules/tdd.md. The CI test workflow runs pnpm test:coverage and fails the build if coverage drops below the thresholds in bunfig.toml.

Component workbench

pnpm cosmos starts React Cosmos on :5050. It renders components against fixture files in isolation, which is how you reach states that are tedious to produce against real data — forty run configs, an empty result, a metric nothing measured.

A fixture is a *.fixture.tsx file next to the component and its test:

src/client/run-configs/
  config-picker.tsx
  config-picker.test.tsx
  config-picker.fixture.tsx    ← default-exports one element, or an object of named states

Cosmos runs in custom-bundler mode: it serves the playground UI, and cosmos/serve-renderer.ts serves the component renderer on :5051 using Bun's HTML bundler — the same bundler that builds production. There is deliberately no Vite or webpack in this path; see .claude/rules/one-bundler.md.

cosmos/cosmos.imports.ts is generated on every start and gitignored — it is derived entirely from which fixture files exist. Because it is generated, the cosmos/ directory is excluded from the root tsconfig.json, so a fresh clone typechecks before anyone has run the workbench. pnpm typecheck:cosmos covers the harness as its own project; cosmos/tsconfig.json excludes the generated map and the renderer entry that imports it, so it runs on a fresh clone and in CI alongside pnpm typecheck.

App code must never import a fixture file — fixtures may import devDependencies that a production install doesn't have. .dockerignore keeps *.fixture.tsx out of the image so that stays true even though the runtime stage copies src/ wholesale.

react-cosmos pins its own server deps (express, glob, http-proxy-middleware, ws) to exact versions, several of which carry open advisories that no release of react-cosmos itself resolves — so four overrides entries in pnpm-workspace.yaml force patched versions to keep pnpm audit honest rather than suppressed. Each carries its GHSA and its reasoning; drop the corresponding entry when react-cosmos bumps that pin. If you upgrade react-cosmos, re-run pnpm audit --audit-level=moderate and start pnpm cosmos once — the glob override crosses three majors, so the fixture scan is verified by behaviour, not by semver.

pnpm docker:smoke is the single most important local check — it builds the production image, runs it, and waits for the container's HEALTHCHECK (which probes /healthz) to report healthy. CI runs the same script in the smoke job of build.yml on every PR and push. If it passes locally, it passes in CI.

Parallel worktrees

Working on two branches at once — or reviewing a PR without losing your current train of thought? Use a git worktree so each checkout gets its own pnpm dev container, host port, SQLite file, and node_modules:

bash scripts/new-worktree.sh feat/foo            # new branch from main
bash scripts/new-worktree.sh fix/bar develop     # new branch from develop
bash scripts/new-worktree.sh pr 67               # check out open PR #67 (requires gh)

The script creates ../xray-<slug> (sibling of this repo), copies your .env so existing local secrets carry over (verify before committing — see .claude/rules/public-repo.md), picks the lowest free port at or above 8081, appends HOST_PORT + COMPOSE_PROJECT_NAME to the new .env, and runs pnpm install --frozen-lockfile. The same summary is also written to <dir>/.worktree-info for deterministic lookup later.

compose.dev.yaml reads HOST_PORT and COMPOSE_PROJECT_NAME, so each worktree's pnpm dev binds a distinct host port and gets its own named Docker volumes (xray-<slug>_dev_data, xray-<slug>_dev_node_modules). The main checkout keeps its xray-dev container and :8080 port because .env.example pins COMPOSE_PROJECT_NAME=xray and leaves HOST_PORT commented out.

If the script aborts partway through (e.g. pnpm install fails on a stale lockfile), it rolls back the partial worktree + branch so re-running starts from a clean slate. The required host tooling is git, pnpm, and (for pr mode) gh; the script checks each before touching anything on disk.

Teardown (the script prints these too):

cd ../xray-feat-foo
docker compose -f compose.dev.yaml down -v     # wipes the worktree's volumes
cd -
git worktree remove ../xray-feat-foo
git branch -D feat/foo                          # name shown in .worktree-info

Code layout

  • Vertical slices, not technical layers. src/server/conversations/, src/server/otlp/vocabularies/, src/client/inspector/ — each folder owns its own router/service/types/tests (client slices own their components/hooks). No top-level components/, hooks/, services/, utils/ god-folders.
  • Tests next to the source. foo.ts and foo.test.ts live in the same folder. No tests/ mirror tree, no __tests__/. See .claude/rules/code-layout.md for the full rationale.

Extending xray

There is no server-side "adapter" system — the alpha rewrite removed it. The current extension points are:

  • A new transport / runtime (SDK side). Subclass Runtime from sdk/python/src/xray/runtime/base.py and implement its abstract methods; LiveKitRuntime is the reference. Pipecat / OpenAI Realtime / raw WebSocket would each be a new module under sdk/python/src/xray/runtime/.
  • A new OTLP vocabulary. Drop one file in src/server/otlp/vocabularies/ and add one line to registry.ts. Each vocabulary is a pure match(span, resource) function — test it against synthetic spans with the slice's test-utils.
  • A new transcription / TTS / judge provider. Add one file in src/server/transcription/, src/server/tts/, or src/server/judges/, then wire it into the selector in src/server/providers/providers.ts. Providers are chosen at runtime via XRAY_TRANSCRIPTION_PROVIDER / XRAY_TTS_PROVIDER / XRAY_JUDGE_PROVIDER.

Tests live next to the file they test. See .claude/rules/code-layout.md.

Adding a dependency

Follow the 5-step gate in .claude/rules/supply-chain.md §3 (need · maintainer plausibility · provenance · install scripts · run). If the package has a postinstall script, it needs an allowBuilds entry in pnpm-workspace.yaml with a date + reason — never add one "to make install work."

Pull requests

  • Branch protection requires the Supply-chain audit check to pass before merge.
  • Do not put secrets, internal hostnames, customer names, or exploit details in PR titles, PR descriptions, commit messages, or issue bodies. GitHub keeps PR metadata even after force-push, and the public Events API surfaces it within seconds. See .claude/rules/public-repo.md §3.
  • One topic per PR. Refactor and feature in the same PR makes review painful.

Releasing

Releases are tag-driven. Pushing a v*.*.* tag is the entire release action — there is no version bump in package.json / sdk/python/pyproject.toml (the image version is derived from the git tag by docker/metadata-action).

  1. Make sure main is green — especially Supply-chain audit. That check also runs daily on a schedule, so pnpm audit drift (a newly-disclosed advisory flagging an already-pinned dep) surfaces here rather than at release time. If it's red, fix it first — usually a surgical overrides bump in pnpm-workspace.yaml (see .claude/rules/supply-chain.md).
  2. Tag and push the commit you want to ship: 
    git tag -s v0.0.1-alpha.N -m "v0.0.1-alpha.N"   # signed annotated tag
git push origin v0.0.1-alpha.N
    A -alpha.N / -rc.N suffix marks a pre-release; a bare vX.Y.Z is a stable release.
  3. The tag push then does everything automatically:
    • publish.yml builds the multi-arch image, pushes ghcr.io/xray-eval/xray:<version>, cosign-signs it (keyless OIDC), and attaches SLSA-provenance + SPDX-SBOM attestations.
    • its release job creates the GitHub Release (pre-release for -suffix tags) with the image ref + digest and an auto-generated changelog.
    • docs.yml rebuilds the VitePress site and deploys it to GitHub Pages.

Verify the published image afterward:

gh attestation verify oci://ghcr.io/xray-eval/xray:<version> --owner xray-eval

Reporting a security issue

Please do not open a public issue for a vulnerability. Email bong.basile@gmail.com with details and we'll coordinate a fix and disclosure timeline.

License of contributions

By contributing, you agree that your contributions are licensed under the Elastic License 2.0 — the same license as the rest of the project. No CLA is required; the act of opening a PR is the agreement.