Skip to content

feat: use arkade-regtest for regtest environment #106

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Kukks wants to merge 54 commits into
base: master
Choose a base branch
from
Open

feat: use arkade-regtest for regtest environment #106

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
54 commits
Select commit Hold shift + click to select a range
da4d629
feat: use arkade-regtest submodule for regtest environment
Kukks Mar 26, 2026
f0134bb
fix: add chmod +x for regtest scripts in CI
Kukks Mar 26, 2026
ce0c33a
chore: trigger CI
Kukks Mar 26, 2026
e130f1b
chore: retrigger CI
Kukks Mar 26, 2026
eb8823e
fix: update container name and nigiri PATH for arkade-regtest
Kukks Mar 26, 2026
f2c7392
chore: trigger CI run
Kukks Mar 26, 2026
8bbeb38
chore: bump arkade-regtest submodule (add delegator)
Kukks Mar 26, 2026
f99bfd9
chore: bump arkade-regtest (increase delegator retries)
Kukks Mar 26, 2026
3bd96a1
ci: add workflow_dispatch trigger for manual CI runs
Kukks Mar 26, 2026
d6c3f55
chore: bump arkade-regtest (arkd config vars)
Kukks Mar 26, 2026
c775255
chore: bump arkade-regtest (export env vars fix)
Kukks Mar 26, 2026
7496acd
fix: add arkd config overrides for v0.8.11 compatibility
Kukks Mar 26, 2026
f055ed4
chore: bump arkade-regtest (always recreate arkd with env vars)
Kukks Mar 26, 2026
91e5d0c
chore: bump arkade-regtest (add wallet signer key)
Kukks Mar 26, 2026
a0422fd
chore: bump arkade-regtest (wallet network + unlocker fix)
Kukks Mar 26, 2026
346fe43
chore: bump arkade-regtest (custom arkd health check)
Kukks Mar 26, 2026
eded761
chore: bump arkade-regtest (simpler health check)
Kukks Mar 26, 2026
1c6066c
chore: bump arkade-regtest (docker exec health check)
Kukks Mar 26, 2026
434cab8
chore: bump arkade-regtest (multi-method health check)
Kukks Mar 26, 2026
e78088d
chore: bump arkade-regtest (TCP health check)
Kukks Mar 26, 2026
47a212d
chore: bump arkade-regtest (combined health check)
Kukks Mar 26, 2026
db43e36
chore: bump arkade-regtest (admin port health)
Kukks Mar 26, 2026
02cbff6
chore: bump arkade-regtest (separate init retry)
Kukks Mar 26, 2026
b7939cf
chore: bump arkade-regtest (longer init retries)
Kukks Mar 26, 2026
14650d9
fix: update arkade-regtest with chain indexing wait and better diagno…
Kukks Mar 28, 2026
e53fbaf
fix: update arkade-regtest with bitcoin RPC readiness check
Kukks Mar 28, 2026
2e15da6
fix: update arkade-regtest submodule (bitcoin restart after arkd init)
Kukks Mar 28, 2026
c3cd329
fix: update arkade-regtest (nbxplorer sync check before custom arkd)
Kukks Mar 28, 2026
74b8e90
fix: update arkade-regtest and add ALLOW_CSV_BLOCK_TYPE to env config
Kukks Mar 28, 2026
43446a7
fix: update arkade-regtest (host curl, clean volumes, more retries)
Kukks Mar 28, 2026
e0edbd2
fix: update arkade-regtest (remove broken nbxplorer check)
Kukks Mar 28, 2026
32e8647
fix: use gocron scheduler for seconds-based VtxoTreeExpiry
Kukks Mar 28, 2026
0cd68c2
fix: use gocron + ALLOW_CSV_BLOCK_TYPE=false
Kukks Mar 28, 2026
c91a2f6
fix: use block scheduler for v0.8.11, update regtest submodule
Kukks Mar 28, 2026
3cc43ed
fix: update regtest submodule (GET for wallet seed endpoint)
Kukks Mar 29, 2026
6104896
fix: update regtest submodule (non-fatal redeem-notes)
Kukks Mar 29, 2026
1c02f3e
fix: update regtest submodule (timeout for note/redeem-notes)
Kukks Mar 29, 2026
5b3cc5e
fix: update regtest submodule (curl timeouts + mine blocks before set…
Kukks Mar 29, 2026
39974c2
fix: add zero intent fees for boltz-swap tests
Kukks Mar 29, 2026
5560b77
fix: update regtest submodule (add ARKD_ROUND_INTERVAL for block sche…
Kukks Mar 29, 2026
db9d666
fix: reduce ARKD_ROUND_INTERVAL to 3 for faster rounds in tests
Kukks Mar 29, 2026
bb0eacd
fix: update regtest submodule (restart chopsticks after bitcoin restart)
Kukks Mar 29, 2026
5853299
fix: update regtest submodule with arkd restart after chopsticks
Kukks Mar 29, 2026
73f6f4c
fix: update regtest submodule - move bitcoin config before arkd startup
Kukks Mar 29, 2026
1354937
fix: skip Bitcoin Core restart to prevent nbxplorer connection loss
Kukks Mar 29, 2026
ec306cf
fix: update regtest submodule — fund server wallet, not client boardi…
Kukks Mar 29, 2026
c993b16
fix: update regtest — 21-tx server funding for fee estimation
Kukks Mar 29, 2026
bf5c5bc
fix: update regtest — fund Boltz wallet for on-chain swaps
Kukks Mar 29, 2026
e292013
fix: update regtest — increase server funding to 21 BTC
Kukks Mar 29, 2026
3fb88a2
fix: increase VTXO expiry and use fee-aware arkade-regtest
Kukks Mar 29, 2026
883eb8f
fix: enable CSV block type for block-based scheduler
Kukks Mar 29, 2026
4e7fb15
fix: VTXO expiry must be < 512 for block-based scheduling
Kukks Mar 29, 2026
d6bf0f3
chore: bump regtest (conditional nbxplorer restart)
Kukks Mar 30, 2026
3fcecfd
chore: point regtest submodule to master after PR merge
Kukks Mar 30, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions .env.regtest
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# boltz-swap arkade-regtest overrides
ARKD_IMAGE=ghcr.io/arkade-os/arkd:v0.8.11
ARKD_WALLET_IMAGE=ghcr.io/arkade-os/arkd-wallet:v0.8.11

# Skip Bitcoin Core restart — avoids nbxplorer connection loss
BITCOIN_LOW_FEE=false

# Match old test.docker-compose.yml arkd config (block scheduler needs CSV block type)
ARKD_SCHEDULER_TYPE=block
ARKD_ALLOW_CSV_BLOCK_TYPE=true
ARKD_VTXO_TREE_EXPIRY=200
ARKD_BOARDING_EXIT_DELAY=1024
ARKD_ROUND_INTERVAL=3

# Zero fees — boltz-swap tests don't account for intent fees
ARK_OFFCHAIN_INPUT_FEE="0.0"
ARK_ONCHAIN_INPUT_FEE="0.0"
ARK_OFFCHAIN_OUTPUT_FEE="0.0"
ARK_ONCHAIN_OUTPUT_FEE="0.0"
148 changes: 74 additions & 74 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,81 +5,81 @@ on:
branches: [ master, next ]
pull_request:
branches: [ master, next ]
workflow_dispatch:

jobs:
test:
lint:
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v4

- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '22.12.0'

- name: Install pnpm
uses: pnpm/action-setup@v4.2.0
with:
version: 10.25.0

- name: Get pnpm store directory
shell: bash
run: |
echo "STORE_PATH=$(pnpm store path --silent)" >> $GITHUB_ENV

- name: Setup pnpm cache
env:
PNPM_CACHE_VERSION: v6 # bump this to bust caches
uses: actions/cache@v3
with:
path: ${{ env.STORE_PATH }}
key: ${{ runner.os }}-pnpm-store-${{ env.PNPM_CACHE_VERSION }}-${{ hashFiles('**/pnpm-lock.yaml') }}
restore-keys: |
${{ runner.os }}-pnpm-store-${{ env.PNPM_CACHE_VERSION }}-

- name: Install dependencies
run: pnpm install

- name: Run linter
run: pnpm lint

- name: Run tests
run: pnpm test:unit

- name: Check TypeScript compilation
run: pnpm build

- name: Run Nigiri
uses: vulpemventures/nigiri-github-action@v1
with:
use_liquid: false
use_ln: true

- name: Build arkd master
run: pnpm run test:build-docker

- name: Run arkd master
run: pnpm run test:up-docker

- name: Run setup script
run: pnpm test:setup-docker

- name: Run integration tests
run: pnpm test:integration-docker

- name: Capture arkd logs
if: failure()
run: |
docker logs arkd > arkd.log || true
if [ -f arkd.log ]; then
echo "=== arkd logs ==="
cat arkd.log
fi

- name: Tear down arkd
run: pnpm run test:down-docker
if: always()

- name: Check TypeScript compilation
run: pnpm build
- uses: actions/checkout@v4
- uses: pnpm/action-setup@v4
- uses: actions/setup-node@v4
with:
node-version: 22.12.0
cache: pnpm
- run: pnpm install --frozen-lockfile
- run: pnpm lint

unit:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: pnpm/action-setup@v4
- uses: actions/setup-node@v4
with:
node-version: 22.12.0
cache: pnpm
- run: pnpm install --frozen-lockfile
- run: pnpm test:unit

build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: pnpm/action-setup@v4
- uses: actions/setup-node@v4
with:
node-version: 22.12.0
cache: pnpm
- run: pnpm install --frozen-lockfile
- run: pnpm build

integration:
runs-on: ubuntu-latest
needs: [lint, unit, build]
steps:
- uses: actions/checkout@v4
with:
submodules: true
- uses: actions/setup-go@v5
with:
go-version: '1.23'
- uses: actions/cache@v4
with:
path: regtest/_build
key: nigiri-${{ hashFiles('regtest/.env.defaults', '.env.regtest') }}
Comment on lines +57 to +60
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot Mar 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "Tracked regtest gitlink:"
git ls-tree HEAD regtest

echo
echo "Cache key snippet:"
rg -n -C2 "actions/cache@v4|key: nigiri-" .github/workflows/ci.yml

echo
python - <<'PY'
from pathlib import Path

root = Path("regtest")
if not root.exists():
    print("regtest submodule is not initialized in this checkout; the gitlink above still shows the revision tracked by the PR.")
else:
    key_inputs = {
        Path("regtest/.env.defaults").resolve(),
        Path(".env.regtest").resolve(),
    }
    ignored = [
        p for p in root.rglob("*")
        if p.is_file() and "_build" not in p.parts and p.resolve() not in key_inputs
    ]
    print(f"Files under regtest that can change without changing the cache key: {len(ignored)}")
    for path in ignored[:40]:
        print(path)
PY

Repository: arkade-os/boltz-swap

Length of output: 536

🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Check if regtest submodule is initialized
echo "=== Submodule status ==="
cd . && git config --file .gitmodules --get submodule.regtest.url || echo "No regtest submodule in .gitmodules"
git submodule status regtest

echo
echo "=== Directory structure of regtest (if initialized) ==="
if [ -d regtest ]; then
  find regtest -type f -not -path "*/\.*" -not -path "*/_build/*" 2>/dev/null | head -20 || echo "regtest exists but find returned nothing"
  echo "---"
  ls -la regtest/ 2>/dev/null | head -15
else
  echo "regtest directory does not exist (submodule not initialized)"
fi

echo
echo "=== Git status for submodule ==="
git ls-files --stage | grep regtest

echo
echo "=== Check .env files under regtest ==="
[ -f regtest/.env.defaults ] && echo "regtest/.env.defaults exists" || echo "regtest/.env.defaults NOT found"
[ -f .env.regtest ] && echo ".env.regtest exists" || echo ".env.regtest NOT found"

Repository: arkade-os/boltz-swap

Length of output: 603

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "=== Check if regtest was bumped in this PR ==="
git log -n 10 --oneline -- regtest || echo "No recent changes to regtest"

echo
echo "=== Content of .env.regtest ==="
cat .env.regtest 2>/dev/null || echo ".env.regtest not found"

echo
echo "=== Workflow file changes ==="
git diff HEAD~1 .github/workflows/ci.yml 2>/dev/null | head -100 || echo "Cannot get diff"

echo
echo "=== Check git history for regtest submodule version ==="
git log -n 5 --oneline -- regtest 2>/dev/null || echo "No submodule history"

echo
echo "=== Current HEAD and parent commits ==="
git log -n 3 --oneline

Repository: arkade-os/boltz-swap

Length of output: 702

Include the regtest submodule revision in this cache key.

The cache key at line 60 only hashes regtest/.env.defaults and .env.regtest, but this PR itself demonstrates the problem: the regtest submodule was bumped to include a "simpler health check" without changing either environment file. This means the integration test will reuse a stale regtest/_build built from older code, causing integration tests to run against outdated nigiri/tooling.

💡 Suggested change 
+      - name: Resolve regtest gitlink
+        id: regtest-sha
+        run: echo "sha=$(git ls-tree HEAD regtest | awk '{print $3}')" >> "$GITHUB_OUTPUT"
       - uses: actions/cache@v4
         with:
           path: regtest/_build
-          key: nigiri-${{ hashFiles('regtest/.env.defaults', '.env.regtest') }}
+          key: nigiri-${{ runner.os }}-${{ steps.regtest-sha.outputs.sha }}-${{ hashFiles('regtest/.env.defaults', '.env.regtest') }}
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
- uses: actions/cache@v4
with:
path: regtest/_build
key: nigiri-${{ hashFiles('regtest/.env.defaults', '.env.regtest') }}
- name: Resolve regtest gitlink
id: regtest-sha
run: echo "sha=$(git ls-tree HEAD regtest | awk '{print $3}')" >> "$GITHUB_OUTPUT"
- uses: actions/cache@v4
with:
path: regtest/_build
key: nigiri-${{ runner.os }}-${{ steps.regtest-sha.outputs.sha }}-${{ hashFiles('regtest/.env.defaults', '.env.regtest') }}
🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In @.github/workflows/ci.yml around lines 57 - 60, The cache key for
actions/cache@v4 uses only hashFiles('regtest/.env.defaults', '.env.regtest') so
regtest/_build can be reused when the regtest submodule itself changes; update
the workflow so the cache key for path: regtest/_build includes the regtest
submodule revision (for example by incorporating the regtest submodule SHA or a
hash of the regtest tree) alongside the existing hashFiles expression — locate
the cache step that sets key: nigiri-${{ hashFiles('regtest/.env.defaults',
'.env.regtest') }} and extend it to include the regtest submodule identifier so
builds invalidate when the regtest submodule is bumped.

- uses: pnpm/action-setup@v4
- uses: actions/setup-node@v4
with:
node-version: 22.12.0
cache: pnpm
- run: pnpm install --frozen-lockfile
- name: Start regtest environment
run: chmod +x regtest/*.sh regtest/helpers/*.sh && ./regtest/start-env.sh
- name: Add nigiri to PATH
run: echo "${{ github.workspace }}/regtest/_build/nigiri/build" >> $GITHUB_PATH
- name: Setup test wallets
run: pnpm test:setup-docker
- name: Run integration tests
run: pnpm test:integration-docker
- name: Capture logs on failure
if: failure()
run: |
docker logs ark 2>&1 || true
docker logs ark-wallet 2>&1 || true
docker logs boltz 2>&1 || true
docker logs boltz-lnd 2>&1 || true
docker logs boltz-fulmine 2>&1 || true
- name: Cleanup
if: always()
run: chmod +x regtest/*.sh regtest/helpers/*.sh 2>/dev/null; ./regtest/clean-env.sh
4 changes: 4 additions & 0 deletions .gitmodules
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
[submodule "regtest"]
path = regtest
url = https://github.com/ArkLabsHQ/arkade-regtest.git
branch = master
26 changes: 0 additions & 26 deletions cors.nginx.conf
View file
Open in desktop

This file was deleted.

14 changes: 4 additions & 10 deletions package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,22 +57,16 @@
"prepack": "pnpm run build",
"test": "vitest run",
"test:unit": "vitest run --exclude test/e2e",
"test:setup": "test/setupRegtestEnv.sh",
"test:integration": "vitest run test/e2e/**",
"test:down": "test/setupRegtestEnv.sh down",
"test:build-docker": "docker compose -f test.docker-compose.yml build --no-cache",
"test:up-docker": "docker compose -f test.docker-compose.yml up arkd arkd-wallet nbxplorer pgnbxplorer -d",
"test:setup-docker": "node test/e2e/setup.mjs",
"test:integration-docker": "vitest run test/e2e/**",
"test:down-docker": "docker compose -f test.docker-compose.yml down -v",
"release": "pnpm run build && bash scripts/release.sh",
"release:dry-run": "bash scripts/release.sh --dry-run",
"release:cleanup": "bash scripts/release.sh --cleanup",
"regtest": "pnpm regtest:down && pnpm regtest:build && pnpm regtest:up && pnpm regtest:setup",
"regtest:build": "docker compose -f test.docker-compose.yml build",
"regtest:up": "docker compose -f test.docker-compose.yml up arkd arkd-wallet nbxplorer pgnbxplorer -d",
"regtest:down": "docker compose -f test.docker-compose.yml down -v",
"regtest:setup": "node test/e2e/setup.mjs"
"regtest:start": "./regtest/start-env.sh",
"regtest:stop": "./regtest/stop-env.sh",
"regtest:clean": "./regtest/clean-env.sh",
"regtest": "pnpm regtest:clean && pnpm regtest:start && pnpm test:setup-docker"
Comment on lines +66 to +69
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot Mar 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Add a submodule preflight before invoking these scripts.

These commands assume ./regtest/*.sh already exists, but a normal clone leaves the regtest gitlink empty until git submodule update --init --recursive is run. That makes pnpm regtest* fail immediately for contributors and any CI checkout that does not fetch submodules.

🔧 Minimal guard 
-        "regtest:start": "./regtest/start-env.sh",
+        "regtest:start": "[ -x ./regtest/start-env.sh ] || { echo 'Missing regtest submodule. Run: git submodule update --init --recursive' >&2; exit 1; }; ./regtest/start-env.sh",

Apply the same guard to regtest:stop and regtest:clean.

🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@package.json` around lines 66 - 69, Update the package.json scripts
regtest:start, regtest:stop, regtest:clean and the composite regtest to perform
a submodule preflight before invoking ./regtest/*.sh; specifically, prepend each
script invocation with a guard that ensures the regtest submodule is populated
(e.g., run a check and call git submodule update --init --recursive if needed)
so the shell scripts exist before running them, and keep the same script names
regtest:start, regtest:stop, regtest:clean, regtest.

},
"keywords": [
"arkade",
Expand Down
1 change: 1 addition & 0 deletions regtest
Open in desktop
Submodule regtest added at 0ddb42
Loading
Loading