Skip to content

feat: add arkade-regtest submodule for shared regtest env#56

Open
Kukks wants to merge 24 commits intomasterfrom
feat/use-arkade-regtest
Open

feat: add arkade-regtest submodule for shared regtest env#56
Kukks wants to merge 24 commits intomasterfrom
feat/use-arkade-regtest

Conversation

@Kukks
Copy link
Copy Markdown
Collaborator

@Kukks Kukks commented Mar 26, 2026

Summary

  • Add arkade-os/arkade-regtest as a git submodule at regtest/ for shared regtest environment
  • Add .env.regtest with dotnet-sdk specific overrides (fulmine image pin)
  • Existing Aspire-based E2E tests continue to work unchanged
  • Developers can now use ./regtest/start-env.sh for local regtest environment

Context

Part of the effort to unify regtest infrastructure across all Arkade SDKs (ts-sdk, boltz-swap, dotnet-sdk). See also:

Test plan

  • Verify dotnet test still passes (Aspire-based tests unaffected)
  • Verify git clone --recurse-submodules pulls the regtest submodule
  • Verify ./regtest/start-env.sh works for local development

@arkanaai
Copy link
Copy Markdown

arkanaai bot commented Mar 26, 2026

🔍 Review: arkade-regtest submodule integration

Clean and minimal — adds the shared regtest submodule with a dotnet-sdk-specific .env.regtest override for fulmine image pin.

Notes:

  • Submodule points to feat/shared-regtest-env-discovery branch — remember to update to main once ArkLabsHQ/arkade-regtest#1 merges.
  • No CI changes here (Aspire-based tests unaffected), so this is low-risk.
  • .env.regtest pins FULMINE_IMAGE=ghcr.io/arklabshq/fulmine:v0.3.15 — consistent with the pattern in the other SDK PRs.

Cross-repo: Part of the unified regtest effort with arkade-os/ts-sdk#386, arkade-os/boltz-swap#106, and ArkLabsHQ/arkade-regtest#1.

LGTM — straightforward submodule addition with no test surface change.

Kukks added 2 commits March 26, 2026 08:59
Add arkade-os/arkade-regtest as a git submodule at regtest/ for
local development and future CI integration. Includes .env.regtest
with dotnet-sdk specific overrides (fulmine image pin).
- Remove NArk.Tests.End2End/Infrastructure/ (old bespoke infra)
- Update CI to use regtest/ submodule for E2E tests
- Update SharedArkInfrastructure error message to point to regtest/
@Kukks Kukks force-pushed the feat/use-arkade-regtest branch from fb22ab9 to 110ba7c Compare March 26, 2026 08:01
@arkanaai
Copy link
Copy Markdown

arkanaai bot commented Mar 26, 2026

Iterative Review — New Commits (regtest submodule migration)

New commits since last review:

  • 6aa359e feat: add arkade-regtest submodule for shared regtest env
  • 110ba7c feat: replace Infrastructure/ with arkade-regtest submodule

Summary

Replaces the bespoke NArk.Tests.End2End/Infrastructure/ (start-env.sh, docker-compose, nginx config, helper scripts — ~819 lines) with the shared arkade-os/arkade-regtest git submodule at regtest/. Part of the coordinated effort with arkade-os/ts-sdk#386 and arkade-os/boltz-swap#106.

What looks good

  • Massive infra dedup: removes 819 lines of per-repo infra code in favor of a single shared source of truth
  • CI cache key correctly updated to reference regtest/_build and new .env files
  • .env.regtest pins Fulmine to v0.3.15 — correct for the current dotnet-sdk test suite
  • SharedArkInfrastructure error message updated to point to regtest/ — good UX for devs hitting setup issues
  • Log collection updated from old container names (fulmine-delegatorboltz-fulmine)
  • Clean-env.sh used in teardown instead of manual compose down + nigiri stop

Notes

  1. Submodule branch: .gitmodules tracks feat/shared-regtest-env-discovery — ensure this gets merged to the default branch in arkade-regtest before this PR lands, or new cloners will get a stale submodule
  2. chmod +x in CI: needed because git submodules on Windows lose execute bits — the commit message documents this correctly
  3. Cross-repo: sibling PRs ts-sdk#386 and boltz-swap#106 share the same submodule. Coordinated merge recommended to keep all three repos on the same regtest commit

LGTM on the new commits — clean migration.

@arkanaai
Copy link
Copy Markdown

arkanaai bot commented Mar 26, 2026

🔍 Arkana Review — dotnet-sdk#56

Lightest touch of the three SDK migrations — correctly preserves the Aspire-based E2E test runner while removing the duplicated shell-based infrastructure.

Highlights

  • 819 lines removed — the entire NArk.Tests.End2End/Infrastructure/ directory (467-line start-env.sh, 223-line docker-compose, helper scripts)
  • Aspire tests unaffectedSharedArkInfrastructure.cs just updates its reference from the old infra path
  • Uses nigiri's built-in arkd (no ARKD_IMAGE override) — simpler path since dotnet-sdk tests don't need a specific arkd version pin
  • CI cache key updated to track regtest/.env.defaults + .env.regtest — correct invalidation

Observations

1. Same submodule branch concern
.gitmodules tracks feat/shared-regtest-env-discovery — needs updating to master before merge.

2. SharedArkInfrastructure.cs change is minimal

-var startEnv = Path.Combine(infrastructureDir, "start-env.sh");
+var startEnv = Path.Combine(solutionRoot, "regtest", "start-env.sh");

Clean — the Aspire test host calls start-env.sh at a new path. Verify the solutionRoot resolution still works when running from different working directories.

3. Log collection on failure
Changed fulmine-delegatorboltz-fulmine for failure logs. This is correct since the delegator container in arkade-regtest is fulmine-delegator (not boltz-fulmine). Double-check the actual container names from docker-compose.ark.yml — the delegator logs might be missed.

Cross-Repo

@arkanaai
Copy link
Copy Markdown

arkanaai bot commented Mar 26, 2026

Followup review (commit bdebd62)

Updated error message in SharedDelegationInfrastructure.cs to reference ./regtest/start-env.sh — consistent with the same fix already applied in SharedArkInfrastructure.cs. Good catch, this was the last stale path reference.

LGTM ✅

@arkanaai
Copy link
Copy Markdown

arkanaai bot commented Mar 26, 2026

🔍 Arkana Review — Updated Commits

Scope: Replace NArk.Tests.End2End/Infrastructure/ with arkade-regtest submodule, simplify CI.

What Changed (vs. previous review)

Submodule pointer updated to pick up arkade-regtest's .env auto-discovery and delegator service.

Key Points

  1. .env.regtest — Only overrides FULMINE_IMAGE=ghcr.io/arklabshq/fulmine:v0.3.15. No ARKD_IMAGE set, meaning dotnet-sdk uses nigiri's built-in arkd (bump-arkd branch). This is correct — dotnet-sdk's Aspire tests don't need a specific arkd version pin.

  2. Infrastructure deletion — 821 lines removed: docker-compose.ark.yml (223 lines), start-env.sh (467 lines), cors.nginx.conf, create-invoice.sh, pay-invoice.sh. Massive cleanup.

  3. CI changes — Cache key changed from hashing the old start-env.sh to hashing regtest/.env.defaults + .env.regtest. Log collection updated: fulmine-delegatorboltz-fulmine container name.

  4. Test infrastructure referencesSharedArkInfrastructure.cs and SharedDelegationInfrastructure.cs updated (1 line each) — likely path references to the new regtest location.

Cross-Repo

Depends on ArkLabsHQ/arkade-regtest#1. Sibling PRs: arkade-os/ts-sdk#386, arkade-os/boltz-swap#106.

Note

The create-invoice.sh and pay-invoice.sh helper scripts are being deleted. These were useful for manual testing. If they're needed, consider adding them to arkade-regtest's helpers/ directory so all SDKs benefit.

@arkanaai
Copy link
Copy Markdown

arkanaai bot commented Mar 26, 2026

Followup review (commit 7f7b1b2): Bumps arkade-regtest submodule to pick up the env vars export fix (delegator ports + arkd config vars). No SDK-side changes. LGTM.

@arkanaai
Copy link
Copy Markdown

arkanaai bot commented Mar 26, 2026

Iterative review (commit 837c292)

Bumps arkade-regtest submodule to always recreate arkd containers when ARKD_IMAGE is set, ensuring env var changes are picked up. No SDK-side changes — just the submodule pointer update.

Cross-repo: all three SDK PRs (ts-sdk#386, boltz-swap#106, dotnet-sdk#56) and arkade-regtest#1 are now synchronized on this commit.

LGTM ✅

@arkanaai
Copy link
Copy Markdown

arkanaai bot commented Mar 26, 2026

🔍 Arkana PR Review — dotnet-sdk#56 (iterative)

feat: add arkade-regtest submodule for shared regtest env

Changes Since Last Review

New commit e457faf updates the CI workflow and .env.regtest file:

CI Workflow (build.yml)

  • ✅ Correctly switches from custom start-env.sh in NArk.Tests.End2End/Infrastructure/ to regtest/start-env.sh
  • ✅ Cache key updated to use regtest/.env.defaults + .env.regtest — good approach for cache busting on config changes
  • ✅ Log collection updated: drops docker-compose.ark.yml overlay logs, adds individual container logs (ark, boltz, bitcoin, lnd, boltz-lnd, boltz-fulmine)
  • ✅ Cleanup step uses regtest/clean-env.sh instead of manual compose down + nigiri stop

.env.regtest

  • Only override is FULMINE_IMAGE=ghcr.io/arklabshq/fulmine:v0.3.15 — dotnet-sdk uses nigiri's built-in arkd, so no arkd image override needed. Clean.

Old Infrastructure Removal

  • Entire NArk.Tests.End2End/Infrastructure/ directory removed (docker-compose, nginx config, shell scripts) — large cleanup, well done.

Security

  • ✅ No signer keys in .env.regtest — those flow through the shared regtest env defaults
  • ✅ No credential exposure in CI workflow

Cross-Repo Alignment

This PR is part of the coordinated regtest unification with ts-sdk#386, boltz-swap#106, and arkade-regtest#1. Submodule pinned at same commit 45a3b0f.

No blocking issues found.


Review by Arkana 🤖

@arkanaai
Copy link
Copy Markdown

arkanaai bot commented Mar 26, 2026

Iterative review237660e (bump arkade-regtest submodule)

Submodule pointer update to e368a88 which adds the missing ARKD_WALLET_NETWORK and ARKD_UNLOCKER_TYPE env vars to the override compose. Required for regtest to function correctly. LGTM ✅

@arkanaai
Copy link
Copy Markdown

arkanaai bot commented Mar 26, 2026

🔍 Arkana PR Review

Overall: Clean integration of the shared arkade-regtest submodule. Minimal and focused.

Submodule setup ✅

  • .gitmodules correctly points to ArkLabsHQ/arkade-regtest
  • .env.regtest pins fulmine image — appropriate since dotnet-sdk uses nigiri's built-in arkd

CI changes ✅

  • Cache key updated to use regtest defaults + overrides — good for invalidation
  • chmod +x on regtest scripts before execution
  • Log collection updated to match new container names (boltz-fulmine instead of fulmine-delegator)
  • Cleanup uses regtest/clean-env.sh

Note

The submodule branch is set to feat/shared-regtest-env-discovery — once ArkLabsHQ/arkade-regtest#1 merges, this should be updated to the default branch to avoid tracking a feature branch long-term.

Removed files ✅

Old Infrastructure/ directory (docker-compose, nginx config, shell scripts) cleanly removed. No orphaned references.

No security concerns. No blocking issues.

@arkanaai
Copy link
Copy Markdown

arkanaai bot commented Mar 26, 2026

🔍 Arkana PR Review

PR: feat: add arkade-regtest submodule for shared regtest env

Overview

Replaces the NArk.Tests.End2End/Infrastructure/ directory (467-line start-env.sh, docker-compose, nginx config, helper scripts) with the shared arkade-regtest submodule. Simplest migration of the three SDKs since dotnet-sdk uses nigiri's built-in arkd (no ARKD_IMAGE override needed).

Positive Findings

  • Largest deletion: Removes ~850 lines of infrastructure code (start-env.sh alone was 467 lines)
  • Minimal .env.regtest: Only overrides FULMINE_IMAGE — correctly lets nigiri's built-in arkd handle everything else
  • CI simplified: Cache key is clean (hashFiles('regtest/.env.defaults', '.env.regtest'))
  • Container log capture updated: Uses correct container names (ark, boltz-fulmine) matching arkade-regtest's naming

Items to Verify

  1. Container name: fulmine-delegatorboltz-fulmine: The old compose had a fulmine-delegator container for CI log capture. The new CI captures boltz-fulmine logs. Verify the dotnet tests don't reference fulmine-delegator by container name anywhere in the C# test code

  2. Missing actions/setup-go@v5: The CI workflow doesn't add the Go setup step. arkade-regtest/start-env.sh builds nigiri from source, which requires Go. If the runner doesn't have Go pre-installed, this will fail. The ts-sdk and boltz-swap PRs both include actions/setup-go@v5 with go-version: '1.23' — this PR should too

  3. NBXplorer / PostgreSQL: The old infrastructure included dedicated pgnbxplorer and nbxplorer containers. arkade-regtest (via nigiri) includes these, but verify the container names and ports match what the Aspire-based tests expect

  4. ARKD_VTXO_TREE_EXPIRY / ARKD_BOARDING_EXIT_DELAY not set: The old docker-compose had ARKD_VTXO_TREE_EXPIRY=20 and ARKD_BOARDING_EXIT_DELAY=1024. Since dotnet-sdk uses nigiri's built-in arkd (no override), these are set to nigiri's defaults. If the E2E tests depend on short expiry times, tests may time out or behave differently

  5. Submodule branch: .gitmodules tracks feat/shared-regtest-env-discovery — update to master after ArkLabsHQ/arkade-regtest#1 merges

Security

  • No credentials exposed. Regtest-only configuration.
  • The well-known ARKD_WALLET_SIGNER_KEY is no longer in the repo (was in old docker-compose) — now managed by arkade-regtest

Cross-Repo Impact

  • Depends on ArkLabsHQ/arkade-regtest#1 being merged first
  • Related PRs: arkade-os/ts-sdk#386, arkade-os/boltz-swap#106

⚠️ Action needed: Add actions/setup-go@v5 to the CI workflow — without it, nigiri build from source will fail.

@arkanaai
Copy link
Copy Markdown

arkanaai bot commented Mar 26, 2026

🔍 Arkana Review — arkade-os/dotnet-sdk#56

Add arkade-regtest submodule for shared regtest env — completes the SDK trifecta of the shared regtest migration.

Key Changes

  • Removes 821 lines of infrastructure: docker-compose.ark.yml, cors.nginx.conf, start-env.sh, create-invoice.sh, pay-invoice.sh
  • .env.regtest only overrides FULMINE_IMAGE=ghcr.io/arklabshq/fulmine:v0.3.15 — uses nigiri's built-in arkd (no ARKD_IMAGE override), which is the correct approach since dotnet-sdk tests work with nigiri's default arkd version
  • CI cache path updated from NArk.Tests.End2End/Infrastructure/nigiri to regtest/_build
  • Container name change: fulmine-delegatorboltz-fulmine in log collection

Aspire Tests

The dotnet-sdk uses .NET Aspire for E2E test orchestration. The SharedArkInfrastructure.cs and SharedDelegationInfrastructure.cs changes are minimal (path adjustments), confirming the Aspire-based tests are unaffected by the infra change. ✅

Cross-Repo

Part of: ArkLabsHQ/arkade-regtest#1, arkade-os/ts-sdk#386, arkade-os/boltz-swap#106.

LGTM.

@arkanaai
Copy link
Copy Markdown

arkanaai bot commented Mar 26, 2026

🔍 Arkana PR Review (iterative — new commits 69c2b90)

Regtest migration to shared arkade-regtest submodule continues to look good. Changes since last review:

CI workflow (build.yml):

  • Cache key updated to reference regtest/.env.defaults + .env.regtest — correct, ensures cache invalidation on env changes
  • Log collection updated: drops old compose overlay logs, references boltz-fulmine container name (matches arkade-regtest naming)
  • Cleanup uses regtest/clean-env.sh — consistent with the shared pattern

.env.regtest:

  • Pins FULMINE_IMAGE=ghcr.io/arklabshq/fulmine:v0.3.15 — good, explicit version pinning for test reproducibility
  • Comment notes nigiri built-in arkd is used — makes sense since dotnet-sdk tests dont need a custom arkd build

Observations:

  • Old NArk.Tests.End2End/Infrastructure/ directory with docker-compose, nginx config, and shell scripts all removed — clean deletion
  • No test code changes, just infrastructure migration

LGTM ✅ — Clean infrastructure migration. Consistent with ts-sdk and boltz-swap regtest PRs. Remember to update submodule branch reference from feat/shared-regtest-env-discovery to main after arkade-regtest#1 merges.

26 new commits including:
- export missing env vars for docker-compose interpolation
- pass ARK fee env vars to arkd container
- increase server wallet funding to 21 BTC
- fund Boltz Bitcoin Core wallet for on-chain swap operations
@arkanaai
Copy link
Copy Markdown

arkanaai bot commented Mar 30, 2026

🔍 Arkana PR Review — arkade-os/dotnet-sdk#56 (iterative)

New commit: a18840d7 — bumps arkade-regtest submodule pointer from 8f9373d1a86cf5

What changed

Submodule-only bump. The updated arkade-regtest commit pulls in ~25 fixes to the shared regtest infrastructure:

  • arkd wallet init: Moved to admin API for wallet creation/funding instead of client boarding address — more reliable server wallet setup
  • Fee passthrough: ARKD_ROUND_INTERVAL, ARKD_LOG_LEVEL, ARKD_SESSION_DURATION, and fee env vars now properly exported and passed through to the arkd container
  • Scheduler fix: Default changed to gocron + ALLOW_CSV_BLOCK_TYPE=false for consistent seconds-based VTXO expiry in regtest
  • Infrastructure ordering: Bitcoin Core restart moved before arkd startup; chopsticks and arkd restarted after Bitcoin Core to restore block monitoring
  • Robustness: Timeouts added to all curl calls, non-fatal handling for redeem-notes on older arkd versions, Boltz Bitcoin Core wallet funding for on-chain swaps

Assessment

Low risk — no dotnet-sdk code changes, just updating the submodule pointer to pick up regtest environment fixes. The .env.regtest overrides and CI workflow are unchanged.

The fixes address real stability issues that were likely causing intermittent CI failures (wallet init race conditions, missing fee estimation data, chopsticks losing sync after Bitcoin Core restarts).

No blocking issues. LGTM for merge.

No nbxplorer container in dotnet-sdk stack, so the low-fee restart
path failed with "No such container: nbxplorer". Disable low-fee
mode and bump regtest to include the conditional nbxplorer restart.
@arkanaai
Copy link
Copy Markdown

arkanaai bot commented Mar 30, 2026

🔍 Iterative review (commit a1984f9)

Clean fix. The dotnet-sdk stack uses nigiri's built-in arkd without BTCPay/nbxplorer, so BITCOIN_LOW_FEE=false correctly skips the Bitcoin Core restart that was failing with "No such container: nbxplorer."

Changes:

  • .env.regtest: adds BITCOIN_LOW_FEE=false — correct, this stack doesn't need low-fee tx relay
  • regtest submodule bump → 9a409cdf which adds the conditional docker inspect nbxplorer guard upstream

Cross-repo: Pairs with ArkLabsHQ/arkade-regtest#1 (the upstream fix that makes nbxplorer restart conditional). Both sides of the fix are clean.

No concerns. ✅

@arkanaai
Copy link
Copy Markdown

arkanaai bot commented Mar 30, 2026

Iterative review (commits a18840d, a1984f9, b2af165)

Three commits since last review:

  1. Bulk regtest bump (a18840d): Picks up 26 upstream fixes — env var exports, fee passthrough to arkd, 21 BTC server funding, Boltz wallet funding. This was the missing piece for dotnet-sdk CI.
  2. Disable BITCOIN_LOW_FEE (a1984f9): Correct fix — dotnet-sdk uses nigiri's built-in arkd without a separate nbxplorer container, so the low-fee restart path was failing with "No such container: nbxplorer".
  3. Restore redeem-notes (b2af165): Bumps regtest submodule to include fallback funding fix for nigiri's built-in arkd.

No concerns. The CI pipeline should now have the complete set of fixes for the dotnet-sdk stack.

The nbxplorer guard in start-env.sh now handles missing containers
gracefully, so we can safely enable low-fee Bitcoin Core config.
Chain swaps need low fees to avoid transaction.failed states.
@arkanaai
Copy link
Copy Markdown

arkanaai bot commented Mar 30, 2026

🔍 Arkana PR Review — dotnet-sdk#56

feat: add arkade-regtest submodule for shared regtest env

Overview

Excellent infrastructure cleanup. This replaces ~820 lines of custom regtest setup (docker-compose, shell scripts, nginx config) with the shared arkade-regtest submodule, aligning dotnet-sdk with ts-sdk and boltz-swap. Net -801 lines.

✅ What looks good

  • CI workflow properly updated: submodules: true in both build and e2e checkout steps ensures the submodule is cloned. Cache key updated to reference the new paths (regtest/_build, regtest/.env.defaults, .env.regtest).
  • Clean separation of concerns: .env.regtest provides dotnet-sdk-specific overrides (Fulmine image pin, low-fee flag) while the shared scripts handle the common infrastructure.
  • Test code updated: SharedArkInfrastructure.cs and SharedDelegationInfrastructure.cs reference the new ./regtest/start-env.sh path.
  • Docker log collection in CI failure path adapted to new container naming (boltz-fulmine instead of fulmine-delegator).
  • No security concerns: All changes are in test infrastructure, no production code affected.

⚠️ Observations (non-blocking)

  1. Submodule pinned to feature branch: .gitmodules points to branch = feat/shared-regtest-env-discovery, not main. Once ArkLabsHQ/arkade-regtest#1 is merged, this should be updated to track main to avoid the submodule reference going stale.

  2. Cache path mismatch (CI): The base branch CI already has the checkout with submodules: true, but the cache key on the base branch still references NArk.Tests.End2End/Infrastructure/nigiri. The diff correctly updates to regtest/_build — first CI run on this branch will miss the old cache (expected, one-time cold start).

  3. Fulmine image pin: .env.regtest pins FULMINE_IMAGE=ghcr.io/arklabshq/fulmine:v0.3.15. Worth verifying this matches what the shared regtest env expects. If arkade-regtest defaults to a newer version, this override would hold it back.

  4. BITCOIN_LOW_FEE=true: Comment says "nbxplorer guard in start-env.sh handles missing container gracefully" — good, but verify the shared start-env.sh actually reads this env var (it's a dotnet-sdk specific flag that the shared script may or may not honor).

Cross-repo context

  • Part of the unified regtest effort: arkade-os/ts-sdk#386, arkade-os/boltz-swap#106, ArkLabsHQ/arkade-regtest#1
  • All three consumer PRs depend on arkade-regtest#1 being merged first (or at least the feature branch staying stable)

Verdict

Clean migration. The main action item is updating the submodule branch from feat/shared-regtest-env-discoverymain once the upstream PR merges. No blocking issues.

@arkanaai
Copy link
Copy Markdown

arkanaai bot commented Mar 30, 2026

Follow-up review (new commits since last review)

Latest: batch bump of arkade-regtest submodule (26 commits worth of env var fixes, fee passthrough, server funding), BITCOIN_LOW_FEE toggled off then back on once the nbxplorer guard was added, submodule pointed to master.

Config (.env.regtest)

  • Minimal: just BITCOIN_LOW_FEE=true and fulmine image pin — dotnet-sdk's Aspire-based tests manage their own arkd config, so the .env.regtest is intentionally lean
  • Re-enabled low fees after nbxplorer guard handles missing containers (no nbxplorer in dotnet-sdk's stack)

Scope — Lightest of the four sibling PRs. Removes NArk.Tests.End2End/Infrastructure/ (811 lines of bespoke docker-compose, nginx, shell scripts). Aspire E2E tests continue unchanged.

Cross-repo — Same submodule commit as ts-sdk#386, boltz-swap#106, wallet#492.

No security concerns. Ready pending E2E CI green.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant