|
| 1 | +# Architecture |
| 2 | + |
| 3 | +## Overview |
| 4 | + |
| 5 | +The blockchain-node-operator is a Kubernetes operator that manages blockchain node workloads declaratively. Users declare the desired state via Custom Resources and the operator continuously reconciles the actual cluster state to match — creating and maintaining StatefulSets, ConfigMaps, Services, and PodMonitors for each blockchain node, handling upgrades, rollbacks, and snapshot bootstrapping automatically. |
| 6 | + |
| 7 | +## Custom Resource Definitions |
| 8 | + |
| 9 | +### BlockchainNode |
| 10 | + |
| 11 | +The primary CRD. Each instance describes one blockchain node: the chain type (via `.spec.chain`), the container image or version tracking policy, resource requirements, storage, networking, and health thresholds. The operator owns all child resources created from a BlockchainNode. |
| 12 | + |
| 13 | +### ChainVersionCatalog |
| 14 | + |
| 15 | +A cluster-scoped CRD that holds discovered image tags per chain. The operator polls configured container registries on a schedule and writes the latest resolved tag back into this resource. BlockchainNodeReconciler reads from it when `VersionPolicy` tracking is enabled on a node. |
| 16 | + |
| 17 | +## Component Diagram |
| 18 | + |
| 19 | +```mermaid |
| 20 | +graph TD |
| 21 | + User([User / GitOps]) -->|apply BlockchainNode CR| K8sAPI[Kubernetes API] |
| 22 | +
|
| 23 | + K8sAPI --> BNR[BlockchainNodeReconciler] |
| 24 | +
|
| 25 | + BNR --> CM[ConfigMap\nchain config / genesis] |
| 26 | + BNR --> STS[StatefulSet\nnode + init containers] |
| 27 | + BNR --> SVC[Service\nRPC / P2P ports] |
| 28 | + BNR --> PM[PodMonitor\nPrometheus scrape] |
| 29 | +
|
| 30 | + STS --> IC[Snapshot Init Container\nMinIO bootstrap] |
| 31 | +
|
| 32 | + BNR --> AR[Adapter Registry\n102 chain adapters] |
| 33 | + AR --> Adapter[Chain Adapter\nDefaultResources / VersionPolicy\nports / config template] |
| 34 | +
|
| 35 | + BNR --> AU[Auto-Upgrade Reconciler] |
| 36 | + AU --> CVC[ChainVersionCatalog] |
| 37 | + CVC --> RC[Registry Clients] |
| 38 | + RC --> DH[DockerHub v2 API] |
| 39 | + RC --> GH[GHCR OCI v2] |
| 40 | + RC --> OCI[OCI v2\nGAR / ECR Public] |
| 41 | +
|
| 42 | + AU -->|rolling restart| STS |
| 43 | + AU -->|CrashLoopBackOff ≥3| RB[Rollback to previous tag] |
| 44 | +
|
| 45 | + K8sAPI --> WH[Admission Webhooks\nvalidating + defaulting] |
| 46 | +``` |
| 47 | + |
| 48 | +## Reconciliation Flow |
| 49 | + |
| 50 | +1. **Fetch CR** — load the `BlockchainNode` object; requeue on not-found after a short delay. |
| 51 | +2. **Resolve adapter** — look up the chain adapter in the registry by `spec.chain`; return a permanent error for unknown chains. |
| 52 | +3. **Handle deletion / finalizer** — if DeletionTimestamp is set, run cleanup (remove PodMonitor, external resources) and strip the finalizer; otherwise ensure the finalizer is present. |
| 53 | +4. **ensureConfigMap** — render the chain-specific config template via the adapter and create-or-update the ConfigMap. |
| 54 | +5. **ensureStatefulSet** — merge adapter defaults with user overrides (resources, storage, env, ports); create-or-update the StatefulSet. If a snapshot URL is configured, inject the init container. |
| 55 | +6. **ensureService** — reconcile the headless Service and, if enabled, a separate LoadBalancer/NodePort Service for RPC exposure. |
| 56 | +7. **ensurePodMonitor** — create or update the Prometheus Operator `PodMonitor` using the metrics port declared by the adapter. |
| 57 | +8. **reconcileUpgrade** — if `VersionPolicy` is active, compare the running image tag against the latest tag in `ChainVersionCatalog`; trigger a rolling restart when a newer tag is found. |
| 58 | +9. **refreshStatus** — update `.status` fields: current image, sync phase, block height, peer count, health condition, last upgrade time. |
| 59 | + |
| 60 | +## Health Monitoring |
| 61 | + |
| 62 | +The operator watches pod metrics and logs to derive node health: |
| 63 | + |
| 64 | +- **Block-lag threshold** — if the node's block height lags behind peers or a reference RPC endpoint by more than `spec.health.maxBlockLag`, the condition is set to `Degraded`. |
| 65 | +- **Sync stall detection** — if the block height does not advance for longer than `spec.health.syncStallTimeout`, the node is considered stalled. |
| 66 | +- **Peer count** — if connected peers fall below `spec.health.minPeers`, a warning condition is emitted. |
| 67 | +- **Auto-restart on degraded timeout** — if the node remains in `Degraded` for longer than `spec.health.autoRestartTimeout`, the operator deletes the pod to trigger a fresh start. The threshold is configurable per node to avoid restart loops on slow-syncing chains. |
| 68 | + |
| 69 | +Full details: [health-monitoring.md](health-monitoring.md). |
| 70 | + |
| 71 | +## ChainVersionCatalog and Registry Polling |
| 72 | + |
| 73 | +Each chain adapter declares a `VersionPolicy` that specifies: |
| 74 | + |
| 75 | +- `registry` — which registry client to use |
| 76 | +- `image` — the repository path |
| 77 | +- `tagPattern` — a regex that filters valid release tags (e.g. `^v\d+\.\d+\.\d+$`) |
| 78 | + |
| 79 | +The catalog controller polls each registry on a configurable interval (default 1 h), collects all matching tags, applies semver normalization and sorting, and writes the latest resolved tag into the `ChainVersionCatalog` status. The `BlockchainNodeReconciler` reads this value during `reconcileUpgrade`. |
| 80 | + |
| 81 | +Adapters where only a `:latest` tag is published (Aptos, Aurora, HyperLiquid, MegaETH, Monad) have `VersionPolicy` disabled — auto-tracking is not possible for these chains. |
| 82 | + |
| 83 | +## Auto-Upgrade State Machine |
| 84 | + |
| 85 | +``` |
| 86 | +Running ──[newer tag available]──> Upgrading |
| 87 | + | |
| 88 | + └──[rollout healthy]──> Running |
| 89 | + | |
| 90 | + └──[CrashLoopBackOff ≥3 restarts]──> Rolling back |
| 91 | + | |
| 92 | + └──> Running (previous tag restored) |
| 93 | +``` |
| 94 | + |
| 95 | +The operator records the previous image tag in an annotation on the StatefulSet before each upgrade. On rollback, that annotation is read and the image is reverted. The upgrade history (timestamp, from-tag, to-tag, outcome) is appended to `.status.upgradeHistory`. |
0 commit comments