Fix validation service and update API documentation

CONTRIBUTING.md

@@ -211,7 +211,7 @@ The BigQuery plugin integrates with HarperDB. When modifying plugin code:
 - `src/validation.js` - Data validation and auditing
 - `src/query-builder.js` - SQL query construction
 - `src/config-loader.js` - Configuration parsing (single/multi-table)
-- `schema/harper-bigquery-sync.graphql` - GraphQL schema
+- `schema/bigquery-ingestor.graphql` - GraphQL schema
 - `test/` - Unit tests for all core functionality
 
 ## Synthesizer Development

README.md

@@ -32,7 +32,7 @@ Install Harper and add this plugin:
 
 # Clone this plugin
 cd your-harper-project
-npm install @harperdb/harper-bigquery-sync
+npm install @harperdb/bigquery-ingestor
 ```
 
 ### 2. Configure (Your Data)
@@ -307,22 +307,115 @@ Perfect for testing sync performance, multi-table coordination, and distributed
 
 ## Monitoring & Operations
 
-### Check Sync Status
+### Distributed Sync Control
 
-Query the REST API:
+The plugin provides cluster-wide sync control via REST API. All commands replicate across nodes automatically.
+
+**Available Commands:**
 
 ```bash
-# Get current status
-curl http://localhost:9926/SyncControl
+# Get current status (GET)
+curl http://localhost:9926/SyncControl \
+  -u admin:HarperRocks!
 
-# Control sync
+# Start sync across entire cluster (POST)
 curl -X POST http://localhost:9926/SyncControl \
+  -u admin:HarperRocks! \
   -H "Content-Type: application/json" \
-  -d '{"action": "start"}'  # or "stop"
+  -d '{"action": "start"}'
+
+# Stop sync across entire cluster (POST)
+curl -X POST http://localhost:9926/SyncControl \
+  -u admin:HarperRocks! \
+  -H "Content-Type: application/json" \
+  -d '{"action": "stop"}'
+
+# Run validation across cluster (POST)
+curl -X POST http://localhost:9926/SyncControl \
+  -u admin:HarperRocks! \
+  -H "Content-Type: application/json" \
+  -d '{"action": "validate"}'
+```
+
+**Status Response Format:**
+
+```json
+{
+	"global": {
+		"command": "start",
+		"commandedAt": "2025-12-16T20:30:00Z",
+		"commandedBy": "node1-0",
+		"version": 42
+	},
+	"worker": {
+		"nodeId": "node1-0",
+		"running": true,
+		"tables": [
+			{ "tableId": "vessel_positions", "running": true, "phase": "steady" },
+			{ "tableId": "port_events", "running": true, "phase": "catchup" }
+		],
+		"failedEngines": []
+	},
+	"uptime": 3600,
+	"version": "2.0.0"
+}
+```
+
+- **global**: Cluster-wide sync command state (replicated across all nodes via HarperDB)
+- **worker**: This specific worker thread's status
+- **nodeId**: Identifies worker as `hostname-workerIndex`
+- **tables**: Status per sync engine (one per configured table)
+- **failedEngines**: Any engines that failed to start
+
+### Data Validation
+
+Run validation to verify data integrity across the cluster:
+
+```bash
+curl -X POST http://localhost:9926/SyncControl \
+  -u admin:HarperRocks! \
+  -H "Content-Type: application/json" \
+  -d '{"action": "validate"}'
+```
+
+**Validation performs three checks per table:**
+
+1. **Progress Check** - Verifies sync is advancing, checks for stalled workers
+   - Status: `healthy`, `lagging`, `severely_lagging`, `stalled`, `no_checkpoint`
+
+2. **Smoke Test** - Confirms recent data (last 5 minutes) is queryable
+   - Status: `healthy`, `no_recent_data`, `query_failed`, `table_not_found`
+
+3. **Spot Check** - Validates data integrity bidirectionally
+   - Checks if Harper records exist in BigQuery (detects phantom records)
+   - Checks if BigQuery records exist in Harper (detects missing records)
+   - Status: `healthy`, `issues_found`, `no_data`, `check_failed`
+
+**View validation results:**
+
+```bash
+# Get recent validation audits
+curl http://localhost:9926/SyncAudit/ \
+  -u admin:HarperRocks!
 ```
 
+Each validation run creates audit records with:
+
+- `timestamp` - When validation ran
+- `nodeId` - Which worker performed the validation
+- `status` - Overall status: `healthy`, `issues_detected`, or `error`
+- `checkResults` - JSON with detailed results per table and check
+
 ### View Checkpoints
 
+```bash
+# REST API
+curl http://localhost:9926/SyncCheckpoint/ \
+  -u admin:HarperRocks!
+```
+
+Or query via SQL:
+
 ```sql
 -- Check sync progress per node
 SELECT * FROM SyncCheckpoint ORDER BY nodeId;
@@ -336,15 +429,45 @@ SELECT
 FROM SyncCheckpoint;
 ```
 
-### View Validation Results
+### Access Synced Data
 
-```sql
--- Check recent audits
-SELECT * FROM SyncAudit
-WHERE timestamp > NOW() - INTERVAL '1 hour'
-ORDER BY timestamp DESC;
+All synced tables are accessible via REST API:
+
+```bash
+# Query vessel positions (note trailing slash)
+curl http://localhost:9926/VesselPositions/ \
+  -u admin:HarperRocks!
+
+# Query port events
+curl http://localhost:9926/PortEvents/ \
+  -u admin:HarperRocks!
+
+# Query vessel metadata
+curl http://localhost:9926/VesselMetadata/ \
+  -u admin:HarperRocks!
+```
+
+**Important:** REST endpoints require a trailing slash (`/TableName/`) to return data arrays. Without the trailing slash, you get table metadata instead of records.
+
+### Postman Collection
+
+A comprehensive Postman collection is included for testing all endpoints:
+
+```bash
+# Import into Postman
+bigquery-ingestor_postman.json
 ```
 
+**Collection includes:**
+
+- **Cluster Control** - Start, stop, validate commands
+- **Status Monitoring** - Check sync status and worker health
+- **Data Verification** - Query PortEvents, VesselMetadata, VesselPositions
+- **Checkpoint Inspection** - View sync progress per node
+- **Audit Review** - Check validation results
+
+**Authentication:** Uses Basic Auth with default credentials (admin / HarperRocks!). Update the collection variables if using different credentials.
+
 ### Query Synced Data
 
 ```sql

ROADMAP.md

@@ -11,9 +11,9 @@ We've shipped v2.0 with comprehensive multi-table support, column selection, and
 - ✅ **Multi-table support** - Sync multiple BigQuery tables simultaneously with independent settings
 - ✅ **Column selection** - Reduce costs by fetching only needed columns from BigQuery
 - ✅ **Per-table configuration** - Independent batch sizes, sync intervals, and strategies per table
-- ✅ **Exponential backoff retry logic** - Smart retry with jitter for transient BigQuery errors ([#3](https://github.com/HarperFast/harper-bigquery-sync/issues/3))
-- ✅ **Comprehensive logging** - Structured logging throughout codebase for Grafana observability ([#11](https://github.com/HarperFast/harper-bigquery-sync/issues/11))
-- ✅ **Optional streaming insert API** - Configurable streaming inserts for production deployments ([#8](https://github.com/HarperFast/harper-bigquery-sync/issues/8))
+- ✅ **Exponential backoff retry logic** - Smart retry with jitter for transient BigQuery errors ([#3](https://github.com/HarperFast/bigquery-ingestor/issues/3))
+- ✅ **Comprehensive logging** - Structured logging throughout codebase for Grafana observability ([#11](https://github.com/HarperFast/bigquery-ingestor/issues/11))
+- ✅ **Optional streaming insert API** - Configurable streaming inserts for production deployments ([#8](https://github.com/HarperFast/bigquery-ingestor/issues/8))
 - ✅ **Multi-table validation** - Independent validation and monitoring per table
 - ✅ **Backward compatibility** - Single-table format still supported
 
@@ -26,15 +26,15 @@ We've shipped v2.0 with comprehensive multi-table support, column selection, and
 
 ### Maritime Data Synthesizer
 
-- ✅ **Multi-table orchestrator** - Generate realistic data for multiple related tables ([#6](https://github.com/HarperFast/harper-bigquery-sync/issues/6))
+- ✅ **Multi-table orchestrator** - Generate realistic data for multiple related tables ([#6](https://github.com/HarperFast/bigquery-ingestor/issues/6))
 - ✅ **Rolling window mode** - Automatic data window maintenance and backfill
 - ✅ **100K+ vessel simulation** - Realistic maritime tracking data at global scale
 - ✅ **Physics-based movement** - Realistic navigation patterns
 - ✅ **Automatic retention** - Configurable rolling window with cleanup
 
 ### Project Quality
 
-- ✅ **Memory leak fixes** - Journey tracking system optimized ([#5](https://github.com/HarperFast/harper-bigquery-sync/issues/5))
+- ✅ **Memory leak fixes** - Journey tracking system optimized ([#5](https://github.com/HarperFast/bigquery-ingestor/issues/5))
 - ✅ **CI/CD pipeline** - Automated lint, test, and format checks
 - ✅ **Comprehensive documentation** - User guides, API docs, design documents
 - ✅ **Project history** - Development milestones and evolution tracking
@@ -44,15 +44,15 @@ We've shipped v2.0 with comprehensive multi-table support, column selection, and
 
 ### Multi-Threaded Ingestion
 
-- [ ] **Multi-threaded ingestion per node** ([#9](https://github.com/HarperFast/harper-bigquery-sync/issues/9))
+- [ ] **Multi-threaded ingestion per node** ([#9](https://github.com/HarperFast/bigquery-ingestor/issues/9))
   - Better CPU utilization on multi-core nodes
   - Code already supports durable thread identity via `hostname-workerIndex`
   - Thread-level checkpointing for fine-grained recovery
   - Automatic thread scaling based on lag
 
 ### Dynamic Rebalancing
 
-- [ ] **Dynamic rebalancing for autoscaling** ([#10](https://github.com/HarperFast/harper-bigquery-sync/issues/10))
+- [ ] **Dynamic rebalancing for autoscaling** ([#10](https://github.com/HarperFast/bigquery-ingestor/issues/10))
   - Detect topology changes → pause → recalculate → resume
   - Graceful node additions/removals without manual intervention
   - Zero-downtime scaling capabilities
@@ -69,7 +69,7 @@ We've shipped v2.0 with comprehensive multi-table support, column selection, and
 
 ### Dynamic Schema Management
 
-- [ ] **Dynamic Harper table creation via Operations API** ([#7](https://github.com/HarperFast/harper-bigquery-sync/issues/7))
+- [ ] **Dynamic Harper table creation via Operations API** ([#7](https://github.com/HarperFast/bigquery-ingestor/issues/7))
   - Currently requires manual schema.graphql definition
   - Could dynamically create tables based on BigQuery schema at runtime
   - Enables automatic table creation from BigQuery metadata
@@ -82,7 +82,7 @@ These are potential enhancements without specific commitments:
 
 ### Production Operations
 
-- [ ] **Production deployment documentation** ([#4](https://github.com/HarperFast/harper-bigquery-sync/issues/4))
+- [ ] **Production deployment documentation** ([#4](https://github.com/HarperFast/bigquery-ingestor/issues/4))
   - Fabric deployment guide with one-click setup
   - Self-hosted installation for on-premise clusters
   - Monitoring dashboards (Grafana/CloudWatch templates)
@@ -143,7 +143,7 @@ These are potential enhancements without specific commitments:
 
 Want to help build v3.0 or tackle future considerations? See [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.
 
-Check out [open issues on GitHub](https://github.com/HarperFast/harper-bigquery-sync/issues) for specific tasks you can pick up.
+Check out [open issues on GitHub](https://github.com/HarperFast/bigquery-ingestor/issues) for specific tasks you can pick up.
 
 ---