hindsight

DO NOT USE (YET): hindsight is experimental and the api/architecture are in flux.

Unified observability hub for Bearcove tools. Distributed tracing + live introspection over roam rpc.

Status

Active development; expect breaking changes.

• Plan/spec: PLAN.md
• Archived drafts: docs/archive/PLAN_v1.md, docs/archive/PLAN_v2_picante.md
• Current plan uses a single HTTP port with Upgrade to select transport (roam vs websocket) + GET / for the ui bootstrap page.

what is hindsight?

hindsight will be a trace collection server + ui that:

• will collect W3C Trace Context spans from apps (via roam rpc transports),
• will discover app capabilities at runtime (service introspection),
• and will adapt its ui dynamically (generic trace views + framework-specific views).

the goal is one place to debug:

• roam (rpc): topology/transport/active calls
• picante (incremental): query graphs + cache hit/miss/validation (see: architecture, guide)
• dodeca (build): build progress/pages/template stats (see: features, query reference, template engine, debugging templates, plugins)

Philosophy

pure roam. one protocol end-to-end. http exists only to serve a tiny static page that loads the browser ui; trace data flows over roam.

extensible by discovery. apps expose introspection services; hindsight calls ServiceIntrospection.list_services() and enables views accordingly.

ephemeral by default. in-memory storage with ttl (persistence/export are planned).

avoid self-tracing loops. hindsight's own roam sessions are untraced; tracing in apps is explicit opt-in.

integration with bearcove projects

hindsight aims to provide generic tracing plus framework-specific views when the app exposes introspection services.

roam (rpc framework)

use roam_stream::connect;
use hindsight::Tracer;

// Create a tracer that exports spans to hindsight.
let tracer = Tracer::connect_http("localhost:1990").await?;

// Emit spans manually
let span = tracer.span("my_operation")
    .with_attribute("key", "value")
    .start();
// ... do work ...
span.end();

picante (incremental computation)

use picante::Runtime;
use hindsight::Tracer;

let tracer = /* ... */;
let runtime = Runtime::new()
    .with_tracer(tracer); // Planned: emit spans with picante.* attributes

// Query execution shows up as spans
let result = db.my_query.get(&db, key).await?;

dodeca (static site generator)

use hindsight::Tracer;

let tracer = /* ... */;

// See your entire build pipeline traced:
// File change → Markdown parse → Image optimization → Template render

Architecture

NOTE: diagram is aspirational; current code does not implement the full hub/discovery/ui.

Apps (native / WASM)                     hindsight (hub)
┌─────────────────────────┐              ┌──────────────────────────┐
│ App emits spans          │──roam RPC──▶│ hindsightservice         │
│ + exposes introspection  │              │ - ingest_spans           │
│ services (optional)      │◀─roam RPC───│ - list/get/stream traces │
└─────────────────────────┘              │                          │
                                         │ UI adapts based on:      │
                                         │ - ServiceIntrospection   │
                                         │ - PicanteIntrospection   │
                                         │ - RoamIntrospection      │
                                         │ - DodecaIntrospection    │
                                         └──────────────────────────┘

Workspace Structure

crates/
├── hindsight/          # Client library (emit/export spans)
├── hindsight-server/   # Server binary (`hindsight`)
├── hindsight-tui/      # TUI client (planned; currently a stub)
└── hindsight-protocol/ # Shared protocol types + RPC service trait

Features

• ✅ W3C Trace Context (traceparent/tracestate)
• ✅ Pure roam rpc ingestion (tcp + websocket transport)
• ✅ Ephemeral in-memory store (TTL)
• 🚧 Service discovery driven ui (planned: dynamic tabs per app capabilities; not implemented yet)
• 🚧 Framework-specific views (planned: picante/roam/dodeca via introspection; not implemented yet)
• 🚧 Persistence / sampling / export (planned; not implemented yet)

Links

• W3C Trace Context: https://www.w3.org/TR/trace-context/
• OpenTelemetry: https://opentelemetry.io/
• HTTP Upgrade (101 Switching Protocols): https://developer.mozilla.org/en-US/docs/Web/HTTP/Protocol_upgrade_mechanism
• WebSocket protocol (RFC 6455): https://www.rfc-editor.org/rfc/rfc6455
• Workspace: Cargo.toml
• Workspace crates: crates/
• Protocol types: crates/hindsight-protocol/src/trace_context.rs, crates/hindsight-protocol/src/span.rs, crates/hindsight-protocol/src/service.rs
• Server entrypoint (router + upgrade handlers): crates/hindsight-server/src/main.rs
• In-memory store: crates/hindsight-server/src/storage.rs

Example: Distributed Trace Across Systems

// mockup: this is the kind of cross-tool trace hindsight aims to show (not working yet)

// In your web server
let span = tracer.span("handle_request").start();

// Make an RPC call (trace context auto-propagated)
let result = rpc_client.call(method, payload).await?;

// That RPC triggers a picante query in another process
// All show up in ONE trace:
//
// handle_request (50ms)
//   ├─ RPC: calculate (40ms)
//   │   ├─ picante: load_data (5ms, cache hit)
//   │   └─ picante: compute (35ms, recomputed)
//   └─ format_response (10ms)

span.end();

Development

Build:

cargo build --workspace

Run tests:

cargo test --workspace

Run the server locally:

cargo run -p hindsight-server -- serve

Contributing

See PLAN.md for the detailed design doc/spec.

Contributions welcome! Please open issues and PRs.

License

Licensed under either of:

• Apache License, Version 2.0 (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
• MIT license (LICENSE-MIT or http://opensource.org/licenses/MIT)

at your option.