Skip to content
iTranscendPublic

About

barebones p2p network for distributed AI workloads

Resources

Stars

1 star

Watchers

1 watching

Forks

Latest commit

 

History

74 Commits

Folders and files

Repository files navigation

Thala

A barebones peer-to-peer network for distributed AI workloads.

Features

  • Peer Discovery: Automatic peer discovery through bootstrap nodes
  • Authenticated Handshake: Mutual ed25519 challenge-response — each peer proves possession of the key behind its PeerId before being admitted; spoofed identities are rejected
  • Connection Management: Maintains persistent connections with automatic reconnection
  • Exponential Backoff: Smart retry logic with configurable backoff parameters
  • Persistent Identity: Node keys are saved to disk for identity persistence across restarts and used to authenticate the node to its peers
  • JSON-RPC Server: Optional HTTP server for querying node state
  • Keep-Alive: Heartbeat mechanism to keep connections alive
  • GPU detection: GPU capability detection for use in workloads (currently supports Nvidia GPUs)

Quick Start

Ensure Rust is installed

1. Run a Bootstrap Node

Start a bootstrap node that other peers can connect to:

cargo run -- -l 127.0.0.1:5674

2. Connect Additional Nodes

Start additional nodes and connect them to the bootstrap node:

cargo run -- -b 127.0.0.1:5674 -l 127.0.0.1:2675

You can specify multiple bootstrap nodes for resilience. If one is unavailable, the node will still join via the others:

cargo run -- -b 127.0.0.1:5674,127.0.0.1:5675 -l 127.0.0.1:2675

Each node will:

  • Attempt to connect to all specified bootstrap nodes concurrently
  • Discover other peers in the network
  • Maintain persistent connections with automatic reconnection

3. Run with RPC Server

Enable the RPC server to query node information:

cargo run -- -l 127.0.0.1:2345 --rpc-addr 127.0.0.1:3330

Configuration

Command-Line Options

Usage: thala [OPTIONS]

Options:
  -l, --listen-address <LISTEN_ADDRESS>
          Address and port to listen on for peer connections
          [default: 127.0.0.1:2345]

  -b, --bootstrap-nodes <BOOTSTRAP_NODES>
          Bootstrap node address(es) to connect to on startup (comma-separated)

  -p, --peer-reconnection-interval <PEER_RECONNECTION_INTERVAL>
          Time interval in seconds between reconnection attempts
          [default: 30]

  -x, --backoff-multiplier <BACKOFF_MULTIPLIER>
          Multiplier for exponential backoff on failed reconnections
          [default: 2.0]

  -m, --max-backoff-interval <MAX_BACKOFF_INTERVAL>
          Maximum backoff interval in seconds
          [default: 2000]

  -r, --reconnection-retries-cap <RECONNECTION_RETRIES_CAP>
          Maximum number of reconnection attempts before giving up
          [default: 13]

  --claim-window <CLAIM_WINDOW>
          Seconds a coordinator collects claims for a task before picking a worker
          [default: 2]

  -a, --models <MODELS>
          Models this node can run (comma-separated)
          [default: resnet50]

  -q, --datasets <DATASETS>
          Datasets this node has available (comma-separated)
          [default: gpqa]

  --rpc-addr <RPC_ADDR>
          RPC server listening address (optional). Bind to loopback,
          e.g. 127.0.0.1:3330, unless remote access is required

  --rpc-token <RPC_TOKEN>
          Bearer token required on every RPC call (optional)
          [env: THALA_RPC_TOKEN]

  -d, --data-dir <DATA_DIR>
          Data directory for storing node identity and state
          [default: ~/.thala]

  -h, --help
          Print help information

Example: Custom Configuration

cargo run -- \
  -b 127.0.0.1:2345,127.0.0.1:2350 \
  -l 127.0.0.1:2346 \
  -p 30 \
  -m 300 \
  -x 2.0 \
  -r 10 \
  --rpc-addr 127.0.0.1:3331 \
  -d ./my-node-data

Task lifecycle

Any node can coordinate a task: the node a client submits it to becomes its coordinator. A task moves Pending → Running → Completed | Failed.

  1. Submit. A client calls broadcast_task on a node. That node records the task as Pending and sends a TaskAnnouncement to each peer it is directly connected to. Announcements are not yet forwarded further.
  2. Claim. A peer that receives the announcement ignores it if it has seen the task before, or if the task expires less than 60 seconds from now. If the peer can run the task, it sends a TaskClaim back to the coordinator and keeps its own Pending copy.
  3. Assign. The coordinator only accepts a claim if it comes from the claiming peer itself, and if that peer's advertised capabilities match the task. The first accepted claim opens a claim window (--claim-window, 2s by default). When the window closes, the claim with the lowest estimated duration wins, and ties go to the earliest claim. The coordinator marks the task Running and sends a TaskAssignment to the winner. Losing peers are not told; their copies stay Pending until the task expires.
  4. Execute. The winner checks that the assignment came from the task's coordinator, marks the task Running, executes it, and sends a TaskResult back to the coordinator.
  5. Complete. The coordinator only accepts a result from the assigned worker, and only while the task is Running. A success marks the task Completed; a failure marks it Failed. The coordinator stores the result with the task.
  6. Collect. The client polls task_status (or thala-client status <id>) on the coordinator to read the status, the assigned worker and the result.

Each node removes a task and everything it knows about it once the task's expiry time has passed, checking once a minute. Fetch results before then.

Execution is not implemented yet. Workers currently report every task as Failed with the error no executor for <kind> tasks yet, so the protocol can be exercised end to end before real engines are wired in.

Node capabilities

A node claims a task only if it can run it. Its capabilities are its detected hardware plus the --models and --datasets it is started with:

Task Requires
Benchmark model in --models and dataset in --datasets
Training model in --models, dataset in --datasets, and total GPU memory of at least min_vram if set
Inference model in --models, and total GPU memory of at least min_vram if set

GPU memory is summed across all detected GPUs. A node with no detected GPU cannot claim any task that sets min_vram.

For example, this node can claim benchmarks of llama-3-8b or resnet50 against gpqa or mmlu:

cargo run -- -b 127.0.0.1:5674 -l 127.0.0.1:2675 \
  --models llama-3-8b,resnet50 --datasets gpqa,mmlu

The coordinator checks claims against the capabilities the claiming peer advertised during the handshake, so a peer cannot win a task it did not advertise support for.

RPC API

When --rpc-addr is specified, a JSON-RPC server is started at the given address. Bind it to loopback (127.0.0.1) unless you need remote access; see RPC authentication below.

Available Methods

Method Description Returns
info Get node information Node ID, listen address, peer counts
peers Get all known peers List of peer IDs and addresses
connections Get active connections List of currently connected peers
capabilities Get a node's capabilities cores, memory, gpus, models
task_status Get a task this node coordinates, by task ID Task with status, assigned worker and result, or null if unknown
broadcast_task Announce a task to the network (state-mutating) true if broadcast

Example

List active connections:

curl -X POST http://127.0.0.1:3330 \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $THALA_RPC_TOKEN" \
  -d '{
    "jsonrpc": "2.0",
    "method": "connections",
    "params": [],
    "id": 3
  }'

RPC authentication

Start the node with a token to require Authorization: Bearer <token> on every method. Prefer the environment variable, since command-line flags are visible in the process list:

export THALA_RPC_TOKEN=$(openssl rand -hex 32)
cargo run -- -l 127.0.0.1:2345 --rpc-addr 127.0.0.1:3330

Tokens are compared in constant time. Calls without a valid token fail with JSON-RPC error code -32001.

Configuration Read-only methods broadcast_task
Token set token required token required
No token, loopback --rpc-addr open open
No token, non-loopback --rpc-addr open (a warning is logged) refused

The token is sent in plaintext over HTTP. To reach a node remotely, put the RPC port behind TLS (a reverse proxy or SSH tunnel) rather than exposing it directly.

Thala CLI client

You can also interact with any thala node via the CLI client

Install

cargo install --locked --path client --bin thala-client

Pass the token with --token, or set THALA_RPC_TOKEN, when the node requires one.

Get node capabilities

thala-client --node-rpc http://127.0.0.1:3330 capabilities

Submit tasks to the network

Nodes claim tasks they are capable of: the model (and dataset, where applicable) must be in the node's --models/--datasets lists, and --min-vram-gb (optional) requires the node's aggregate GPU VRAM to meet the given amount.

thala-client --node-rpc http://127.0.0.1:3330 task benchmark --model resnet50 --dataset gpqa
thala-client --node-rpc http://127.0.0.1:3330 task inference --model resnet50 --min-vram-gb 16
thala-client --node-rpc http://127.0.0.1:3330 task training --model resnet50 --dataset gpqa --min-vram-gb 24

Check a task's status

Query the node the task was submitted to, using the ID printed when it was created. The report shows the status (Pending, Running, Completed or Failed), the assigned worker, and the worker's result once it has reported one. Finished tasks are kept until they expire.

thala-client --node-rpc http://127.0.0.1:3330 status 00ee0e4a-388b-4102-b53d-b7791f27f252

Security

Peer connections are mutually authenticated at handshake time: each side signs a challenge bound to the connection (fresh nonces from both peers, a role byte, and the signer's identity claims), verified against the ed25519 public key inlined in its claimed PeerId. A peer that cannot produce a valid signature is rejected before it enters connections or known_peers.

Two limitations to be aware of:

  • Gossiped peer info (known_peers shared during the handshake) is hearsay until that peer is dialed and authenticates directly. Gossip never overwrites entries learned from an authenticated handshake.
  • Traffic is authenticated at handshake time but not encrypted — transport encryption (Noise) is a planned follow-up.

Peer Reconnection

The node implements exponential backoff for reconnection attempts:

  1. Initial reconnection interval: peer-reconnection-interval seconds
  2. After each failed attempt, the interval is multiplied by backoff-multiplier
  3. The interval is capped at max-backoff-interval seconds
  4. After reconnection-retries-cap failures, the peer is considered dead

Environment Variables

Set RUST_LOG to control logging verbosity:

# Show all logs
RUST_LOG=trace cargo run

# Show only warnings and errors
RUST_LOG=warn cargo run

# Show info from thala, debug from other modules
RUST_LOG=thala=info,debug cargo run

About

barebones p2p network for distributed AI workloads

Resources

Stars

1 star

Watchers

1 watching

Forks

Releases

Packages

Used by

Contributors

Languages