metadata_contract.md

Contract: metadata.json

Purpose

This document defines the schema for metadata.json, the build artifact generated by @speakeasy-api/docs-mcp-core alongside the LanceDB index. It acts as the bridge between the indexing pipeline and the runtime server, providing the dynamic taxonomy and operational metrics required to boot the MCP tools.

Schema Definition

{
  // Required semver for forward/backward compatibility checks.
  // Example: "1.1.0"
  "metadata_version": "1.1.0",

  // User-configured. The server injects this into dynamic tool descriptions
  // so the LLM knows what the corpus is and when to search it.
  "corpus_description": "Speakeasy SDK documentation",

  // Taxonomy fields discovered during indexing.
  // Keys are the field names.
  // The server maps each key into a JSON Schema enum on the search_docs tool.
  "taxonomy": {
    "language": {
      "description": "Filter results by programming language.",
      "values": ["go", "java", "python", "typescript"]
    },
    "scope": {
      "description": "Filter by documentation scope.",
      "values": ["global-guide", "sdk-specific"]
    }
  },

  // Corpus-level stats for observability and the eval harness.
  "stats": {
    "total_chunks": 2847,
    "total_files": 312,
    "indexed_at": "2026-02-21T14:32:00Z",

    // Optional source traceability marker when the corpus came from Git.
    // Null for non-git/local corpora.
    "source_commit": "4d2f9b4d6a42c377815f9f6c7db25a54d2aa9cf9"
  },

  // The embedding provider used at build time (if any).
  // Null means FTS-only index — the server disables vector search.
  "embedding": null,

  // Optional MCP prompts discovered from *.template.md and *.template.yaml files.
  // These files are excluded from search indexing and exposed via prompts/list + prompts/get.
  "prompts": [
    {
      "name": "guides/convert-currency",
      "title": "Convert Currency",
      "description": "Convert 100 USD to a target currency.",
      "arguments": [
        {
          "name": "currency",
          "description": "The target currency for conversion",
          "required": true
        }
      ],
      "messages": [
        {
          "role": "user",
          "content": {
            "type": "text",
            "text": "Convert 100 USD to {{currency}}. Use current forex mcp tools and APIs if available."
          }
        }
      ]
    }
  ]
}

embedding may also be:

{
  "provider": "openai",
  "model": "text-embedding-3-large",
  "dimensions": 3072
}

Normalization & Validation Rules (Required)

• metadata_version
  • Must be valid semver (MAJOR.MINOR.PATCH).
  • Server must fail fast if major version is unsupported.
• taxonomy normalization
  • Keys are trimmed.
  • values array: items are trimmed, deduplicated, sorted ascending. Canonical casing is preserved as discovered in the corpus (no forced lowercase).
  • Empty strings are rejected.
  • description is optional. If omitted, the server defaults to "Filter results by {key}."
• taxonomy limits
  • Max keys: 64
  • Max key length: 64 chars
  • Max values per key: 512
  • Max value length: 128 chars
• stats.source_commit
  • Optional.
  • If present, must be a 40-char lowercase Git SHA-1.
  • Used for provenance and eval traceability; never a runtime boot blocker.
• embedding
  • Type: null | { provider: string; model: string; dimensions: number }
  • If object is provided, dimensions must be > 0.
• prompts
  • Type: optional array of { name, title?, description?, arguments[], messages[] }.
  • name is required non-empty string.
  • messages must be a non-empty array of { role, content } where role is user|assistant and content is currently text ({ type: "text", text }).
  • arguments entries require name; support optional description and required.

Rationale for each field

• metadata_version: Enables explicit compatibility checks as the contract evolves, preventing silent runtime drift between indexers and servers.
• corpus_description: A key feature highlighted in the architecture documentation. It needs to flow from the build configuration (docs-mcp build --description "...") into the runtime server without requiring the server to parse external config files (like a gen.yaml) itself.
• taxonomy: This is the load-bearing field for the Dynamic Schema feature. The server reads it at boot to inject enum arrays into the search_docs JSON Schema. Keys are strictly dynamic (not hardcoded to language), guaranteeing the core engine remains domain-agnostic per the architecture's design goal.
• stats: Cheap to compute at index time, useful for the eval harness and the Host's telemetry pipeline. source_commit adds lightweight source provenance without introducing brittle boot-time coupling.
• embedding: The runtime server needs to know whether to execute vector search pathways (table.search().nearestTo()) or fall back to pure FTS if the index was built with --embedding-provider none. It is also required by the eval harness to record the specific provider/model permutation in its markdown delta reports.
• prompts: Allows docs authors to ship reusable MCP prompts next to docs content via *.template.md (single user-text shorthand) and *.template.yaml (multi-message format), while keeping prompt templates out of search indexing.

System Boundaries (Who Writes vs. Reads)

• Written by: @speakeasy-api/docs-mcp-cli (driving @speakeasy-api/docs-mcp-core) during the docs-mcp build step. Saved directly alongside the .lancedb/ directory.
• Read by: @speakeasy-api/docs-mcp-server at boot (to construct the dynamic JSON Schema for MCP tools) and @speakeasy-api/docs-mcp-eval (to record execution metadata in benchmarking reports).
• Never read by: The CLI authoring tools (validate, fix). These commands operate strictly on the markdown source content and manifests, not on build artifacts.