.gitignore

# Cargo build output — matches any target/ dir in the workspace tree,
# including per-crate ones left over from standalone-crate development.
target/
**/*.rs.bk

# Per-crate Cargo.lock files can appear when a crate is developed as its
# own standalone workspace (e.g. before being wired into the outer
# workspace). Only the top-level Cargo.lock is authoritative — except
# for `stringcheese-tokenizer-tiktoken-conformance`, which is
# deliberately excluded from the workspace so its parity test with
# `--all-features` does not force network access on every workspace
# build. Its lockfile must be committed so its own CI job (which uses
# `--locked`) has a reproducible dependency graph.
crates/*/Cargo.lock
!crates/stringcheese-tokenizer-tiktoken-conformance/Cargo.lock
# `stringcheese-tokenizer-hf-bench` is excluded from the workspace
# for the same reason (oracle deps `tokenizers-rs` + `tiktoken-rs`
# never enter the workspace `Cargo.lock`). Its own lockfile must be
# committed so its `--locked` CI job has a reproducible graph.
!crates/stringcheese-tokenizer-hf-bench/Cargo.lock
# `stringcheese-tokenizer-component-cl100k` is excluded from the
# workspace so its parity-real-vocab build.rs cache-fetch never
# enters `--workspace --all-features`. Its own lockfile must be
# committed so its `--locked` CI job has a reproducible graph.
!crates/stringcheese-tokenizer-component-cl100k/Cargo.lock

# Real HF-shape tokenizer.json blobs materialised into the
# `stringcheese-tokenizer-hf` conformance runner's local lookup root.
# These are tens of MB per checkpoint and belong under a per-contributor
# cache or the env-var `STRINGCHEESE_REAL_VOCABS_DIR`, not in-tree — see
# `docs/design/tokenizer-conformance.md` for the lookup contract.
crates/stringcheese-tokenizer-hf/tests/conformance/vocabs/