diff --git a/README.md b/README.md index bdee0a0..86a5177 100644 --- a/README.md +++ b/README.md @@ -1,112 +1,355 @@ -# verified-executable-search (VES) +# Verified Executable Search (VES) -Verifier-first executable search runtime: AI candidates produce raw -artifacts, the host verifies them into structured `Evidence`, and search only -acts on verified facts. Distribution name: `verified-executable-search` -(import package: `ves`). +> **AI can propose. It cannot grade itself.** -## What VES is +**VES is a verifier-first runtime for AI-generated executable solutions.** +LLMs can write candidate programs, search strategies can improve them, but the +candidate never gets to decide whether it is correct or good. The host loads +its artifact, recomputes the facts, records the evidence, and only then lets a +judge or search policy act on it. -- **Verifier-first**: candidate programs are untrusted; every quality number - is recomputed by a host `EvidenceVerifier` inside a - `VerificationContext`. Candidate self-reported scores are ignored. -- **Verification ≠ Judgment**: the pipeline establishes facts - (`VerificationStatus` / `VerificationResult`); feasibility and ranking live - in the typed `Judge` layer (`Gate` / `ObjectiveSpec` / `JudgeSpec`). -- **Search on verified facts only**: `AnchorPolicy` selects improvement - anchors from `VerifiedCandidate`s; candidates keep `Candidate` lineage - (draft/improve) for audit and repair. +[中文 README](README.zh-CN.md) · [v0.1.0](../../releases/tag/v0.1.0) · [Write your first verifier](docs/writing-a-verifier.md) -## What works in v0.1 (stable API) +VES is aimed at **autoresearch, scientific/modeling workflows, optimization, +and agentic code search** where a generated program must be evaluated against +host-owned truth, tests, constraints, or simulators. -- `ArtifactContract` / `SafeArtifactLoader` — trusted host specs + race-safe, - symlink-safe artifact reads -- `VerificationContext` / `Evidence` / `EvidenceVerifier` — - facts, not scores -- `Judge` / `Gate` / `ObjectiveSpec` / `JudgeSpec` — typed preferences -- `VerificationPipeline` / `VerificationRecord` / `VerifiedCandidate` — - verified results and auditable records (schema v1 with fingerprints) -- CLI `ves verify` (built-in examples + `--problem`) and `ves replay` - (REPRODUCIBLE / MISMATCH closed loop) +--- -## Experimental in v0.1 +## The problem -- `SearchEngine` + `AnchorPolicy` (`GreedyTop1Policy` ~ A arm, - `BranchDiverseTopKPolicy` ~ B arm; interface may evolve in v0.2) -- Real Docker + LLM search adapters (`examples/search_demo.py`) — source - checkout only, not part of the wheel +A typical agent loop looks like this: -## What remains experimental +```text +LLM writes code + ↓ +code reports a score + ↓ +agent trusts the score + ↓ +agent improves the code +``` + +That is convenient, but the trust boundary is wrong. A candidate can be buggy, +miscompute a metric, leak evaluation data, violate constraints, or simply print +a great-looking number. + +VES changes the loop to: + +```text +LLM / generator + ↓ +Candidate program untrusted + ↓ +Sandboxed runner + ↓ +Raw artifact untrusted + ↓ +──────────── host trust boundary ──────────── + ↓ +ArtifactContract + SafeArtifactLoader + ↓ +EvidenceVerifier + VerificationContext + ↓ +Evidence verified facts + ↓ +Judge preferences / feasibility + ↓ +Search acts only on verified evidence +``` + +The core rule is simple: -- Pareto search semantics (SearchEngine rejects `rule=PARETO`; v0.1 supports - total-order selection only) -- Stochastic verification replay determinism (deferred to v0.2) +> **Creation authority and judgment authority are separate.** -## Out of scope for v0.1 +--- -- Task decomposition / multi-agent orchestration -- MCTS / rollout-based planning -- Full mathematical modeling (mechanism design, statistical significance) +## 30-second demo -## Install +No LLM is required for the built-in demos. ```bash -# PyPI 发布前请从源码安装(Python 3.11+) git clone https://github.com/Zhuchen00123/Verified-Executable-Search.git cd Verified-Executable-Search pip install .[dev,ml,search] + +python -m examples.demo_adversarial +python -m examples.demo_search --mock +``` + +The adversarial demo shows that candidate claims do not become truth: + +```text +candidate says: objective = 999999 +VES: self-reported score ignored + +malformed / NaN / oversized / symlink artifact +VES: rejected or blocked + +valid artifact +VES: host recomputes the evidence +``` + +The search demo shows the same verification pipeline driving improvement: + +```text +draft #1 value=33 feasible=True +draft #2 value=40 feasible=True +improve value=51 feasible=True + +BEST VERIFIED SOLUTION: value=51 ``` -## Quick example +The point is not that knapsack is hard. The point is that **the search loop can +change while the verification boundary stays fixed**. + +--- + +## Why VES instead of one more `eval.py`? + +A hand-written evaluator is often enough for one experiment. VES becomes useful +when you want the evaluation boundary to be reusable, auditable, and safe to +connect to generated code. + +| Ad-hoc agent loop | VES | +|---|---| +| candidate may report its own metric | host recomputes observations | +| score and feasibility often mixed together | `Evidence` and `Judge` are separate | +| hidden truth handling is application-specific | host-owned `VerificationContext` | +| artifact loading is usually plain file I/O | size/path/symlink/UTF-8/JSON checks | +| comparison logic is embedded in scripts | typed gates, objectives and directions | +| search history is ephemeral | candidate lineage + verification records | +| reruns are informal | fingerprinted verification replay | + +VES does **not** try to implement a verifier for every domain. It provides a +**verification runtime**; your application supplies the domain truth. + +Examples: + +- regression: candidate emits predictions; host recomputes RMSE/MAE +- constrained optimization: candidate emits a solution; host recomputes the + objective and violations +- mechanism fitting: candidate emits parameters; host checks physical + constraints and fit quality +- code search: candidate emits executable behavior; host runs tests or a + benchmark + +--- + +## Core model + +VES deliberately separates facts from preferences. + +```text +Artifact + ↓ +Verifier + ↓ +Evidence ← facts + ↓ +Judge + ↓ +Judgment ← preferences +``` + +A verifier can say: ```python -from ves.artifact import ArtifactContract -from ves.judge import Direction, JudgeSpec, ObjectiveSpec - -contract = ArtifactContract( - filename="answer.json", - media_type="application/json", - required_fields=("value",), - numeric_fields=("value",), +Evidence( + observations=( + Observation(name="rmse", value=0.31, provenance="deterministic:host"), + Observation(name="runtime", value=1.8, provenance="deterministic:host"), + ) ) -spec = JudgeSpec( +``` + +The judge separately decides what matters: + +```python +JudgeSpec( objectives=( - ObjectiveSpec(observation="error", direction=Direction.MINIMIZE), + ObjectiveSpec(observation="rmse", direction=Direction.MINIMIZE), ), ) ``` -See `docs/writing-a-verifier.md` for a complete from-scratch example. +That distinction matters: `runtime` may be useful evidence without silently +becoming an optimization objective. + +--- + +## Public API + +The v0.1 facade is intentionally small and split into four layers. + +### Verification + +`RawArtifact` · `ArtifactContract` · `SafeArtifactLoader` · +`VerificationContext` · `Observation` · `Evidence` · `EvidenceVerifier` · +`VerificationStatus` · `VerificationResult` + +### Judgment + +`Gate` · `ObjectiveSpec` · `JudgeSpec` · `Direction` · `ComparisonRule` · +`ComparisonOutcome` · `Feasibility` · `Judge` · `Verdict` · `Comparison` + +### Problems and records + +`VerifiedProblem` · `VerificationPipeline` · `Candidate` · +`VerificationRecord` · `VerifiedCandidate` · `ComparisonRecord` + +### Search -## CLI +`CandidateGenerator` · `CodeRunner` · `RunResult` · `SearchEngine` · +`SearchResult` · `AnchorPolicy` · `GreedyTop1Policy` · +`BranchDiverseTopKPolicy` + +For new code, prefer the top-level facade: + +```python +from ves import ( + ArtifactContract, + Direction, + Evidence, + EvidenceVerifier, + JudgeSpec, + ObjectiveSpec, + Observation, + VerifiedProblem, +) +``` + +See [Writing a Verifier](docs/writing-a-verifier.md) for a complete problem +from scratch. + +--- + +## CLI: verify and replay + +Verify an artifact with a built-in example: ```bash ves verify solution.json --example knapsack --out record.json -ves replay record.json # REPRODUCIBLE / MISMATCH ``` -## Demos (no LLM required) +Or provide your own `VerifiedProblem`: ```bash -python -m examples.demo_adversarial # six adversarial verdicts -python -m examples.demo_search --mock # mock search evolution +ves verify answer.json --problem my_problem:problem --out record.json ``` -Real search needs `VES_LLM_BASE_URL` (or `AIDE_BASE_URL` fallback) + Docker; -see `docs/ves-demo.md` and `docs/ves-search.md`. +Replay the verification later: -## Documentation +```bash +ves replay record.json +``` + +Replay v1 checks the problem reference, contract/context/judge fingerprints, +verifier version, artifact hash, and newly recomputed evidence before reporting +`REPRODUCIBLE` or `MISMATCH`. + +This is **verification replay**, not bit-for-bit replay of the original LLM or +container execution. + +--- + +## Search + +`SearchEngine` is problem-agnostic. A domain provides only: + +```text +VerifiedProblem +CandidateGenerator +CodeRunner +AnchorPolicy +``` + +The same engine is tested across unrelated problem types; domain prompts and +artifact semantics stay outside the engine. + +```python +result = SearchEngine( + problem=problem, + generator=generator, + runner=runner, + drafts=3, + improves=5, +).search() +``` + +In v0.1, search supports total-order selection. Pareto judging exists, but +`SearchEngine` deliberately fails fast for Pareto search until frontier +semantics are defined properly. + +--- + +## Security model + +Generated code is untrusted. + +- `DockerProcessRunner` is the supported execution boundary for untrusted code + on Linux Docker. +- `LocalProcessRunner` is **not** a security boundary. +- `SafeArtifactLoader` rejects path traversal, symlinks (where `O_NOFOLLOW` is + available), non-regular files, oversized payloads, invalid UTF-8, malformed + JSON, and non-finite JSON values. +- hidden host truth is not mounted into the candidate container; CI includes a + real Docker attack test that attempts to read it. + +Read [SECURITY.md](SECURITY.md) before using VES with hostile or generated +code. + +--- + +## Status: v0.1.0 + +VES v0.1 is an **early experimental release**. -- `docs/writing-a-verifier.md` — new-user tutorial -- `docs/ves-cli.md` — CLI + replay + schema v1 -- `docs/ves-demo.md` / `docs/ves-search.md` — hero demos + SearchEngine -- `docs/ves-release-roadmap.md` — release checklist -- `SECURITY.md` — threat model, sandbox boundaries, private reporting +### Stable enough to build on -## CI status +- verifier-first artifact → evidence pipeline +- explicit verification / judgment separation +- typed objectives and feasibility gates +- verification records and deterministic replay +- candidate lineage +- safe artifact ingestion + +### Experimental + +- `SearchEngine` / `AnchorPolicy` APIs may evolve in v0.2 +- Pareto search is not implemented in v0.1 +- stochastic verifier replay semantics are deferred +- real LLM + Docker search adapters currently live in source-checkout examples, + not in the wheel + +VES is intentionally **not** a multi-agent framework, task decomposer, or full +mathematical-modeling system. Those belong above the core runtime. + +--- + +## Where this is going + +The next proving ground is **VES Modeling**: single-file iterative search for +well-defined computational modeling problems, where every candidate is graded +by independent host verification rather than by the model that generated it. + +The goal is not to make VES Core bigger. The goal is to find out whether this +trust boundary makes real AI modeling/search workflows more reliable, +reproducible, and easier to audit. + +--- + +## Documentation -See CI badge / workflow runs — no hand-maintained test counts. +- [Writing a Verifier](docs/writing-a-verifier.md) — start here +- [CLI and replay](docs/ves-cli.md) +- [Adversarial and search demos](docs/ves-demo.md) +- [Search architecture](docs/ves-search.md) +- [Security model](SECURITY.md) +- [Changelog](CHANGELOG.md) -## License +## License and attribution -MIT +MIT. VES is derived from and inspired by +[AIDE / aideml](https://github.com/WecoAI/aideml); see [NOTICE](NOTICE) for +attribution. diff --git a/README.zh-CN.md b/README.zh-CN.md index 92e1f65..16e88b5 100644 --- a/README.zh-CN.md +++ b/README.zh-CN.md @@ -1,109 +1,348 @@ -# verified-executable-search(VES) +# Verified Executable Search(VES) -Verifier-first 可执行搜索运行时:AI 候选产生原始产物,宿主将其验证为结构化 -`Evidence`,搜索只能基于经过验证的事实推进。发行名:`verified-executable-search` -(import 包名:`ves`)。 +> **AI 可以提出方案,但不能给自己打分。** -## VES 是什么 +**VES 是一个面向 AI 生成可执行解的 Verifier-first 验证与搜索运行时。** +LLM 可以写候选程序,搜索策略可以不断改进它,但候选本身没有资格决定“我是否正确、我有多好”。宿主负责读取产物、独立复算事实、记录 Evidence,再由 Judge 和 Search 基于这些可信事实做决策。 -- **Verifier-first**:候选程序不可信;每个质量数字都由宿主 - `EvidenceVerifier` 在 `VerificationContext` 内独立复算,候选自报分数一律 - 忽略。 -- **Verification ≠ Judgment**:pipeline 只建立事实(`VerificationStatus` / - `VerificationResult`);可行性判定与排名属于 typed `Judge` 层 - (`Gate` / `ObjectiveSpec` / `JudgeSpec`)。 -- **搜索只作用于已验证事实**:`AnchorPolicy` 从 `VerifiedCandidate` 选择改进 - 锚点;候选带 `Candidate` 血缘(draft/improve)供审计与 repair。 +[English README](README.md) · [v0.1.0](../../releases/tag/v0.1.0) · [从零编写 Verifier](docs/writing-a-verifier.md) -## v0.1 可用(稳定 API) +VES 面向 **Autoresearch、科学计算/建模、优化、Agentic Code Search** 等场景:AI 负责生成可执行方案,而评估依据来自宿主持有的隐藏真值、测试、约束、仿真器或外部求解器。 -- `ArtifactContract` / `SafeArtifactLoader` —— 可信宿主规格 + 竞态安全、 - 防符号链接的产物读取 -- `VerificationContext` / `Evidence` / `EvidenceVerifier` —— 事实,不是分数 -- `Judge` / `Gate` / `ObjectiveSpec` / `JudgeSpec` —— typed 偏好 -- `VerificationPipeline` / `VerificationRecord` / `VerifiedCandidate` —— - 验证结果与可审计记录(schema v1 含指纹) -- CLI `ves verify`(内置三例 + `--problem`)与 `ves replay` - (REPRODUCIBLE / MISMATCH 闭环) +--- -## v0.1 experimental +## 为什么需要 VES? -- `SearchEngine` + `AnchorPolicy`(`GreedyTop1Policy` ~ A 臂、 - `BranchDiverseTopKPolicy` ~ B 臂;接口可能在 v0.2 演进) -- 真实 Docker + LLM search adapters(`examples/search_demo.py`)——仅源码 - checkout 可用,不在 wheel 内 +很多 Agent 搜索循环实际上是: -## 仍属 experimental +```text +LLM 写代码 + ↓ +代码自己输出一个分数 + ↓ +Agent 相信这个分数 + ↓ +继续改代码 +``` + +这样很方便,但信任边界是错的。 + +候选程序可能: + +- 把指标算错; +- 泄漏评测数据; +- 违反约束但没有正确上报; +- 输出格式错误; +- 甚至直接打印一个“看起来很强”的分数。 + +VES 把流程改成: + +```text +LLM / Generator + ↓ +Candidate Program 不可信 + ↓ +Sandboxed Runner + ↓ +Raw Artifact 不可信 + ↓ +──────────── 宿主信任边界 ──────────── + ↓ +ArtifactContract + SafeArtifactLoader + ↓ +EvidenceVerifier + VerificationContext + ↓ +Evidence 已验证事实 + ↓ +Judge 偏好 / 可行性 + ↓ +Search 只消费已验证 Evidence +``` -- Pareto 搜索语义(SearchEngine 拒绝 `rule=PARETO`;v0.1 仅支持 - total-order selection) -- 随机化验证的 replay 确定性(推迟到 v0.2) +核心原则只有一句: -## v0.1 范围外 +> **生成权和裁判权必须分离。** -- 任务分解 / 多智能体编排 -- MCTS / rollout 规划 -- 完整数学建模(机理设计、统计显著性) +--- -## 安装 +## 30 秒体验 + +内置 Demo 不需要 LLM。 ```bash -# PyPI 发布前请从源码安装(Python 3.11+) git clone https://github.com/Zhuchen00123/Verified-Executable-Search.git cd Verified-Executable-Search pip install .[dev,ml,search] + +python -m examples.demo_adversarial +python -m examples.demo_search --mock ``` -## 快速示例 +对抗 Demo 会展示:候选自报成绩不会自动变成事实。 + +```text +candidate says: objective = 999999 +VES: 忽略候选自报分数 + +malformed / NaN / oversized / symlink artifact +VES: REJECTED / BLOCKED + +valid artifact +VES: 宿主重新计算 Evidence +``` + +Search Demo 则展示同一条验证链如何驱动迭代: + +```text +draft #1 value=33 feasible=True +draft #2 value=40 feasible=True +improve value=51 feasible=True + +BEST VERIFIED SOLUTION: value=51 +``` + +这里 Knapsack 本身并不重要。真正重要的是: + +> **搜索策略可以换,但可信验证边界不需要跟着重写。** + +--- + +## 为什么不是再写一个 `eval.py`? + +对于单次实验,一个手写 evaluator 完全够用。VES 的价值在于:当你希望把“生成代码 → 验证 → 比较 → 搜索”做成可复用、可审计、可以安全连接 AI 生成代码的运行时,很多原本散落在脚本里的问题需要一个统一边界。 + +| 普通 Agent Loop | VES | +|---|---| +| 候选可能自己报告指标 | Host 独立复算 Observation | +| 分数、约束和偏好经常混在一起 | `Evidence` 与 `Judge` 分离 | +| hidden truth 隔离靠每个项目自己实现 | Host-owned `VerificationContext` | +| 通常直接读文件 | 路径、symlink、大小、UTF-8、JSON 防护 | +| 比较逻辑散落在脚本里 | typed Gate / Objective / Direction | +| 搜索历史通常只是日志 | Candidate lineage + VerificationRecord | +| “复现”依赖人工重跑 | fingerprinted verification replay | + +VES **不试图给每一种问题都内置一个 Verifier**。 + +它提供的是一个 **Universal Verification Runtime**:领域真值由应用自己定义。 + +例如: + +- 回归:候选只输出 predictions,宿主重新计算 RMSE / MAE; +- 约束优化:候选输出 solution,宿主重新计算 objective 与 violation; +- 机理参数拟合:候选输出参数,宿主检查物理约束与拟合质量; +- Code Search:候选输出可执行行为,宿主运行测试或 benchmark。 + +--- + +## 核心模型:事实和偏好分开 + +VES 把验证过程明确拆成: + +```text +Artifact + ↓ +Verifier + ↓ +Evidence ← 事实 + ↓ +Judge + ↓ +Judgment ← 偏好 +``` + +Verifier 可以建立这样的事实: ```python -from ves.artifact import ArtifactContract -from ves.judge import Direction, JudgeSpec, ObjectiveSpec - -contract = ArtifactContract( - filename="answer.json", - media_type="application/json", - required_fields=("value",), - numeric_fields=("value",), +Evidence( + observations=( + Observation(name="rmse", value=0.31, provenance="deterministic:host"), + Observation(name="runtime", value=1.8, provenance="deterministic:host"), + ) ) -spec = JudgeSpec( +``` + +而 Judge 再单独决定优化目标: + +```python +JudgeSpec( objectives=( - ObjectiveSpec(observation="error", direction=Direction.MINIMIZE), + ObjectiveSpec(observation="rmse", direction=Direction.MINIMIZE), ), ) ``` -完整从零示例见 `docs/writing-a-verifier.md`。 +这样 `runtime` 可以是有价值的 Evidence,却不会因为“出现在结果里”就莫名其妙变成优化目标。 -## CLI +--- + +## v0.1 Public API + +v0.1 的 facade 有意保持很薄,分成四层。 + +### Verification + +`RawArtifact` · `ArtifactContract` · `SafeArtifactLoader` · +`VerificationContext` · `Observation` · `Evidence` · `EvidenceVerifier` · +`VerificationStatus` · `VerificationResult` + +### Judgment + +`Gate` · `ObjectiveSpec` · `JudgeSpec` · `Direction` · `ComparisonRule` · +`ComparisonOutcome` · `Feasibility` · `Judge` · `Verdict` · `Comparison` + +### Problem / Records + +`VerifiedProblem` · `VerificationPipeline` · `Candidate` · +`VerificationRecord` · `VerifiedCandidate` · `ComparisonRecord` + +### Search + +`CandidateGenerator` · `CodeRunner` · `RunResult` · `SearchEngine` · +`SearchResult` · `AnchorPolicy` · `GreedyTop1Policy` · +`BranchDiverseTopKPolicy` + +新代码建议直接从顶层 facade 导入: + +```python +from ves import ( + ArtifactContract, + Direction, + Evidence, + EvidenceVerifier, + JudgeSpec, + ObjectiveSpec, + Observation, + VerifiedProblem, +) +``` + +完整接入示例见 [Writing a Verifier](docs/writing-a-verifier.md)。 + +--- + +## CLI:Verify 与 Replay + +使用内置例子验证 artifact: ```bash ves verify solution.json --example knapsack --out record.json -ves replay record.json # REPRODUCIBLE / MISMATCH ``` -## Demo(无需 LLM) +或者传入自己的 `VerifiedProblem`: ```bash -python -m examples.demo_adversarial # 六段对抗判定 -python -m examples.demo_search --mock # mock 搜索演进 +ves verify answer.json --problem my_problem:problem --out record.json ``` -真实搜索需要 `VES_LLM_BASE_URL`(或 `AIDE_BASE_URL` fallback)+ Docker; -见 `docs/ves-demo.md` 与 `docs/ves-search.md`。 +之后可以重新验证: -## 文档 +```bash +ves replay record.json +``` + +Replay v1 会检查 problem reference、contract/context/judge fingerprints、verifier version、artifact hash,并重新运行 Verifier 生成新的 Evidence,然后输出: + +```text +REPRODUCIBLE +``` + +或: + +```text +MISMATCH +``` + +需要注意:这目前是 **verification replay**,并不是对原始 LLM 推理和容器执行环境做 bit-for-bit replay。 + +--- + +## Search + +`SearchEngine` 本身不认识 Knapsack、Regression 或 Mechanism Fitting。 + +一个领域只需要提供: + +```text +VerifiedProblem +CandidateGenerator +CodeRunner +AnchorPolicy +``` + +然后使用同一个 SearchEngine: -- `docs/writing-a-verifier.md` —— 新用户教程 -- `docs/ves-cli.md` —— CLI + replay + schema v1 -- `docs/ves-demo.md` / `docs/ves-search.md` —— 英雄 Demo + SearchEngine -- `docs/ves-release-roadmap.md` —— 发布检查清单 -- `SECURITY.md` —— 威胁模型、沙箱边界、私有漏洞报告 +```python +result = SearchEngine( + problem=problem, + generator=generator, + runner=runner, + drafts=3, + improves=5, +).search() +``` + +v0.1 只支持 total-order search。Judge 已经能表达 Pareto 比较,但 `SearchEngine` 会对 Pareto 模式明确 fail-fast,而不是偷偷把 `INCOMPARABLE` 强行排成唯一 best。 + +--- + +## 安全模型 + +AI 生成代码默认不可信。 + +- `DockerProcessRunner` 是 Linux Docker 下支持的不可信代码执行边界; +- `LocalProcessRunner` **不是**安全沙箱; +- `SafeArtifactLoader` 会拒绝路径逃逸、symlink(平台支持 `O_NOFOLLOW` 时)、非普通文件、超大文件、非法 UTF-8、错误 JSON 与非有限数值; +- hidden host truth 不挂载进候选容器;CI 中包含真实 Docker attack test,恶意候选会主动尝试读取隐藏真值。 + +如果要让 VES 执行不可信/AI 生成代码,请先阅读 [SECURITY.md](SECURITY.md)。 + +--- + +## 当前状态:v0.1.0 + +VES v0.1 是一个 **Early Experimental Release**。 + +### 已经适合作为底座验证 -## CI 状态 +- verifier-first Artifact → Evidence pipeline; +- Verification 与 Judgment 明确分离; +- typed Objective / Gate; +- VerificationRecord 与 deterministic replay; +- Candidate lineage; +- 安全 Artifact ingestion。 + +### 仍属 experimental + +- `SearchEngine` / `AnchorPolicy` API 可能在 v0.2 调整; +- v0.1 尚未实现 Pareto Search; +- stochastic verifier 的 replay 语义留到后续; +- 真实 LLM + Docker Search adapters 目前属于源码 checkout 示例,不在 wheel 内。 + +VES 目前**不是**多 Agent 框架、自动任务分解器,也不是完整数学建模系统。这些能力应该构建在 Core 之上,而不是塞进验证运行时。 + +--- + +## 下一步:VES Modeling + +VES Core 接下来的第一个真实试验场是 **VES Modeling**: + +> 对定义良好的计算建模子问题,让 AI 反复修改单文件可执行方案,但每一次成绩都由宿主独立验证,而不是由生成它的模型自己决定。 + +目标不是继续把 VES Core 做得越来越大,而是用真实建模任务验证: + +> **这种 trust boundary 是否真的能让 AI 建模/搜索更可靠、更容易复现,也更容易审计。** + +--- + +## 文档 -见 CI badge / workflow 运行记录——不手写维护测试数字。 +- [Writing a Verifier](docs/writing-a-verifier.md) —— 推荐从这里开始 +- [CLI 与 Replay](docs/ves-cli.md) +- [对抗 Demo / Search Demo](docs/ves-demo.md) +- [Search 架构](docs/ves-search.md) +- [安全模型](SECURITY.md) +- [Changelog](CHANGELOG.md) -## License +## License / Attribution -MIT +MIT。VES 源自并受到 [AIDE / aideml](https://github.com/WecoAI/aideml) 启发,详见 [NOTICE](NOTICE)。