[docs] update readme

Melmaphother · Melmaphother · commit 1f463abc68dd · 2026-03-26T17:41:33.000+08:00
diff --git a/README.md b/README.md
@@ -1,4 +1,4 @@
-<h1 align="center"> Claw-R1: Empowering OpenClaw with <br> Advanced Agentic RL. </h1>
+<h1 align="center"> Claw-R1: The Data Foundation for <br> Agentic Reinforcement Learning </h1>
 
 <p align="center">
   <a href="https://agentr1.github.io/"><img src="https://img.shields.io/badge/Project-Home-orange.svg" alt="Project Home"></a>
@@ -13,37 +13,39 @@
 
 - **[2026.03.06]** 📖 **Claw-R1 Documentation Released.** Project page and documentation are now available at [Claw-R1 Project Page](https://agentr1.github.io/) and [Claw-R1 docs](https://agentr1.github.io/Claw-R1/).
 
-- **[2026.03.03]** 🚧 **Claw-R1 Project Init.** We are actively updating the framework. Stay tuned for more features and documentation.
+- **[2026.03.03]** 🚧 **Claw-R1 Project Init.** We are actively developing the framework. Stay tuned for more features and documentation.
 
 ## Overview
 
-**Agentic RL** has become the dominant approach for training powerful LLM agents. Meanwhile, **General Agents** (e.g., OpenClaw, Claude Code, Open Code, etc.) have emerged as game-changing systems that redefine what agents can do. Yet there remains critical gaps:
+The **Agentic RL** ecosystem is thriving — frameworks like [verl](https://github.com/volcengine/verl), [Agent-R1](https://github.com/0russwest0/Agent-R1), and [MiniMax Forge](https://www.minimax.io/news/forge-scalable-agent-rl-framework-and-algorithm) have made remarkable progress in RL runtime and training algorithms. Meanwhile, **General Agents** (e.g., [OpenClaw](https://github.com/openclaw/openclaw), Claude Code, Open Code) are producing interaction data that is far richer and more complex than traditional ReAct trajectories.
 
-- **General Agent for Agentic RL**: Traditional Agentic RL frameworks typically rely on simple agents like ReAct. General agents (e.g., OpenClaw, Claude Code, Open Code) offer far richer capabilities—but existing RL pipelines were not designed for them.
+As agents grow more capable, a critical question emerges: **How do we systematically collect, evaluate, and curate high-quality training data from diverse agent interactions?** This is a relatively under-explored yet important direction — especially when human feedback is available as a natural quality signal.
 
-- **Agentic RL for General Agent**: Modern base models have not been fully adapted to thrive inside general agent architectures. We aim to enable models to play a larger, more effective role within these next-generation agents.
-
-**Claw-R1** is training framework that bridges this gap. It introduces a **Middleware Layer** (Gateway Server + DataPool) as the sole bridge between Agent Side and Training Side. Agents—white-box or black-box—access the framework via standard HTTP. This enables three modes: white-box offline, black-box offline, and black-box online service. No framework today adequately supports this paradigm—Claw-R1 is designed to fill that void.
+**Claw-R1** provides the **data foundation** for Agentic RL. It introduces a Middleware Layer (Gateway + DataPool) between the Agent Side and the Training Side, focusing on data collection, evaluation, and curation rather than training algorithms themselves.
 
 <p align="center"><img src="./assets/framework.png" width="800px" alt="Claw-R1 Framework" /></p>
 
 ## Key Features
 
-- **Asynchronous Training & Rollout**: Decouples RL training from rollout in the framework, enabling scalable and efficient data collection and model updates.
-
-- **Agent–Training Decoupling**: Supports online-service agents where execution and training run independently. Data flows from live user requests into DataPool; the Trainer continuously fetches batches for training—no dataset required.
+- **Universal Data Collection**: White-box agents submit Steps via API; black-box agents integrate by simply pointing `base_url` to the Gateway (zero code changes); online services collect data from live user interactions in real-time.
 
-- **Zero-Code Intrusion**: Black-box agents (LangChain, AutoGen, CrewAI, etc.) integrate with zero modification—just point `base_url` to the Gateway. The framework automatically collects interaction data and trains models.
+- **Data Evaluation & Curation**: Multi-dimensional reward system (rule-based / discriminative RM / generative RM), human feedback signal integration, policy version tracking for freshness-aware curation, and channel-based data partitioning.
 
+- **Flexible Data Serving**: Pluggable `TrainingBackend` to convert curated data into any training engine's native format, with GRPO-aware grouping, train/val channel isolation, and real-time monitoring.
 
 ## Get Started
 
-Explore our comprehensive documentation for setup, configuration, and advanced usage:
-
 - 📖 **[Full Documentation](https://agentr1.github.io/Claw-R1/)**
-- 🚀 [Installation Guide](docs/Installation.md)
+- 🚀 [Installation Guide](docs/getting-started/installation.md)
 - 🛠️ [Architecture Overview](https://agentr1.github.io/Claw-R1/components/)
 
+## Roadmap
+
+- [ ] **Data Quality Dashboard**: Visual monitoring of data quality metrics, reward distributions, and collection statistics.
+- [ ] **Human Feedback Pipeline**: Structured pipeline for capturing and integrating explicit and implicit human feedback signals from online agent services.
+- [ ] **Dataset Export & Versioning**: Export curated datasets with full provenance tracking for reproducibility and sharing.
+- [ ] **Extended TrainingBackend Support**: Native adapters for additional RL frameworks beyond verl.
+
 ## Contributors
 
 **Team Members**: Daoyu Wang, Jie Ouyang, Shuo Yu
@@ -52,16 +54,15 @@ Explore our comprehensive documentation for setup, configuration, and advanced u
 
 **Affiliation**: State Key Laboratory of Cognitive Intelligence, University of Science and Technology of China
 
-
 ## Acknowledgements
 
-Claw-R1 builds upon [Agent-R1](https://github.com/0russwest0/Agent-R1). We extend our gratitude to [MiniMax Forge](https://www.minimax.io/news/forge-scalable-agent-rl-framework-and-algorithm) for their architectural insights on the Middleware design, and to [rLLM](https://github.com/rllm-org/rllm) for their pioneering work on RL framework design for language agents. We also thank [OpenClaw](https://github.com/openclaw/openclaw) for their remarkable work on personal AI assistants—the modern agent paradigm that inspires our vision. We are grateful to the broader Agentic RL community and all contributors for their support.
+We extend our gratitude to [Agent-R1](https://github.com/0russwest0/Agent-R1), [MiniMax Forge](https://www.minimax.io/news/forge-scalable-agent-rl-framework-and-algorithm), [verl](https://github.com/volcengine/verl), and [rLLM](https://github.com/rllm-org/rllm) for their pioneering work on Agentic RL training infrastructure. We also thank [OpenClaw](https://github.com/openclaw/openclaw) for their remarkable work on personal AI assistants. We are grateful to the broader Agentic RL community and all contributors for their support.
 
 ## Citation
 
 ```bibtex
 @misc{clawr1-2026,
-  title={Claw-R1: Agentic RL for Modern Agents},
+  title={Claw-R1: The Data Foundation for Agentic Reinforcement Learning},
   author={Wang, Daoyu and Ouyang, Jie and Yu, Shuo and Cheng, Mingyue and Liu, Qi},
   year={2025},
   howpublished={\url{https://github.com/AgentR1/Claw-R1}},
diff --git a/docs/components/datapool.md b/docs/components/datapool.md
@@ -1,19 +1,20 @@
 # DataPool
 
-DataPool 是一个 **Ray Actor**，作为 Agent 侧（Gateway）和 Training 侧（Trainer）之间的中央 trajectory 缓冲区。
+DataPool 是 Claw-R1 的**数据管理核心** — 一个 Ray Actor，承担着 Agent 交互数据的存储、索引、质量追踪、分区管理和按需供给。它不仅是 Agent 侧与 Training 侧之间的缓冲区，更是整个数据基础设施的中枢。
 
 ## 在架构中的角色
 
 ```
-Gateway ──► DataPool.submit_step()      (异步，fire-and-forget)
-Trainer ◄── DataPool.fetch_batch()      (阻塞拉取，batch-ready 的组)
+Gateway ──► DataPool.submit_steps()     (数据采集：异步写入)
+Trainer ◄── DataPool.fetch_batch()      (数据供给：阻塞拉取就绪组)
+            DataPool.get_statistics()   (数据监控：实时统计)
 ```
 
-DataPool 完全解耦了写入速度（由 Agent 请求频率驱动）和读取速度（由训练吞吐量驱动）。双方互不等待。
+DataPool 完全解耦了数据采集速度（由 Agent 请求频率驱动）和数据消费速度（由训练吞吐量驱动）。双方互不等待。
 
-## Channel 系统
+## Channel 系统（数据分区）
 
-DataPool 通过 **channel** 对数据进行分区。默认 channel 为 `"train"`，验证流程使用 `"val"` channel 以隔离数据。
+DataPool 通过 **channel** 对数据进行分区管理。默认 channel 为 `"train"`，验证流程使用 `"val"` channel 以隔离数据。
 
 ```python
 # 训练数据
@@ -81,18 +82,18 @@ while True:
         train_on_batch(batch)
 ```
 
-## 容量管理
+## 容量管理与背压控制
 
-当设置 `max_queue_size` 时，DataPool 在队列满时丢弃最旧的就绪组，防止 Trainer 较慢时内存无限增长：
+当设置 `max_queue_size` 时，DataPool 在队列满时自动丢弃最旧的就绪组，防止数据堆积导致内存无限增长。这种背压机制也确保了训练侧消费的数据尽可能新鲜：
 
 ```yaml
 async_training:
   max_queue_size: null   # null = 无限
 ```
 
-## Training Backend
+## Training Backend（数据供给适配）
 
-DataPool 使用 `TrainingBackend` 将 `list[Step]` 转换为训练引擎的原生格式：
+DataPool 通过可插拔的 `TrainingBackend` 将 `list[Step]` 转换为任意训练引擎的原生格式，实现数据管理与训练框架的解耦：
 
 ```python
 class VerlBackend(TrainingBackend):
@@ -106,9 +107,9 @@ class VerlBackend(TrainingBackend):
         ...
 ```
 
-## Off-policy 支持
+## Off-policy 支持（数据新鲜度管控）
 
-Trainer 可以通过 staleness threshold 配置来处理历史（off-policy）数据：
+每个 Step 都记录了生成时的 `policy_version`，DataPool 和 Trainer 可以据此判断数据的新鲜度。Trainer 通过 staleness threshold 配置来处理历史（off-policy）数据：
 
 ```yaml
 async_training:
diff --git a/docs/components/index.md b/docs/components/index.md
@@ -1,77 +1,83 @@
 # Components
 
-Claw-R1 由六个独立可运行的组件组成，通过 HTTP 和 Ray RPC 通信。
+Claw-R1 的组件围绕**数据流**组织：从 Agent 交互的采集，到数据的管理与质量评估，再到向训练引擎的供给。各组件通过 HTTP 和 Ray RPC 通信。
 
 <div class="grid cards" markdown>
 
--   **Gateway Server**
+-   **Gateway Server** · 数据采集入口
 
     ---
 
-    FastAPI HTTP 服务。所有 Agent LLM 调用的网络层入口。管理 vLLM 负载均衡，自动收集训练数据并提交到 DataPool。
+    FastAPI HTTP 服务。所有 Agent LLM 调用的统一入口，自动从交互中采集训练数据（Step）并提交到 DataPool。支持白盒显式提交和黑盒自动采集两种模式。
 
     [:octicons-arrow-right-24: Gateway Server](gateway.md)
 
--   **DataPool**
+-   **DataPool** · 数据管理核心
 
     ---
 
-    Ray Actor。Agent 侧和 Training 侧之间的中央 trajectory 缓冲区。支持 Gateway 的异步写入和 Trainer 的批量读取。
+    Ray Actor。Claw-R1 的数据管理中枢 — 存储、索引、分区和供给交互数据。支持 Channel 隔离、GRPO 分组、容量背压控制和实时统计监控。
 
     [:octicons-arrow-right-24: DataPool](datapool.md)
 
--   **Agent Flow**
+-   **Reward System** · 数据质量评估
 
     ---
 
-    Agent 执行生命周期管理框架。支持白盒（Python）和黑盒（OpenAI API）两种模式。
+    `RewardLoopWorker` Ray Actor。多维度数据质量评估：rule-based、discriminative RM、generative RM，以及人类反馈信号的整合。
 
-    [:octicons-arrow-right-24: Agent Flow](agent-flow.md)
+    [:octicons-arrow-right-24: Reward System](reward-system.md)
 
--   **Black-box Agent**
+-   **Agent Flow** · 白盒数据采集
 
     ---
 
-    黑盒 Agent 系统。任何使用 OpenAI 兼容 API 的 Agent 通过 `base_url` 透明接入训练循环。
+    Agent 执行生命周期管理。白盒 Agent 通过 Python API 显式提交 Step，完整控制数据采集过程。
 
-    [:octicons-arrow-right-24: Black-box Agent](blackbox-agent.md)
+    [:octicons-arrow-right-24: Agent Flow](agent-flow.md)
 
--   **Async Training**
+-   **Black-box Agent** · 黑盒数据采集
 
     ---
 
-    `AsyncTrainer` 和 `AsyncRollouter` Ray Actor。持续、非阻塞的训练循环，带参数同步。
+    零代码侵入的黑盒 Agent 接入。任何使用 OpenAI 兼容 API 的 Agent 通过 `base_url` 透明接入，Gateway 自动采集交互数据。
 
-    [:octicons-arrow-right-24: Async Training](async-training.md)
+    [:octicons-arrow-right-24: Black-box Agent](blackbox-agent.md)
 
--   **Reward System**
+-   **Async Training** · 数据消费与训练
 
     ---
 
-    `RewardLoopWorker` Ray Actor。从 rule-based、discriminative 或 generative reward model 计算 step 级别的 reward。
+    `AsyncTrainer` 和 `AsyncRollouter` Ray Actor。持续从 DataPool 消费高质量数据进行训练，带参数同步。
 
-    [:octicons-arrow-right-24: Reward System](reward-system.md)
+    [:octicons-arrow-right-24: Async Training](async-training.md)
 
 </div>
 
-## 组件交互图
+## 数据流全景
 
 ```
+                        数据采集层
                       ┌─────────────────────────────────────────┐
   黑盒 Agent ────────►│                                         │
   (base_url)          │         GATEWAY SERVER                  │
                       │         (FastAPI, 端口 8100)             │
-  白盒 Agent ────────►│                                         │
+  白盒 Agent ────────►│         自动采集交互 Step                 │
   (AgentFlow)         └────────────┬────────────────────────────┘
-                                   │ Ray RPC (submit_step)
+                                   │ Ray RPC (submit_steps)
                                    ▼
+                        数据管理层
                       ┌─────────────────────────────────────────┐
                       │         DATAPOOL                         │
                       │         (Ray Actor)                      │
-                      │         Channel: train / val              │
+                      │                                          │
+                      │  • 存储与索引    • Channel 分区            │
+                      │  • GRPO 分组     • 容量背压控制            │
+                      │  • 质量评估      • 实时统计监控            │
                       └──────────────────┬──────────────────────┘
                                          │ fetch_batch()
                                          ▼
+                        数据消费层
                       ┌─────────────────────────────────────────┐
                       │         ASYNC TRAINER                    │
                       │         (Ray Actor, Training GPU Pool)   │
diff --git a/docs/concepts/index.md b/docs/concepts/index.md
@@ -1,59 +1,60 @@
 # Core Concepts
 
-Claw-R1 的设计围绕三个核心概念展开，它们共同构成一个闭环飞轮。
+Claw-R1 的设计围绕三个核心概念展开：**通用数据采集**、**数据中间件管理**和**数据驱动的持续进化**。它们共同构成一个从采集到训练的数据飞轮。
 
 <div class="grid cards" markdown>
 
--   **Base URL Integration**
+-   **Base URL Integration** · 通用数据采集
 
     ---
 
-    零代码侵入的黑盒 Agent 接入机制。任何使用 OpenAI 兼容 API 的 Agent 只需修改 `base_url` 即可透明接入训练系统。
+    零代码侵入的 Agent 数据采集机制。任何使用 OpenAI 兼容 API 的 Agent 只需修改 `base_url`，Gateway 即可自动采集其交互数据。
 
     [:octicons-arrow-right-24: Base URL Integration](base-url-integration.md)
 
--   **Middleware Layer**
+-   **Middleware Layer** · 数据中间件
 
     ---
 
-    Gateway + DataPool 中间件架构。完全解耦 Agent 侧和 Training 侧，支持异步数据收集和训练。
+    Gateway + DataPool 数据基础设施。统一解决数据的采集入口、质量管理、分区缓冲和按需供给。
 
     [:octicons-arrow-right-24: Middleware Layer](middleware-layer.md)
 
--   **Production Scenario**
+-   **Production Scenario** · 数据驱动进化
 
     ---
 
-    "部署 = 训练" 范式。Agent 在服务用户的同时持续收集数据和改进，无需离线重训。
+    "部署 = 训练" 范式。Agent 在服务用户的同时持续采集交互数据，用户行为天然成为数据质量信号，驱动模型持续进化。
 
     [:octicons-arrow-right-24: Production Scenario](production-scenario.md)
 
 </div>
 
-## 闭环飞轮
+## 数据飞轮
 
 ```
                     base_url
                  ┌────────────┐
-                 │ 黑盒 Agent  │
-                 │ (任意框架)   │
+                 │ 任意 Agent  │
+                 │ (白盒/黑盒) │
                  └──────┬─────┘
                         │ OpenAI API
                         ▼
               ┌──────────────────┐
-              │    Gateway       │ ← Middleware Layer
-              │  (自动收集数据)   │
+              │    Gateway       │ ← 数据采集入口
+              │  (自动采集 Step)  │
               └────────┬─────────┘
                        │
                        ▼
               ┌──────────────────┐
-              │    DataPool      │
+              │    DataPool      │ ← 数据管理核心
+              │  (评估·筛选·供给) │    (质量评估 + 分区管理)
               └────────┬─────────┘
                        │
                        ▼
               ┌──────────────────┐
-              │    Trainer       │ ← Production Scenario
-              │  (持续训练)       │    (部署 = 训练)
+              │    Trainer       │ ← 数据消费
+              │  (持续训练)       │
               └────────┬─────────┘
                        │ 权重同步
                        ▼
@@ -65,6 +66,6 @@ Claw-R1 的设计围绕三个核心概念展开，它们共同构成一个闭环
 
 三个概念的协同：
 
-1. **Base URL** 让任何 Agent 零成本接入
-2. **Middleware** 异步收集和缓冲训练数据
-3. **Production Scenario** 让模型在服务中持续进化
+1. **Base URL** 让任何 Agent 的交互数据零成本被采集
+2. **Middleware** 管理数据的质量、分区和供给
+3. **Production Scenario** 让人类反馈信号自然融入数据，驱动模型持续进化
diff --git a/docs/concepts/middleware-layer.md b/docs/concepts/middleware-layer.md
diff --git a/docs/index.md b/docs/index.md
