Skip to content

Commit 012f6eb

Browse files
committed
correct api ref in tools & cli evals docs
1 parent b849705 commit 012f6eb

3 files changed

Lines changed: 13 additions & 13 deletions

File tree

CHANGELOG.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ Big CLI update with breaking changes and an overall focus on improving stability
99
- Support for multi-language Langium projects, previously we only handled the singular case
1010
- Added the `-y` flag to most commands in lai to accept defaults, allowing automatic setup in CI environments and as part of scripted workflows
1111
- Detection of outdated descriptors: loading a descriptor generated by an older LAI version now warns and recommends regenerating with `lai gen descriptor`
12-
- Detection of outdated `lai.config.jsonc` files (also prompts to regen with `lai init`)
12+
- Detection of outdated `lai.config.jsonc` files (also prompts to regen with `lai init config`)
1313
- Auto-summarization of validations over 1k lines in the default generated system prompt
1414
- Per-language validator detection: multi-language projects that add one validator per language are now picked up, and the generated system prompt adds a `Validation Rules` section for each
1515

packages/cli/docs/evaluations.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ Evaluation cases are written as `.eval.ts` files using the vitest-style testing
1010

1111
```typescript
1212
import { describe, evaluation, beforeEach } from 'langium-ai-tools/evals';
13-
import type { EvalContext } from 'langium-ai-tools/testing';
13+
import type { EvalContext } from 'langium-ai-tools/evaluator';
1414

1515
describe('Basic Code Generation', () => {
1616

packages/langium-ai-tools/README.md

Lines changed: 11 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -155,26 +155,26 @@ In general we stick to focusing on what Langium can do to help with evaluation,
155155
156156
### Testing API
157157
158-
Langium AI Tools provides a vitest-style testing API for writing programmatic evaluation test suites. This allows you to define test cases in TypeScript with familiar features like:
158+
Langium AI Tools provides a vitest-style evaluation API for writing programmatic evaluation suites. This allows you to define evaluation cases in TypeScript with familiar features like:
159159
160-
- **Test suites** with `describe()`, `describe.skip()`, and `describe.only()`
160+
- **Evaluation suites** with `describe()`, `describe.skip()`, and `describe.only()`
161161
- **Lifecycle hooks**: `beforeAll()`, `afterAll()`, `beforeEach()`, `afterEach()`
162-
- **Parametrized tests** with `evaluation.each()` for testing multiple data sets
163-
- **Test filtering** with `.skip()` and `.only()` modifiers
162+
- **Parametrized tests** with `evaluation.each()` for evaluating multiple data sets
163+
- **Evaluation filtering** with `.skip()` and `.only()` modifiers
164164
165165
```typescript
166-
import { describe, evaluation, beforeAll, afterAll, beforeEach } from 'langium-ai-tools/testing';
166+
import { describe, evaluation, beforeAll, afterAll, beforeEach } from 'langium-ai-tools/evals';
167167

168168
describe('DSL Generation Tests', () => {
169169
let model;
170170

171171
beforeAll(async () => {
172-
// runs once before all tests
172+
// runs once before all evals
173173
model = await setupModel();
174174
});
175175

176176
beforeEach(() => {
177-
// runs before each test
177+
// runs before each eval
178178
clearCache();
179179
});
180180

@@ -186,7 +186,7 @@ describe('DSL Generation Tests', () => {
186186
};
187187
});
188188

189-
// parametrized tests
189+
// parametrized evals
190190
evaluation.each([
191191
{ input: 'person Alice', expected: 'Alice' },
192192
{ input: 'person Bob', expected: 'Bob' }
@@ -198,17 +198,17 @@ describe('DSL Generation Tests', () => {
198198
});
199199

200200
afterAll(() => {
201-
// cleanup after all tests
201+
// cleanup after all evals
202202
cleanupModel();
203203
});
204204
});
205205
```
206206
207-
**For detailed documentation on the Testing API**, see the [Testing API Reference](../cli/docs/testing-api.md).
207+
**For detailed documentation on the Evaluation API**, see the [Evaluation API Reference](../cli/docs/evaluation-api.md).
208208
209209
### Evaluation Matrix
210210
211-
The Evaluation Matrix provides a framework for testing multiple model configurations against a set of test cases using Langium AI evaluators. This is particularly helpful when comparing across models, prompt strategies, RAG setups, or other variations in your AI stack.
211+
The Evaluation Matrix provides a framework for checking multiple model configurations against a set of evaluation cases using Langium AI evaluators. This is particularly helpful when comparing across models, prompt strategies, RAG setups, or other variations in your AI stack.
212212
213213
In practice an evaluation matrix can be helpful when deciding between which models or services to use up front, but this can also be done externally by levering the evaluator directly yourself.
214214

0 commit comments

Comments
 (0)