Skip to content

Commit e09e002

Browse files
committed
docs: Add FAQ section for common questions
1 parent cbc35a9 commit e09e002

1 file changed

Lines changed: 85 additions & 0 deletions

File tree

README.md

Lines changed: 85 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -338,3 +338,88 @@ We use the AgentLab framework to run and manage our experiments \cite{workarena2
338338
```
339339
## Traces
340340
Traces from “The BrowserGym Ecosystem for Web Agent Research” paper are available in [Huggingface](https://huggingface.co/datasets/agentlabtraces/agentlabtraces/tree/main).
341+
342+
## Frequently Asked Questions (FAQ)
343+
344+
### What is AgentLab?
345+
346+
AgentLab is an open-source framework from ServiceNow for developing and evaluating web agents. It provides easy large-scale parallel experiments, building blocks for web agents, and unified LLM API support.
347+
348+
### What benchmarks does AgentLab support?
349+
350+
| Benchmark | Task Templates | Max Steps | Multi-tab | Hosted Method |
351+
|-----------|---------------|-----------|-----------|---------------|
352+
| WebArena | 812 | 30 | Yes | Docker |
353+
| WebArena-Verified | 812 | 30 | Yes | Self hosted |
354+
| WorkArena L1/L2/L3 | 33/341/341 | 30/50 | No | Demo instance |
355+
| WebLinx | 31586 | 1 | No | Dataset |
356+
| VisualWebArena | 910 | 30 | Yes | Docker |
357+
| AssistantBench | 214 | 30 | Yes | Live web |
358+
| GAIA | - | - | - | Live web (soon) |
359+
| MiniWoB | 125 | 10 | No | Static files |
360+
| OSWorld | 369 | - | - | Self hosted |
361+
| TimeWarp | 1386 | 30 | Yes | Self hosted |
362+
363+
### How do I install AgentLab?
364+
365+
```bash
366+
pip install agentlab
367+
playwright install
368+
```
369+
370+
AgentLab requires Python 3.11 or 3.12.
371+
372+
### What LLM providers are supported?
373+
374+
| Provider | Setup |
375+
|----------|-------|
376+
| OpenAI | `export OPENAI_API_KEY=your_key` |
377+
| OpenRouter | `export OPENROUTER_API_KEY=your_key` |
378+
| Azure | Configure in settings |
379+
| Self-hosted (TGI) | Configure endpoint |
380+
381+
### How do I run experiments?
382+
383+
1. Set environment variables:
384+
```bash
385+
export AGENTLAB_EXP_ROOT=<results directory>
386+
export OPENAI_API_KEY=<your key>
387+
```
388+
389+
2. Prepare benchmark (see setup links above)
390+
391+
3. Launch experiments with ray parallelization
392+
393+
### What features does AgentLab provide?
394+
395+
| Feature | Description |
396+
|---------|-------------|
397+
| Parallel Experiments | Scale experiments with ray |
398+
| Unified LLM API | Single interface for multiple providers |
399+
| BrowserGym Integration | Standard web agent interface |
400+
| Reproducibility | Built-in reproducibility features |
401+
| Leaderboard | Unified benchmark leaderboard |
402+
403+
### Where is the leaderboard?
404+
405+
Visit [Hugging Face Leaderboard](https://huggingface.co/spaces/ServiceNow/browsergym-leaderboard) for benchmark results.
406+
407+
### What is BrowserGym?
408+
409+
BrowserGym is the underlying interface for web agent benchmarks. See [BrowserGym repo](https://github.com/ServiceNow/BrowserGym) for details.
410+
411+
### Is AgentLab free to use?
412+
413+
AgentLab is open-source under Apache 2.0 license. It is meant for research, not consumer products.
414+
415+
### How can I contribute?
416+
417+
Contributions welcome via GitHub. See [AgentLab repo](https://github.com/ServiceNow/AgentLab) for guidelines.
418+
419+
### Where can I get help?
420+
421+
| Resource | Link |
422+
|----------|------|
423+
| GitHub | [ServiceNow/AgentLab](https://github.com/ServiceNow/AgentLab) |
424+
| BrowserGym Paper | [arXiv](https://arxiv.org/abs/2412.05467) |
425+
| Leaderboard | [HF Spaces](https://huggingface.co/spaces/ServiceNow/browsergym-leaderboard) |

0 commit comments

Comments
 (0)