_index.md

title	Experiments
description	An overview of Agent Observability Experiments feature.
aliases	/llm_observability/experiments_preview /llm_observability/experiments/
further_reading	link tag text /llm_observability/improve/experiments/setup Documentation Set up and use Agent Observability Experiments link tag text https://www.datadoghq.com/blog/llm-experiments/ Blog Create and monitor LLM experiments with Datadog link tag text https://www.datadoghq.com/blog/llm-otel-semantic-convention Blog Datadog LLM Observability natively supports OpenTelemetry GenAI Semantic Conventions link tag text https://www.datadoghq.com/blog/manage-ai-cost-and-performance-with-datadog/ Blog Driving AI ROI: How Datadog connects cost, performance, and infrastructure so you can scale responsibly link tag text https://www.datadoghq.com/blog/llm-observability-at-datadog-nlq Blog How we cut our NLQ agent debugging time from hours to minutes with LLM Observability link tag text https://www.datadoghq.com/blog/offline-llm-evaluations/ Blog Offline evaluation for AI agents: Best practices link tag text https://www.datadoghq.com/blog/datadog-google-cloud-ai-stack/ Blog Evaluate, optimize, and secure your Google Cloud AI stack with Datadog link tag text https://www.datadoghq.com/blog/llm-experimentation-autoresearch/ Blog How we made a SQL query optimization agent 59% more accurate using autoresearch and Agent Observability link tag text https://www.datadoghq.com/blog/from-traces-to-experiments-a-loop-for-improving-ai-agents/ Blog From traces to experiments: A loop for improving AI agents

{{< img src="llm_observability/experiments/Experiments_LLMO.png" alt="Agent Observability, Experiment view. Heading: 'Comparing 6 experiments across 9 fields'. Line graph visualization charting the accuracy, correctness, duration, estimated cost, and other metrics of various experiments." style="width:100%;" >}}

Agent Observability Experiments supports the entire lifecycle of building LLM applications and agents. It helps you understand how changes to prompts, models, providers, or system architecture affect performance. With this feature, you can:

• Create and version datasets
• Run and manage experiments
• Compare results to evaluate impact

Cookbooks

For in-depth examples of what you can do with LLM Experiments, see Datadog's provided Jupyter notebooks.

Further reading

{{< partial name="whats-next/whats-next.html" >}}