Skip to content

Commit 987a8d1

Browse files
committed
feat: Complete QSE harness implementation with automation and documentation
- Fixed double instrumentation by removing manual spans - Updated TEST_SCENARIOS with comprehensive evaluation metric coverage - Implemented cross-platform automated test runner (run_qse_harness.sh) - Added comprehensive README.md with architecture, flow map, and debugging guide - Created Docker deployment configuration with updated Dockerfile.alpha - Added Kubernetes CronJob configuration for automated testing - Cleaned up directory structure by removing over-engineered files - Fixed macOS compatibility issues (timeout command) - Added proper scenario indexing and environment variable handling - Included deployment assets migration from manual/ to multi-agent-qse/ QSE harness now provides: - 6 comprehensive test scenarios covering all evaluation metrics - Automated execution with logging and error handling - Production-ready deployment configurations - Complete documentation and troubleshooting guide
1 parent c3694a1 commit 987a8d1

22 files changed

+984
-4394
lines changed

instrumentation-genai/opentelemetry-instrumentation-langchain/examples/manual/.env

Lines changed: 0 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,3 @@
1-
OPENAI_API_KEY=
2-
#Circuit
3-
CLIENT_ID=
4-
CLIENT_SECRET=
5-
APP_KEY=
61
# OpenTelemetry Basic Configuration
72
OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317
83

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
# OpenTelemetry Basic Configuration
2+
OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317
3+
4+
# OpenTelemetry Advanced Configuration for GenAI Development
5+
OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE=DELTA
6+
OTEL_LOGS_EXPORTER=otlp
7+
OTEL_PYTHON_LOGGING_AUTO_INSTRUMENTATION_ENABLED=true
8+
OTEL_SERVICE_NAME=ai-test-svc-val
9+
OTEL_RESOURCE_ATTRIBUTES=deployment.environment=ai-test-val
10+
11+
# GenAI Instrumentation Configuration
12+
OTEL_SEMCONV_STABILITY_OPT_IN=gen_ai_latest_experimental
13+
OTEL_INSTRUMENTATION_GENAI_EMITTERS=span_metric_event,splunk
14+
OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=true
15+
OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT_MODE=SPAN_AND_EVENT
16+
#OTEL_INSTRUMENTATION_GENAI_EVALS_EVALUATORS=deepeval(LLMInvocation(bias,toxicity,hallucination,relevance,sentiment))
17+
OTEL_INSTRUMENTATION_GENAI_EVALS_RESULTS_AGGREGATION=true

instrumentation-genai/opentelemetry-instrumentation-langchain/examples/multi-agent-qse/Dockerfile

Lines changed: 0 additions & 41 deletions
This file was deleted.

0 commit comments

Comments
 (0)