QSE-4732 alpha release testing support

instrumentation-genai/opentelemetry-instrumentation-langchain/examples/alpha-release-testing/.gitignore

@@ -0,0 +1,86 @@
+# Alpha Release Testing - Git Ignore
+
+# Environment files with credentials
+.env
+.env.*
+!.env.*.template
+config/.env.*
+!config/.env.*.template
+
+# Python
+__pycache__/
+*.py[cod]
+*$py.class
+*.so
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+
+# Virtual Environment
+.venv/
+venv/
+ENV/
+env/
+
+# Testing
+.pytest_cache/
+.coverage
+htmlcov/
+*.cover
+.hypothesis/
+.tox/
+logs/*.log
+logs/*.html
+logs/*.xml
+logs/*.json
+!logs/.gitkeep
+
+# IDE
+.vscode/
+.idea/
+*.swp
+*.swo
+*~
+.DS_Store
+
+# Playwright
+test-results/
+playwright-report/
+playwright/.cache/
+
+# Temporary files
+*.tmp
+*.bak
+*.swp
+temp/
+tmp/
+
+# Test data
+test_data/
+*.db
+*.sqlite
+
+# Screenshots (UI tests)
+screenshots/
+*.png
+!docs/*.png
+
+# Credentials and secrets
+secrets/
+*.pem
+*.key
+*.crt
+credentials.json

instrumentation-genai/opentelemetry-instrumentation-langchain/examples/alpha-release-testing/README.md

@@ -0,0 +1,185 @@
+# Alpha Release Testing
+
+Manual testing framework for validating Alpha release AI observability features against customer documentation.
+
+## 📁 Structure
+
+```
+alpha-release-testing/
+├── config/
+│   └── .env                         # Single configuration file (edit this)
+├── tests/apps/                      # Test applications
+│   ├── retail_shop_langchain_app.py # NEW: Retail multi-agent (unified traces)
+│   ├── langchain_evaluation_app.py  # LangChain multi-agent (6 scenarios)
+│   ├── langgraph_travel_planner_app.py  # LangGraph workflow (5 agents)
+│   ├── direct_azure_openai_app.py   # Manual GenAI instrumentation
+│   └── traceloop_travel_planner_app.py  # Traceloop translator
+├── docs/
+│   ├── ALPHA_RELEASE_TEST_PLAN.md   # Test plan with all use cases
+│   └── TEST_EXECUTION_CHECKLIST.md  # Execution tracking
+└── README.md                        # This file
+```
+
+## 🎯 Purpose
+
+Validate customer documentation use cases:
+- Instrument AI Applications (zero-code & code-based)
+- LangChain/LangGraph instrumentation
+- Traceloop SDK integration
+- Configuration settings
+- Splunk APM UI verification
+
+## 🚀 Quick Start
+
+### One-Time Setup
+
+```bash
+cd alpha-release-testing
+
+# Run setup script (one time only)
+./setup.sh
+
+# Edit config/.env and verify your OPENAI_API_KEY
+vim config/.env
+```
+
+### Run Tests (Automated)
+
+```bash
+# Run all tests once (includes both zero-code and manual modes)
+./run_tests.sh
+
+# Run only LangChain test
+./run_tests.sh langchain
+
+# Run LangGraph test (both zero-code and manual modes)
+./run_tests.sh langgraph
+
+# Run LangGraph with zero-code instrumentation only
+./run_tests.sh langgraph_zerocode
+
+# Run LangGraph with manual instrumentation only
+./run_tests.sh langgraph_manual
+
+# Run all tests continuously every 30 seconds
+./run_tests.sh loop_30
+
+# Run only LangChain test every 60 seconds
+./run_tests.sh langchain loop_60
+
+# Run only LangGraph test every 120 seconds
+./run_tests.sh langgraph loop_120
+```
+
+The script automatically:
+- Activates virtual environment
+- Loads environment variables (with proper export)
+- Runs selected test application(s)
+- **LangGraph runs in BOTH modes**: Zero-code (opentelemetry-instrument) and Manual (hardcoded)
+- Shows summary of results
+- **Loop mode**: Runs continuously at specified intervals (Press Ctrl+C to stop)
+
+---
+
+## 📝 Manual Setup (Alternative)
+
+If you prefer manual setup:
+
+### 1. Install Dependencies
+
+```bash
+cd alpha-release-testing
+
+# Create virtual environment
+uv venv .venv-langchain
+source .venv-langchain/bin/activate
+
+# Install pip
+uv pip install pip
+
+# Install local Splunk packages
+pip install -e ../../../../util/opentelemetry-util-genai --no-deps && \
+pip install -e ../../../../util/opentelemetry-util-genai-emitters-splunk --no-deps && \
+pip install -e ../../../../util/opentelemetry-util-genai-evals --no-deps && \
+pip install -e ../../../../util/opentelemetry-util-genai-evals-deepeval && \
+pip install -e ../../../../instrumentation-genai/opentelemetry-instrumentation-langchain/
+```
+
+### 2. Configure Environment
+
+```bash
+# Edit the single .env file
+vim config/.env  # Update OPENAI_API_KEY, SPLUNK_REALM, SPLUNK_ACCESS_TOKEN
+
+# Export environment variables (important!)
+set -a
+source config/.env
+set +a
+```
+
+### 3. Run Tests Manually
+
+```bash
+cd tests/apps
+
+# LangChain evaluation (6 scenarios)
+python langchain_evaluation_app.py
+
+# LangGraph travel planner - Manual instrumentation (hardcoded)
+python langgraph_travel_planner_app.py
+
+# LangGraph travel planner - Zero-code instrumentation
+opentelemetry-instrument python langgraph_travel_planner_app.py
+```
+
+## 📊 Verify in Splunk APM
+
+1. Navigate to Splunk APM (check your `SPLUNK_REALM` in config/.env)
+   - rc0: https://app.rc0.signalfx.com
+   - us1: https://app.us1.signalfx.com
+   - lab0: https://app.lab0.signalfx.com
+2. Go to **APM → Traces**
+3. Search for service: `sf_service:alpha-release-test`
+4. Verify:
+   - Agent names appear correctly
+   - Evaluation metrics visible
+   - Token usage tracked
+   - Trace hierarchy correct
+
+## 📚 Documentation
+
+- **Test Plan**: `docs/ALPHA_RELEASE_TEST_PLAN.md` - All test cases and use cases
+- **Checklist**: `docs/TEST_EXECUTION_CHECKLIST.md` - Track execution progress
+- **Test Apps**: `tests/apps/README.md` - Detailed app documentation
+
+## 🔧 Troubleshooting
+
+**Environment variables not loaded:**
+```bash
+# Verify environment is loaded
+echo $OPENAI_API_KEY
+echo $OTEL_SERVICE_NAME
+
+# Reload if needed
+source config/.env
+```
+
+**Import errors:**
+```bash
+# Verify virtual environment is active
+which python  # Should show .venv-langchain/bin/python
+
+# Reinstall packages if needed
+pip install -e ../../../../instrumentation-genai/opentelemetry-instrumentation-langchain/
+```
+
+**No telemetry in Splunk:**
+- Check OTEL Collector is running: `curl http://localhost:4317`
+- Verify `OTEL_EXPORTER_OTLP_ENDPOINT` in `.env`
+- Check service name matches in Splunk APM
+
+---
+
+**Status**: Ready for manual testing  
+**Configuration**: Single `config/.env` file (no templates)  
+**Last Updated**: November 12, 2025

instrumentation-genai/opentelemetry-instrumentation-langchain/examples/alpha-release-testing/config/.env.lab0.template

@@ -0,0 +1,65 @@
+# Alpha Release Testing - lab0 Environment Configuration
+# Copy this file to .env.lab0 and configure for your environment
+
+OPENAI_API_KEY=your-openai-api-key-here
+
+# =============================================================================
+# Splunk Observability Cloud Configuration - lab0
+# =============================================================================
+SPLUNK_REALM=lab0
+SPLUNK_ACCESS_TOKEN=your-lab0-access-token-here
+SPLUNK_HEC_TOKEN=your-lab0-hec-token-here
+SPLUNK_HEC_URL=https://bits.splunk.com:8088/services/collector/event
+SPLUNK_COLLECTD_DIR=/usr/local/opt/collectd
+
+# =============================================================================
+# OpenTelemetry Core Configuration
+# =============================================================================
+OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317
+OTEL_EXPORTER_OTLP_PROTOCOL=grpc
+OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE=DELTA
+OTEL_LOGS_EXPORTER=otlp
+OTEL_PYTHON_LOGGING_AUTO_INSTRUMENTATION_ENABLED=true
+
+# =============================================================================
+# Service Configuration
+# =============================================================================
+OTEL_SERVICE_NAME=alpha-release-test
+OTEL_RESOURCE_ATTRIBUTES=deployment.environment=ai-test-val,test.phase=alpha,realm=lab0
+
+# =============================================================================
+# GenAI Instrumentation Configuration
+# =============================================================================
+OTEL_SEMCONV_STABILITY_OPT_IN=gen_ai_latest_experimental
+OTEL_INSTRUMENTATION_GENAI_EMITTERS=span_metric_event,splunk
+OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=true
+OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT_MODE=SPAN_AND_EVENT
+OTEL_INSTRUMENTATION_GENAI_EVALS_RESULTS_AGGREGATION=true
+OTEL_INSTRUMENTATION_GENAI_DEBUG=false
+
+# =============================================================================
+# Evaluation Configuration
+# =============================================================================
+OTEL_INSTRUMENTATION_GENAI_EVALS_EVALUATORS="deepeval(LLMInvocation(bias,toxicity,hallucination,relevance,sentiment))"
+OTEL_INSTRUMENTATION_GENAI_EVALUATION_SAMPLE_RATE=1.0
+OTEL_GENAI_EVAL_DEBUG_SKIPS=false
+OTEL_GENAI_EVAL_DEBUG_EACH=false
+
+# =============================================================================
+# DeepEval Configuration
+# =============================================================================
+DEEPEVAL_FILE_SYSTEM=READ_ONLY
+DEEPEVAL_TELEMETRY_OPT_OUT=YES
+
+# =============================================================================
+# Azure OpenAI Configuration
+# =============================================================================
+AZURE_OPENAI_ENDPOINT=https://your-endpoint.openai.azure.com
+AZURE_OPENAI_API_KEY=your-azure-openai-api-key-here
+AZURE_OPENAI_DEPLOYMENT=gpt-4
+AZURE_OPENAI_API_VERSION=2024-08-01-preview
+
+# =============================================================================
+# LangChain Instrumentation
+# =============================================================================
+OTEL_INSTRUMENTATION_LANGCHAIN_CAPTURE_MESSAGE_CONTENT=true