Skip to content

ANIMAL-CONFIG-FLAKINESS-ADVICE.md

Latest commit

 

History

History
196 lines (153 loc) · 6.66 KB

ANIMAL-CONFIG-FLAKINESS-ADVICE.md

File metadata and controls

196 lines (153 loc) · 6.66 KB
Raw

Animal Config Validation Flakiness - Root Cause Analysis & Solutions

Executive Summary

The animal config validation failures stem from a fundamental architectural mismatch between the OpenAPI code generation templates and the hexagonal architecture implementation pattern. Each regeneration breaks the controller-to-implementation connection.

Root Causes Identified

1. Controller Template vs Handler Architecture Mismatch

Problem: The generated controllers look for specific handler functions (handle_animal_config_get) but the hexagonal architecture uses a generic routing function (handle_()).

Evidence:

  • Controller template line 38: impl_function_name = "handle_{{operationId}}"
  • Handlers.py line 18: Only has generic handle_() function
  • Error: "cannot import name 'handlers'" when handlers.py missing or misnamed functions

2. OpenAPI Regeneration Overwrites Controllers

Problem: Every make generate-api completely overwrites controllers, losing any manual fixes or connections to implementation.

Impact:

  • Business logic connections broken after each regeneration
  • Import errors appear immediately after OpenAPI spec changes
  • Manual fixes are temporary and lost on next generation

3. Frontend Tabbed Interface Validation Failure

Problem: Form validation expects all elements in DOM simultaneously, but tabs only render active tab elements.

Evidence:

  • Validation requires 11 form element IDs across multiple tabs
  • Error: "Element with ID 'max-response-length-input' not found" when on Basic Info tab
  • Save button completely non-functional regardless of active tab

Permanent Solutions

Solution 1: Fix Controller Template (RECOMMENDED)

Modify /backend/api/templates/python-flask/controller.mustache line 47-48:

# Pattern 2: Generic handler routing
from {{package}}.impl import handlers
# Use the generic handle_ function that routes based on caller
impl_function = handlers.handle_

Benefits:

  • Permanent fix - survives all regenerations
  • Aligns with hexagonal architecture pattern
  • No manual intervention needed after regeneration

Solution 2: Add Handler Function Aliases

Enhance handlers.py to include specific function aliases:

# Add after line 68 in handlers.py
# Create aliases for controller compatibility
handle_animal_config_get = handle_
handle_animal_config_patch = handle_
handle_animal_list_get = handle_
# ... repeat for all operations

Benefits:

  • Works with existing template
  • Clear function mapping
  • But requires maintenance when adding new endpoints

Solution 3: Frontend React Controlled Components

Replace DOM-dependent validation with React state management:

// AnimalConfig.tsx - Use controlled components
const [formData, setFormData] = useState({
  name: '',
  species: '',
  personality: '',
  maxResponseLength: 200,
  // ... all fields
});

// Validation reads from state, not DOM
const validateForm = () => {
  return formData; // Always available regardless of active tab
};

Benefits:

  • Form data always available regardless of tab
  • Better React patterns
  • Enables progressive validation

Quick Fixes (Temporary)

When Validation Fails After Regeneration:

  1. Check handlers.py exists: ls impl/handlers.py
  2. Verify handle_ function: grep "def handle_" impl/handlers.py
  3. Restart API server: make stop-api && make run-api

When Frontend Save Fails:

  1. Switch to all tabs once: Load data into browser memory
  2. Use browser dev tools: Manually set form values via console
  3. Temporary CSS fix: Show all tabs with display: block !important

Implementation Priority

Immediate (Today):

  1. ✅ Apply Controller Template Fix (Solution 1)
  2. ✅ Test with fresh regeneration
  3. ✅ Document in CLAUDE.md

Short-term (This Week):

  1. Implement React controlled components
  2. Add comprehensive error messages
  3. Create regeneration validation script

Long-term (This Sprint):

  1. Fully decouple generated code from implementation
  2. Add integration tests for controller-handler connection
  3. Consider moving to OpenAPI 3.1 with better code generation

Validation Checklist

After any OpenAPI regeneration, verify:

  • handlers.py exists in impl/ directory
  • handle_() function is present and has routing logic
  • Controllers import handlers successfully
  • API endpoints return data (not 501 Not Implemented)
  • Frontend can load animal list
  • Configuration modal opens
  • Form validation completes (even if save fails)

Prevention Strategies

1. Pre-Generation Backup

#!/bin/bash
# backup-impl.sh
cp -r impl/ impl.backup.$(date +%Y%m%d_%H%M%S)/
make generate-api
# Verify impl/ still intact

2. Post-Generation Validation

#!/bin/bash
# validate-generation.sh
python -c "from openapi_server.impl import handlers; print('✅ Handlers OK')"
curl -X GET http://localhost:8080/animal_list || echo "❌ API Failed"

3. Git Pre-Commit Hook

#!/bin/bash
# .git/hooks/pre-commit
if git diff --cached --name-only | grep -q "openapi_spec.yaml"; then
  echo "⚠️ OpenAPI spec changed - remember to validate handlers after regeneration"
fi

Common Error Patterns

Error: "cannot import name 'handlers'"

Cause: handlers.py missing or moved Fix: Restore from backup or recreate with handle_() function

Error: "not_implemented" responses

Cause: Handler routing broken Fix: Check handler_map in handlers.py includes all operations

Error: "Element with ID not found"

Cause: Tabbed interface validation Fix: Implement React controlled components

Monitoring & Alerts

Add these checks to your CI/CD:

  1. Import Test: python -c "from openapi_server.impl import handlers"
  2. API Health: curl http://localhost:8080/system_health
  3. Handler Count: Verify handler_map has all expected operations
  4. Frontend Build: Ensure no TypeScript errors in form components

Team Communication

When making OpenAPI changes:

  1. Before: Announce in Slack/Teams
  2. During: Run validation script
  3. After: Confirm handlers intact
  4. Document: Update this file with new patterns

Conclusion

The flakiness is 100% preventable with proper template configuration. The root cause is a mismatch between code generation assumptions and hexagonal architecture patterns. Apply Solution 1 (template fix) for immediate permanent resolution.

References

  • OpenAPI Generator Templates: /backend/api/templates/python-flask/
  • Hexagonal Architecture Handlers: /backend/api/src/main/python/openapi_server/impl/handlers.py
  • Frontend Validation: /frontend/src/hooks/useSecureFormHandling.ts
  • Original Investigation: VALIDATE-ANIMAL-CONFIG-EDIT.md