Walt

AI Dispatch Cascade

Quality-aware provider cascade -- cheap first, Claude Haiku review, escalate + Annie RAG inject.

AI Dispatch Cascade (v2026-08-02T101544Z-ab2bbdebf5) 2026-08-01 | Quality-aware provider cascade -- cheap first, Claude Haiku review, escalate + Annie RAG inject. 1 - AI Dispatch API - Handles incoming requests and dispatches to providers POST /api/ai/dispatch daemon/src/ai_dispatch_api.py:1437 dispatch_request _get_worker_token() _walt_timeout_for_tier() jobs.db user_prefs table _user_profile ollama → groq → claude http://localhost:11434 http://localhost:{KEVIN_PORT_DAEMON} user request dispatched job provider respo… 2 - Quality Review - Claude Haiku review and escalation quality_review _bus_publish() _role_has_authority() Claude API ANTHROPIC_API_KEY claude-haiku-4-5 FLO rules cache _FLO_RULES_CACHE provider respo… quality score 3 - RAG Context Injection - Annie service integration _inject_rag_context annie:rag_inject:request annie:rag_inject:response Annie Service http://localhost:{KEVIN_PORT_ANNIE} prompt with co… 1. Receive user request v… 2. Dispatch request to pr… 3. Perform quick quality… 4. Check quality score ag… 5. Escalate to next tier… 6. Inject RAG context usi… 7. Return best response w… User request payload {prompt: 'What is the meaning of life?', provider: 'ollama', context: {}, role: 'user', quality_threshold: 6} Provider response payload {provider: 'claude', score: 8, response: 'The meaning of life is subjective.', cost: 0.3, latency: 200} The system uses environment variables for configuration and external service URLs. Legend actor = initiates work · process = code path · store = state on disk · bus = durable queue · dashed = separate process cyan = request flow · pink = state read/write · dashed green = pull / return path · red = refusal

Provenance

This drawing is generated, not drawn. It is rebuilt from the source files below, so when they change the picture changes — a diagram here cannot quietly describe a system that no longer works this way.

Owner
Walt
Slug
ai-dispatch-cascade
Rendered
2026-08-02T101544Z
From commit
ab2bbdebf5
Watches
2 paths
  • daemon/src/ai_dispatch_api.py
  • daemon/src/ai_proxy_router.py