Athena
Planned multi-agent BI system that will turn natural language into verified analytical insights. Not built yet (October 2026)
Athena is a design on my roadmap. This page sets out the problem it targets and how I plan to build it. It reports no results, because there are none yet.
The Problem
Business intelligence is broken for most organizations. Analysts spend much of their time finding data rather than analyzing it. Self-serve BI tools promised democratization but delivered complexity — dashboards that only their creators understand, SQL queries that take days to write, and insights that arrive too late to act on.
The goal is a system where any stakeholder can ask a business question in plain English and get a verified, sourced analytical answer — not a hallucinated guess from a chatbot.
The Architecture
Athena is a planned multi-agent system on LangGraph with 4 coordinated AI agents, each with a distinct role:
Will classify incoming queries by intent, complexity, and required data sources. It will route simple lookups to a fast path and complex analytical questions to the full pipeline, so routine questions are answered faster.
Will run hybrid search across structured and unstructured enterprise data using Qdrant (vector) + BM25 (keyword). The dual approach is meant to give high recall for exact-match queries (product IDs, dates) while maintaining semantic understanding for fuzzy natural language questions.
Will take retrieved context and generate analytical insights using GPT-4o. This is not raw summarization — the agent applies analytical frameworks: trend detection, anomaly flagging, comparative analysis. Every claim is grounded in specific data points with citations.
Will review the Analyst's output against source data: check numerical accuracy, flag unsupported claims, and add confidence scores. This agent is what should make Athena's outputs trustworthy — it is designed to catch made-up answers before users see them.
Technical Decisions
Explicit state management. LangGraph gives me a directed graph with typed state, conditional edges, and checkpoint persistence. When Agent 3 fails, I need deterministic retry logic — not an autonomous agent deciding what to do next. Production systems need predictable failure modes.
Vector search alone misses exact matches. When a user asks "Q3 2024 revenue for product SKU-4421," pure semantic search might return similar products. BM25 catches the exact SKU. The hybrid retriever uses reciprocal rank fusion to combine both result sets.
Speed to a working interface. Streamlit gives a functional, polished UI in weeks, not months, while I focus engineering time on the agent pipeline — which is where the actual complexity lives.
Observability
Every agent interaction will be traced end-to-end through LangSmith, showing:
- Token usage per agent per query
- Retrieval relevance scores
- Validator rejection rates and reasons
- Latency breakdown by pipeline stage
This is not logging — it is structured observability that feeds back into system optimization. When retrieval quality drops for a specific data domain, it should show within hours, not weeks.
What I Will Measure
Once Athena is built, these are the numbers it has to earn before this page reports any:
- Share of retrieved passages that are relevant to the question
- Share of unsupported claims the Validator catches
- Median time from question to checked answer
- How often the Router's fast path answers correctly