Ask GovAI

A small, dependency-free RAG pipeline over this site's dataset — built to be measured, honest, and cheap, not a demo. A structured query router + BM25 retrieval (key-free) feed grounded LLM synthesis that cites award IDs and refuses to invent. The retrieval below runs entirely in your browser — same algorithm as the repo, no API key, no server.

Try the retrieval — live

Loads the full 670+-contract dataset (~650 KB) on first search, then everything runs locally.

This page demonstrates the retrieval half. The grounded, cited synthesis half runs in the repo CLI with a free Gemini key or a Claude key — run it yourself.

How it works

question ─▶ query router ─▶ structured path
               │              (superlative / NAICS →
               ▼               filter + sort by amount)
        BM25 retrieval ─▶ top-k contracts
        (key-free, offline)     │
                       grounded LLM synthesis ─▶ cited answer
                       (cite award_ids, refuse
                        if unsupported)

The cost decision: don't pay a model to find documents a ranking function can find. BM25 handles topical lookup for free; the router handles what lexical ranking cannot do by construction (superlatives, explicit NAICS codes) by answering structurally against the data. The LLM does only the one thing it's uniquely good at — synthesizing a grounded answer from the retrieved records. Synthesis is provider-agnostic: free Gemini or paid Claude.

The eval — measure, diagnose, fix

Metric	Result
Retrieval recall@8 (routed)	92% (12/13)
Retrieval recall@8 (BM25 baseline)	69% (9/13)
Answer accuracy / latency / cost	printed by `ask:eval` with a key

The baseline measured 69%. The misses were diagnosed, not hidden: superlative queries ("largest contract…") and explicit NAICS codes are structurally unanswerable by term overlap — so a thin router now answers those against the data (a general mechanism, gold set frozen before it was built), taking recall to 92%. The one remaining miss is a semantic-disambiguation limit of lexical retrieval — that's the honest residual, and the roadmap (dense embeddings) targets exactly it.

Full eval write-up →

Run it yourself

git clone https://github.com/Nandanhegde1/govai-contracts && cd govai-contracts && npm install

npm run ask -- "which vendors won the biggest DoD AI contracts?"   # retrieval — no key needed
GEMINI_API_KEY=... npm run ask -- "who supports the DoD JAIC?"     # grounded, cited answer (free key)
npm run ask:eval                                                    # reproduce the eval