# RAG in Goa

Multilingual voice Q&A with hybrid retrieval inside a hard 200 ms deadline, and a reranker we measured and cut.

- Dates: Aug 2026 · Hacker House Goa 2026 task
- Team: team of 3 with Merlyn Michelle, Shoban Ravichandran
- My part: Retrieval pipeline rework, Sarvam Bulbul TTS, Colab GPU index, latency and architecture docs, deploy, mobile UI
- Verified: 76.85 ms retrieval P95, live; 215,608 chunks indexed; 200 ms hard retrieval deadline; ~160 ms reranker cost, so it shipped off
- Result: no result recorded
- Case study: https://solaimuthu.vercel.app/work/rag-in-goa

### The problem

Voice assistants tend to fail in one of two ways: they answer fluently and wrongly, or they answer slowly. Indic-language users usually get the worse version of both. The brief asked for grounded answers and strict latency reporting on retrieval.

### How it works

Speech in. Sarvam's Saaras speech-to-text, in translate mode, turns Hindi, Bengali, Tamil or Marathi speech into an English query. Input gate. Unsafe, injected or garbled transcripts are refused before retrieval. Hybrid retrieval against a memory-mapped index of 215,608 chunks: ONNX int8 bge-small embeddings into an HNSW index, BM25 alongside it, reciprocal rank fusion, then the top passages. A 200 ms wall-clock deadline skips stages and logs a degradation rather than miss it. Domain gate. If retrieval scores are too low, the system says it doesn't know. Grounded answer. The model returns schema-enforced JSON citing passage numbers; an output gate rejects invented citations. Speech out. The answer streams over SSE and is spoken back in the user's language through Sarvam's Bulbul.

### The numbers, honestly

Retrieval on the live deployment: P50 47.5 ms, P95 76.85 ms, P99 88.8 ms, measured over the benchmark queries after warm-up. The claim covers retrieval only. A full live request takes about ~1.1 s at P50 because of the model (about ~617 ms) and the network; that is shown separately rather than hidden.

### Decisions

We cut the reranker. A MiniLM cross-encoder improved ranking but cost about ~160 ms of the budget. It ships switched off. A refusal beats a fluent miss. Three gates refuse instead of guessing.

### My part

Team of three (Team McQuade). I reworked the retrieval pipeline, built the Colab GPU index notebook and the Bulbul voice output, wrote the latency and architecture docs, set up the deploy, and fixed the mobile UI and the evals. Merlyn Michelle built the FastAPI service, the guardrail gates and the SSE path. Shoban Ravichandran built the voice UI and ran the latency benchmark.
