AI · Full-stack · Product · 2026
DocLens.
Hybrid knowledge assistant that routes questions across uploaded documents and SQL-backed data.
Role
Solo Engineer
Team
Solo project
Stack
Python · FastAPI · React · FAISS · BM25 · sqlglot · OpenAI
Year
2026
01 / The problem
Why this needed building.
Most document assistants stop at RAG, while business questions often cross into structured data. DocLens needed one chat surface that could answer from uploaded files, a SQL database, or both — without turning user uploads into permanent product memory.
02 / Approach
How I broke it down.
- 01
Route each question to document RAG, Text-to-SQL, or a hybrid answer path so users do not need to choose tools before asking.
- 02
Index uploaded PDF, Markdown, and HTML files with OpenAI embeddings in FAISS, fuse dense retrieval with BM25, and rerank the final context with a local cross-encoder.
- 03
Generate read-only SQL from schema context and retrieved few-shot examples, then validate every query with sqlglot before executing against a synthetic demand-planning database.
- 04
Keep uploads and retrieval indexes scoped to a browser session with file-count, file-size, storage, query-length, and TTL limits so the product does not accumulate hidden memory.
03 / System
The pipeline, stage by stage.
Scroll to walk through each stage. Each is small on its own; what matters is the composition.
STAGE / 01
Route.
A lightweight classifier sends each prompt to documents, SQL, or both. The interface stays one chat box.
STAGE / 02
Retrieve.
Uploaded files are chunked, embedded, indexed in FAISS, fused with BM25, and reranked before citation.
STAGE / 03
Query.
Structured questions generate SQL from schema docs, glossary terms, and dynamically retrieved examples.
STAGE / 04
Guard.
sqlglot enforces SELECT-only queries before read-only execution with row caps and statement timeout.
STAGE / 05
Expire.
Session uploads and indexes are bounded and deleted when the session ends or the TTL expires.
04 / Outcomes
What it ended up being good at.
Live at doclens.eddyislearning.ai with a landing page, a streaming chat app, cited document answers, and generated SQL tables.
Text-to-SQL path ships with SELECT-only AST validation, row caps, statement timeout, read-only SQLite access, and a reproducible multi-model evaluation harness.
Session-scoped cache cleanup prevents old test files from becoming product knowledge while still preserving a useful short-lived RAG workflow.