AI · Full-stack · Product · 2025
Penny.
Local-first finance tracker that keeps the LLM out of the hot path.
Role
Full-stack Engineer · AI Integration
Team
2 engineers — with Nicholas Kaplun
Stack
FastAPI · Python · Gemini API · React · Vite · Pandas
Year
2025
01 / The problem
Why this needed building.
Most finance trackers treat the LLM as the spine — every transaction round-trips through an API. That's slow, fragile when offline, and expensive at scale. I wanted to see how far rule-based classification could go before the model had to step in.
02 / Approach
How I broke it down.
- 01
Rule-based classifier first — merchant strings, amount heuristics, and a learned table over 15 categories. Handles the long tail of recurring merchants cheaply.
- 02
LLM only as fallback for the unclassified residual. Routes to Gemini with a constrained-output prompt so the response is parseable without retries.
- 03
Eval harness with a 428-transaction test set hand-labeled across all 15 categories. Every change to rules or prompt re-runs the eval.
- 04
Local-first state — the API key is the only thing the user gives up; transactions never leave the device unless they hit the LLM fallback.
03 / Outcomes
What it ended up being good at.
Rule layer alone catches the vast majority of transactions; LLM fallback is reserved for genuinely ambiguous cases.
428-transaction eval gates every change — accuracy is observable, not vibed.
Lesson encoded: LLMs are great rescue tools and bad spines. Build the cheap deterministic path first, route to the model only when it's the right tool.