What it does
The models in Vision answer from my own material: the project’s docs, my notes and my PDFs. The knowledge core is what makes that material searchable. An indexer splits each document by its headings, stores a vector and the keywords for every piece, and keeps the index current as documents change. Search runs both rankings and combines them.
None of this can change a note. My notes reach the server in one direction only, private ones never reach it at all, and the models get to the index only through read tools on the MCP servers. How well it finds things is measured in the eval harness.
Why this way
Decision 01
Prove the path on content that is safe to get wrong
The first corpus was the project’s own docs, which hold nothing private. The whole path, from indexer through store, read-only server and gateway to the client, ran on them before my notes were connected. The indexer reads the reviewed copy of the main branch, never a working copy, so only merged text becomes searchable, and it is searchable about a quarter of an hour after the merge.
Rejected: starting with my notes
A bug in the first version would have met my private notes first.
Decision 02
Hybrid search, and one embedding model everywhere
Every search runs a vector ranking and a keyword ranking and fuses the two by reciprocal rank. Documents are split by heading into pieces of 500 to 800 tokens. One embedding model is used everywhere, and every piece records the exact model version that embedded it. The GPU PC embeds first; a CPU instance takes over only if it runs exactly the same version. Otherwise the work waits in a queue, and search falls back to keywords and says so.
Rejected: a different model on the CPU as the fallback
Vectors from two models can’t be compared, so results would quietly get worse.
Decision 03
My notes are read-only, and private notes fail closed
My notes reach the server through a sync tool, in a mode where nothing changed on the server is ever sent back to my devices, and every other service sees them read-only. My private folder is never shared with the server, so the server doesn’t even learn that a private file exists. A note marked private, or one whose header can’t be read, is stored as an ID and a hash only: no title, no text, no links. Agents can write only to their own inbox folder, and only by creating new notes through one narrow tool.
Rejected: ignoring the private folder only on the server
My devices would still announce the names of private files to it.
Decision 04
No model in the indexer
A note’s summary comes from its own header or its first paragraph, not from a model. Links between documents are pulled out by code, with the same patterns my doc checks use.
Rejected: summaries written by the local model
A model call for every changed note, load on the GPU next to the chat’s classifier, and a first index that waits for the model. The note search eval shows whether model summaries are ever worth it.
Decision 05
Read a named document instead of searching around it
When I name a document and want its content, the model searches once to find it, then reads it section by section with a read tool, instead of answering from search snippets.
Rejected: raising the limit from 6 to 8 tool calls per message
It was my planned last resort. It would have made every message with tools more expensive, and the reading rule alone was enough.
How it's tested
Retrieval on three kinds of material
Each corpus has a dev set and a held-out set of my own questions. For the project docs, hit@5 started at 0.967 on dev and 0.875 held-out. As the corpus grew it fell to 0.933 and 0.75; a ranking fix brought dev back to 1.0, while the held-out set stayed at 0.75. For my notes, searching the pieces went from 0.81 to 1.0 on dev and 0.91 to 0.95 held-out with the same fix, and searching whole notes by title and summary from 0.88 to 1.0 and 0.59 to 0.68. The test PDFs scored 1.0 on both.
Fresh within minutes, and free to measure
A merged doc was searchable 14 minutes and 47 seconds after the merge. The retrieval evals use only the local embedding model, so the ranking fix and every rerun cost nothing.
Named documents, before and after the reading rule
After the reading rule, the agent eval found the right source in 23 of 23 named-document cases, and the read tool was used in all 5 new ones. Those runs cost $0.64 and $0.39 for the two halves.
| What | Value | n | Date |
|---|---|---|---|
| Named document read tool used | 1.0 | 5 | 2026-11-12 |
| Notes note search hit at 5 held out | 0.68 (before: 0.59) | — | 2026-11-12 |
What I learned
Case 01
Keyword search lost to stopwords
What happened
As the corpus grew, docs hit@5 dropped from 0.967 to 0.933. Scores per piece showed why: the right documents ranked first to third by vectors, but so far down by keywords that they fell out of the candidates. The keyword query joined every word with OR, including “why”, “the” and “to”, so long pieces won, and it split names that the index keeps whole, so those never matched.
What I changed
I had suspected that documents with many pieces crowd out short ones, but in a simulation a cap per document made things 7 to 37 points worse. The fix uses the index’s own parser and drops English, German and Spanish stopwords.
Result
Dev hit@5 went back to 1.0, and no held-out set got worse.
0.933 → 1.0 Docs hit at 5 dev
Case 02
“ADR 18” was never found
What happened
My decision records carry four-digit numbers in their titles. New dev cases asked for “ADR 18” and “ADR 11”, and the baseline failed both: no search returned the right record, and a guessed ID came back as not found. In an earlier real run, the model had the right document’s ID in hand and still didn’t read it.
What I changed
The reading rule now says: search once, with the number written as the title writes it, read the headings, then read the text.
Result
The agent eval found the right source in 23 of 23 named-document cases, and the read tool was used in all 5 new ones. The eval caught the mismatch before a phone test would have.
Case 03
A busy GPU made search fail
What happened
The GPU PC holds one model at a time. Once, a cold load of the chat’s classifier took over two minutes, and two search embeddings waited behind it until the gateway gave up. My health probe answered quickly during the load, so it saw nothing wrong.
What I changed
A query embedding now tries the GPU for 10 seconds, then the CPU with the same model, then keywords only.
Result
Each attempt is its own step in the run record, so I can see which path served a search.
One limit I kept
A very long document, such as one runbook of about 33,000 characters, can’t be read whole within the 6 tool calls a message gets. I kept the limit at 6: raising it for one document would raise the cost of every message.