What it does
My chat app only ever showed me answers. The console shows what happened on the way: which model a message went to, every model call and tool call with what it saw and returned, what it cost, and how long each step took. It has chat sessions with streamed answers and their sources, a live trace, an explorer for past runs, and flags that turn runs into a review queue.
The first version is built. Notifications and switching off the old chat channel are next. This page names the access layers and why each exists; how they work stays private.
Why this way
Decision 01
Private network only, and several layers behind it
The console has no open port; it is reachable only inside my private network. It takes my identity from the network itself, in a way that a local process can’t fake. The console’s server, not the browser, signs every request it passes on to agent core, which trusts only signed senders, and it uses its own credential, separate from the chat’s. The coding agent that builds Vision can’t use the console’s data or its message endpoint at all. The threat model is written down: access rests on my own devices being locked and current, not on the app.
Rejected: trusting the identity of any local connection
Any process on the machine, the coding agent included, could claim to be me.
Decision 02
Full traces, kept for 30 days
Every step of a chat run stores what went in and what came out: the messages a model saw, a tool’s arguments, and the result exactly as the model got it. That is what makes the trace worth reading. Traces and chat history are deleted 30 days after the run, unless I flag it; a flagged run stays until I mark it handled and then expires like any other. A daily job does the deleting and records its own run, and the console’s header turns red if anything is older than 31 days. Logs carry IDs, names and counts only, never content, so they can’t outlive the expiry.
Rejected: a separate database for traces
It couldn’t be joined with the run records.
Decision 03
A budget for every session, counted by agent core
Each session has its own budget, from $0.10 to $2, raised in $1 steps by a button only I can press. A daily cap of $2 covers the console and the chat together, and at most 3 runs work at once. Agent core checks the budget before every model call, so lowering a budget stops a running run at its next call. The gateway’s global cap stays behind all of it.
Rejected: one gateway key per session
Far too much key handling for something agent core can count from its own records.
Decision 04
Untrusted content is shown as text
Tool results carry text from my documents, and later from the web. The console shows every one of them as plain text. Links in answers are inert text that shows the whole address, and there is no list of allowed sites, because an allowed site can carry data out in its query just as well.
Rejected: rendering formatted text with images and live links
Opening a trace would load whatever a document had told the model to write.
Decision 05
Build my own instead of adopting a tracing tool
A small backend that reads the database and passes on agent core’s events, and a web front end on top. The trace view is my own: indented rows on a shared time axis, so a tool loop reads like a timeline.
Rejected: an existing tracing tool
Langfuse is too heavy for the machine it would run on, and Open WebUI can’t see agent core’s inner steps, which is the part I wanted to see.
Rejected: a graph view of the trace in the first version
A graph can come later as its own item; the timeline answers the questions I actually ask.
How it's tested
Tested live from my phone
Two sessions ran in parallel, one stopped at a budget of $0.10, the +$1 button raised it, and “stop all” and resume worked, all as designed.
Streaming changed nothing it shouldn’t
After answers started streaming, the cost of all 158 paid steps matched the gateway’s spend log to within a thousandth of a cent. The regression evals with streaming on kept the same quality as before, at $0.015 and $0.012 per case for their two halves.
Nothing leaks into live events, and the trace stays smooth
A marker placed in a step’s input and output never reaches a live event; it comes back only when that step is opened on purpose. The trace view holds 58 to 60 frames per second on a 60-step run, and its test requires at least 55.
What I learned
Case 01
The first phone test after the trace work failed
What happened
My first message after trace storage went live failed. The GPU PC had cold-loaded the classifier, two search lookups waited behind it, and two timeouts on the way were both 60 seconds. The timeout landed in a cleanup step after the strong model had already answered, and it failed the whole run.
What I changed
Agent core now waits longer than the gateway, a failed cleanup after an answer fails only that step, and trace writes are shielded from cancellation.
Result
The trace work wasn’t the cause, but the search for it found and closed a real bug of its own: a cancelled step could lose its trace record.
Case 02
Tests that didn’t test the change
What happened
The first streaming build passed all its browser tests, but they ran on the demo build with recorded data, which never touches the live path. My review caught it.
What I changed
A second test mode now runs the real build against scripted live events.
Result
Three deliberate breaks of the streaming code each make it fail. A test proves something only if breaking the thing breaks the test.
A long answer still arrived all at once
With streaming deployed, a long answer still appeared in one piece. I measured layer by layer: agent core and the console both passed text on as it came. The gateway’s own record showed 870 tokens arriving 1.18 seconds after the first, far faster than a model writes. Once a tool result is in the conversation, the stream reaches agent core in one piece; without one, it streams normally. I accepted it as a known limitation and wrote it down.