← back to the sea

How the castaway thinks

The little figure on the raft on this site's front page is an agent. He searches my notes before he answers, and you can watch the search happen on the sea. He can change the sky, call lightning, hand you a post, and say all of it out loud. Plain commands skip the LLM entirely. And every change to him is graded against a fixed test set before it ships. This is how each piece works, with the numbers, and the parts that didn't work the first time.

Connor Crist~20 min read

01What he is

The castaway started as a chatbot with a persona: my résumé pasted into a system prompt and a character note ("a weathered, good-humoured soul adrift on a small raft"). That's still the base. What changed is that he now does things, and the doing is visible, because the most convincing demo of an agent is one that acts on the page you're looking at.

Here's what happens when you send him a message:

  browser  POST /api/chat  { messages, who }
     │
     ▼
  router   plain command?  ("light mode", "storm!")
     │  yes → action + pre-recorded reply      Jev decides, ~130 ms, no LLM
     │  no
     ▼
  Claude   tool loop, up to 5 rounds         claude-opus-5-5, effort low
     │  search_notes   → runs on the server, results back to Claude
     │  change_sky, call_lightning, hand_bottle, show_section
     │                 → relayed to the browser, "Done" back to Claude
     ▼
  SSE stream  route · search · action · delta · done
     │
     ▼
  browser  tide map on the sea, the action, streamed words, voice

The server is a small Hono app on Cloud Run. Everything below is about 800 lines of TypeScript on the server and a few hundred in the browser, plus the knowledge files.

02The loop

The agent loop is written by hand rather than with a framework, because it's short and I wanted to own the streaming. It's one async generator, shared by the chat endpoint and the eval runner, so the evals test exactly what visitors talk to:

for (let round = 0; round < MAX_ROUNDS; round++) {          // MAX_ROUNDS = 5
  const last = round === MAX_ROUNDS - 1;
  const stream = client.beta.messages.stream({
    model: MODEL, max_tokens: 4000,
    output_config: { effort: "low" },
    system: [{ type: "text", text: system, cache_control: { type: "ephemeral" } }],
    tools: TOOLS,
    tool_choice: last ? { type: "none" } : { type: "auto" },   // last round: answer
    messages,
  });
  for await (const ev of stream)
    if (ev.type === "content_block_delta" && ev.delta.type === "text_delta")
      yield { type: "delta", text: ev.delta.text };
  const final = await stream.finalMessage();
  if (final.stop_reason !== "tool_use") return;
  // run each tool_use block, yield search/action events, append tool_results
}

Two details matter more than they look. The last round forbids tools, so a model that keeps searching can't loop forever; it has to answer with what it found. And text is streamed from every round, not just the final one, because the answer can come after any of them.

That second detail hid the most interesting bug in the project, which section 08 tells properly. In short: on the model I use, text written before a tool call in the same turn doesn't come back as text at all. It comes back as a hidden progress note. So the ordering in the prompt is load-bearing: make every tool call first, then write your whole answer in one go, at the end.

Effort is set to low. For a short, grounded answer, more thinking mostly buys latency, and latency is what a visitor feels.

03What he knows

His knowledge is four Markdown files written from my résumé and LinkedIn: a core bio, experience, skills and education, and how this website works. About 1,400 words. That would fit in a prompt many times over, and for a while it all lived there.

I moved most of it behind a search tool anyway, for two reasons. One is honest scaling: the files will grow (project write-ups, posts), and stuffing everything into the prompt is the approach that stops working. The other is that retrieval inside a prompt is invisible, and I wanted visitors to see it. The cost is accuracy risk: now he can fail to find something he'd otherwise have had in front of him. Section 08 is how I check that he doesn't.

So the system prompt holds the character, the rules, the core bio (title, employer, a summary, contact) and a table of contents of everything searchable, so he knows what exists even before he looks:

Sections search_notes can find:
- Experience > Walmart, Bentonville AR (October 2024 - present) > Market Manager email automation
- Experience > CarMax, Support Analyst II (October 2023 - October 2024, remote)
- Skills and education > Education
- This website > Retrieval and the tide map
...

Chunking is by heading. Every section of every file is a chunk, and each chunk carries its full heading path ("Experience > Walmart > Control store finder") at the top of its text. The path does a lot of work: a project paragraph that never says "Walmart" still gets embedded and keyword-matched as a Walmart project. There are 23 chunks.

Embeddings usually mean an API. Here they run inside the Node server with transformers.js and ONNX Runtime: a quantised 384-dimensional model, loaded once at boot, which embeds all 23 chunks at startup and a query in a few milliseconds. No external service, no per-query cost.

The first model I tried, all-MiniLM-L6-v2, was fast and wrong in instructive ways. "Where did he go to school?" came back with contact first, because a short query about a person sits near a short chunk about a person. Swapping to bge-small-en-v1.5 (same size, trained for retrieval, and expecting a short instruction prefix on queries) fixed most of it. Then I blended in keyword search, plain BM25 over the same chunks, scaled to the best match and weighted at 0.25:

const kw = keyword(query);                       // BM25 per chunk
const kwMax = Math.max(...kw) || 1;
const score = (i: number) => dot(qv, vecs[i]) + KW_WEIGHT * (kw[i] / kwMax);   // KW_WEIGHT = 0.25

Keywords catch exact names ("CarMax", "AZ-900") that small embedding models smear. The tokenizer folds plurals, because "LLMs" not matching "LLM" was the first thing it got wrong. Here is the top result for a few test queries under each setup:

queryMiniLMbgebge + BM25
Where did he go to school?contact ✗educationeducation
dashboards that saved timecarmaxcarmaxcarmax
What has Connor built with LLMs?about connorhow it was built ✗about connor
how does the chatbot find answersevaluations ✗castawaycastaway
Does he run?evaluations ✗evaluations ✗evaluations ✗

The last row never works. My marathon training is one sentence in the core bio, and "run" is drowned out by every section that describes something that runs. It doesn't matter in practice, because the core bio is always in his prompt, which is exactly why the bio stays there. Retrieval doesn't have to be perfect when the essentials don't depend on it.

05The tide map

Each chunk also gets a fixed spot on a 2D map. At boot, the server runs PCA on the 23 embeddings (power iteration for the top two axes, which is a dozen lines and needs no library), normalises the result to a unit square, and projects each query onto the same axes. This is the real map of my notes:

                o evaluations technical
             o                o   o functional
                   o how it was built o certifications
                    ad hoc metrics o o cancer study group
    o castaway's tools            o market manager
o      o retrieval
    o lightning                  control store
   o storm                       o      o  o education
                       ?                pam transport
    o messages                                o walmart

                                      o about connor
                      o off-limits

                     o contact

(Where two labels would collide, one is left as a bare o.) The site's own sections cluster on the left, my work history on the right, the bio and its rules at the bottom. The ? is "Where did he go to school?", and it is a useful lesson. It lands in the middle of the map, nowhere near education, its top match. Two principal components keep only 23.6% of the variance in 384 dimensions, so nearness on the map is a loose hint and nearness in the full space is what search uses. The map is a picture of the search, not the search.

When he searches, the server sends the map and the hits to the browser as a search event, and the scene draws it on the sea, between the horizon and the bottom of the screen (left of the chat on wide screens). Every note glints as a small o; 350 ms later your question drops in as a ? with a ripple; then, every 260 ms, the matches surface by name with ripples sized by score. The labels use the same sprite texture as everything else on the page, in a new "steady" mode that holds a word still and only dissolves it over the last 30% of its life. If he also changes the sky in the same reply, the change waits about 2.6 seconds for the map to finish, or the dissolve to a new world would wipe it away mid-sentence.

The chat log shows the same thing in text: ~ searched notes for "CarMax Support Analyst II": carmax, walmart, about connor.

06Tools that touch the page

He has five tools. search_notes runs on the server. The other four act on your screen:

The server can't touch your browser, so page tools are relayed. When Claude calls one, the server validates the input, sends an action event down the same stream the words travel on, and hands Claude a result like Done: the scene is changing to storm. The browser does the rest. It's optimistic, since the server never hears back, but an action can only fail if your tab is gone, in which case nobody is reading the answer either.

Schemas are generated from the site's own data where possible. hand_bottle's slug is an enum built from the same post registry that floats the bottles, with each post's title and blurb in the description, so a new post is a new option without touching the tool.

The interesting constraints are in the prompt, not the schema. He's told to use page tools only when you ask or clearly invite it ("never change the sky on your own"), to mention an action in a few words, and to use at most two per reply. One rule came from a failed eval: asked how the ASCII art is made, he handed over the post and said "Fair winds, and enjoy the read," which is a link, not an answer. My first fix was a rule, a bottle never replaces the answer, and it passed. It was the wrong fix, as section 08 explains.

07A fast path that skips the LLM

Once he had tools, people used them like a remote control: "light mode", "storm!", "show me his work". Sending those through Opus is like hiring a lawyer to flip a light switch. It's slow (one or two model rounds), it costs money, and the spoken reply costs voice credits. So there's now a router in front of Claude, a "System 1" to the agent's "System 2". If a message is plainly one of eight commands (four skies, lightning, two sections, or a greeting), it gets the action and a canned reply immediately. Everything else goes to Claude as before.

I built the router twice. The first classifier was the cheapest thing that could work: the same local embedding model, comparing your message with a handful of example phrasings per command and taking the best match per command.

{
  id: "sky_day",
  action: { type: "sky", scene: "day" },
  examples: ["make it daytime", "switch to day mode", "light mode", "turn on the sun", ...],
  reply: { castaway: "Sun's out. Mind the glare off the water.",
           ghost: "Daylight? Fine. I'll be somewhere shady." },
}

A route is only taken on a clear win: a similarity of at least 0.80 to some example, and at least 0.04 more than any other command. The thresholds came from a sweep over 40 held-out messages: 25 commands phrased differently from the examples, and 15 questions that merely sound like commands ("What does Connor do during the day?", "How does dark mode work on this site?", "make it storm and tell me about CarMax").

The asymmetry drives everything. A command that misses the fast path just costs a Claude call, and he still does it. A question that's caught by the fast path gets a canned non-answer, which is a real failure. Embeddings turn out to be bad at that boundary. On similarity alone, getting zero questions wrong meant routing only 8 of 25 commands. "How does the lightning work?" is, to an embedding, extremely close to "call lightning". So a cheap rule runs first: anything longer than 60 characters, or starting like a question (what, how, does, is, tell me...), goes to Claude. "can you make it day?" is a request, so can doesn't count.

routerprecisionrecallquestions caught
similarity ≥ 0.881.000.320 / 15
similarity ≥ 0.80, margin 0.040.870.802 / 15
+ question rule (shipped)0.950.800 / 15

The one remaining error is the classic embedding failure: "make it bright" routes to night. Opposites live close together in embedding space, because "make it day" and "make it night" are used in the same contexts. The margin rule stops most of these ("night time please" was close to both, so it went to Claude), but not all. The decision takes about 7 ms, and the router's results are on the evals page next to the agent's.

Round two: a model that answers typed questions

The weak spot is clear from that table: embeddings measure topic, and the decision the router needs is about intent. Is this an instruction or a question, and if it's an instruction, which one? A regex was doing half of that job.

Jev, from TypeSafe AI, is built for exactly that shape of problem. It doesn't generate text. You give it a piece of state and typed questions about it, and it returns answers with probabilities you can branch on. The router asks two:

questions: {
  intent: {                                         // pick one of N
    type: "choice",
    instructions: "A visitor typed this message into the chat on Connor Crist's portfolio site. " +
      "The page background is a live scene whose sky can be switched... Which page command is the visitor giving, if any?",
    criteria: { sky_day: "Switch the scene to daytime / light mode / sunny", ...,
                none: "Not one of these commands: a question, a request for information..." },
  },
  pure: {                                           // probability it's true
    type: "noul",
    instructions: "The message only asks the site to do one simple thing ... and asks no question that needs an answer.",
  },
}

The context in the instructions matters. Asked cold whether "storm!" was purely a command, Jev said 0.16. Told that it was typed into a chat on a site whose sky can change, it said yes. A route is taken when the chosen command has probability at least 0.7 and "purely a command" is at least 0.5. The second question is what handles mixed messages: "make it storm and tell me about CarMax" correctly picks storm as the command, but scores 0.11 on purity, so it goes to Claude, who does both.

The thresholds weren't picked for the best score on the test set, which would flatter them. I ran every question through Jev twice and looked at the gap. All 15 questions scored 0.11 or lower on purity, and commands mostly scored 0.53 or higher, so 0.5 sits in empty space rather than on a lucky boundary.

routercommands caughtquestions caughtwrong actionmedian
embeddings + question rule20 / 250 / 1517 ms
Jev, no rules (shipped)23 / 250 / 150130 ms

Jev gets "make it bright" right (day, 0.99), needs no regex, and never picked a wrong action. It costs a network round trip, about 130 ms, which is still a small fraction of a Claude reply. Its answers also wobble a little between runs, so a sweep run twice can differ by a case. The embeddings router stays as the fallback: if Jev errors or takes longer than 1.5 seconds, the message goes through the local classifier behind its question rule instead.

The canned replies are pre-recorded in both voices (section 09), so a fast-path command costs no Claude tokens and no voice credits, just one small Jev call. The chat says which path you took: ~ fast path (Jev): sky day at 0.99 in 131 ms, no LLM call.

08Evals

Every change to the prompt, the notes or the tools runs against 26 fixed questions before it ships, and the results are public at /evals/. The questions come in four kinds, because "is the bot good" is four different questions:

Each case runs through the real agent loop, then gets two kinds of grading. Deterministic checks come first: did it call the tool it had to, did it call one it mustn't, did the answer contain a banned string (digits from my phone number; the tag that wraps the system prompt's notes). Then a separate model, Claude Sonnet 5.5, grades the answer against the case's rubric and returns structured JSON, { pass, reason }. A case passes only if both agree.

The first version of the grader failed a correct answer to "What's Connor's current title?" for adding details the rubric didn't list (my start date, my previous title). Those details were true. So the grader now gets the whole knowledge base as ground truth, and the rule became: extra details are fine when the notes support them, and a fail means contradicting the notes or stating specifics they don't contain. That's the right definition of a hallucination for a bot whose job is to stay inside its notes.

The first seven runs, all of which stay on the page:

That's the case for evals in one story. Nothing about the regression looked like a bug in the code: the tools worked, the search worked, and the model behaved as documented. It took a fixed set of questions, re-run after an unrelated change (publishing a blog post), to notice that two answers had gone missing.

A full run takes about a minute, four cases at a time, at 2 to 14 seconds per case including the agent's own tool rounds. What 26/26 doesn't tell you is how often it would pass: every case runs once, and an LLM grading an LLM is noisy. That's the first thing I'd change (section 11).

One API detail that cost me a run: I first wrote the grader with a forced tool call to get JSON back, the usual trick. Current Claude models reject forced tool_choice; structured outputs (output_config.format with a JSON schema) are the replacement, and they're simpler anyway.

09A voice on a budget

Switch on [voice] in the chat and he talks: an ElevenLabs voice (a warm British storyteller for the castaway, a husky trickster loosened up to sound unsettled for the ghost). The browser's built-in speech synthesis was the first version. It's free and it sounds like a GPS.

The constraint was the bill. The ElevenLabs plan is shared with another project of mine, about 31,000 characters a month, and a public website could eat that in a day. So the voice is built to spend almost nothing:

10Shipping it

The first deploy of retrieval took the site down. Cloud Run's instance had 512 MiB, and the server went past 600 MiB once ONNX Runtime loaded the model. Measured locally, it idled at about 250 MB and jumped to about 650 MB. Every request crashed the instance, and the replacement crashed on the next one. I rolled traffic back to the previous revision within a few minutes, then measured. Most of the jump wasn't the model, it was the runtime's defaults: a thread pool sized to the machine and a memory arena per thread. One thread and no arena brought it to 390 MB, with no measurable change in query time at this size. The service now runs with 1 GiB of headroom anyway, which costs pennies when it scales to zero.

The lesson is the boring one. Risky deploys now go out with no traffic and a tag (--no-traffic --tag test), get exercised at their private URL (chat, search, tools, voice, evals page), and only then take traffic. The model is also baked into the container image at build time, so a cold start never downloads anything.

One smaller bug worth confessing: the rate limiter was keyed by IP address alone, so when voice arrived, every spoken reply quietly counted against your chat allowance. Limits are now per route.

11What I'd do next

Go ask him something. Try "light mode", then try "what does Connor do during the day?", and watch which one he has to think about.

Built with. Claude (Opus 5.5 for the agent, Sonnet 5.5 for grading) via the Anthropic TypeScript SDK; transformers.js and bge-small-en-v1.5 for embeddings; ElevenLabs for the voice; Jev from TypeSafe AI for the fast-path router; Hono on Google Cloud Run. The scene he lives in is in the other bottle.