~/evals
  .----------------------------------.
  |  every answer the castaway gives |
  |  is graded before it ships       |
  '----------------------------------'
 ~^~~^~~^~~^~~^~~^~~^~~^~~^~~^~~^~~^~

Castaway evals

The chatbot on this site is tested against a fixed set of questions: facts it must get right, things it must admit it doesn't know, things it must decline, and page actions it must take. Each answer runs through the real agent (retrieval and tools included), then a separate Claude model grades it against a rubric and the bot's own notes.