Skip to main content
Ryan Orban

Ryan Orban

RSS
3 essays

Writing

Essays on testing and measuring systems that don't behave the same way twice.

  1. What agent trace divergence actually tells you

    I spent three weeks trying to extract training signal from agent trace divergence. Most of it didn't survive held-out data. What did is narrower than I expected.

  2. What Stochastic Variation Reveals About AI Agents

    Same agent, same task, different outcomes. Here's what the variation tells you about why agents fail — and what to do about it.

  3. Stop Testing AI Agents Like Deterministic Code

    Your AI agent CI is a coin flip and you don't know it. Here's how to test stochastic systems like an adult.