Replace static evals with an eval flywheel driven by customer signals — turn post-launch hope into data
Why 80% Reliability Isn't Good Enough — Felipe Blanes, Amazon AGI Lab — AI Engineer · Read the original ↗
This page is a translation of the Korean original. Read the Korean original →
What you get
- Eval scenarios come from real production use, not imagination, and are refreshed every week
- Launch decisions can rest on the trust threshold customers actually feel (the gap between 80% and 90%)
- Stating limitations up front earns more trust than benchmark scores do
When to use it
Apply it when you have shipped a product to customers but still run only static evals, leaving a gap with real usage. It also applies when customers move through the adoption journey (use-case discovery → early adopters → scale → product gaps) and your eval criteria go stale.
How to do it
- Define success by the customer's standard first, not your own.
- Collect data through instrumentation, but always supplement those signals by talking to customers directly in meetings and visits.
- Sort the signals into three buckets: model problems, harness (engineering) problems and product problems.
- Prioritize the gaps you found, hand them to science, engineering and product teams as decisions, and repeat the loop.
- Draw eval scenarios from production rather than imagination, share capabilities and limitations honestly with customers, and spin the flywheel as fast as you can.
What to borrow
Start by checking whether your eval set is filled only with scenarios you imagined, or with ones drawn from real usage logs and user conversations. Log metrics alone are easy to misread, so build direct questioning into the loop, and adopt the habit of labeling each failure as a model, harness or product problem and passing it to the right team as a priority. When you publish reliability figures, it is better to state what does not work alongside what does.
Links
Background — what the original covers
In this talk, Felipe Blanes of Amazon AGI Lab draws on his experience handling customers directly while running Nova Act, a browser-agent tool, from research preview to general availability on AWS. Scores raised on public benchmarks and synthetic data collapsed when real users used the product in ways no one expected; he calls this gap the benchmark illusion. As an alternative, he proposes a customer-centered loop that runs from defining success through collecting signals and classifying gaps to feeding them into decisions.