In the 1952 novel “Player Piano”, Kurt Vonnegut spoke of a future where machines conduct all meaningful work. The machines are monitored by engineers who are monitored by managers. Monitoring is humanity’s last problem.
At Raindrop, we’re working to solve that last problem. When we call monitoring agents the “last problem,” friends often look at us incredulously. But within seconds, they understand.
Give us a chance to explain.
The non-obvious truth.
Agents will be responsible (directly or indirectly) for the majority of economic output.
These agents will make mistakes.
Their mistakes will, in the best of cases, be extremely expensive. In other cases, they will be catastrophic.
Tracking these mistakes will become increasingly critical.
We’re already seeing glimpses of how costly problems with agents can be. Production data is getting deleted. Critical security vulnerabilities are being introduced. Sycophancy is straining real-world relationships.
We don’t know exactly when or how these problems will be solved, but we do know that our last problem will be preventing agents from having these problems.
It’s only getting more important.
A few years ago, we naively thought that as agents got “better”, they’d have less problems. It’s now obvious this couldn’t be further from the truth. Not only do agents have more problems today, these problems are more critical than ever.
We call this the “double-whammy” of increased capabilities:
1. Increased capabilities = increased complexity
As capabilities increase, agents become more complex. As agents become more complex, they become harder and more expensive to test. There’s simply no comprehensive set of evals that will capture every edge-case.
Agents are failing in ways stranger than their developers could have ever imagined (this is more and more true as developers use coding agents to build their agents!) As agents run longer and are given more open-ended tools, production becomes the source of truth.
2. Increased capabilities = higher stakes deployments
At the same time, those same hard-to-test agents are being deployed in increasingly high-stakes environments. They control production data. Give legal counsel. Prescribe medication. Provide religious and financial advice. And, soon, fight our wars.
Higher stakes = less room for mistakes.
Our last problem, or last purpose?
Our purpose as humans has largely come from solving problems. But there’s a quickly shrinking gap between what makes us special, and what agents are capable of.
Growing up, my (Ben’s) mother was a translator. She was one of the best translators in the world. She speaks 4 languages: English, Spanish, Portuguese, and Russian.
Being a good translator took intelligence. It took skill and grit. It was so human.
Now, it has been almost entirely automated. It was one of the first careers to die, but it won’t be the last. Medicine will be automated. Software engineering will be automated. As of last month (February 2026), many engineers have already stopped writing code.
In many ways, it feels hard to believe. There are still so many things that humans are uniquely capable of. Starting a company, writing a great book, perfectly architecting a production-scale backend.
Internally, we define a problem as any case where a human can still improve an agent’s output. (e.g. where a human has a better sense of what “good” looks like)
A lot of these problems look really dumb. They’re easy to laugh at on Twitter, because the failure modes are so foreign to us.
But someday, agents will have no problems left, and we’ll be left figuring out what to do with ourselves. And until that day, you’ll need production monitoring.
“’If it weren’t for the people, the god-damn people’ said Finnerty, ‘always getting tangled up in the machinery. If it weren’t for them, the world would be an engineer’s paradise.’”
― Kurt Vonnegut, Player Piano
Special thanks to Zubin Koticha, Paul Graham, Tiago Sada, Kodey Converse, Ryan D’Onofrio, and Bridget Hylak for their thoughts and feedback.
