We're living in the age of the stochastic machine.
Computers are getting more and more powerful, and they’re dominating more and more things we do in life. It's not just that we're staring at the computer all day to do what's commonly called “work.” We are impacted by the very decisions that computers make – a credit approval, a tax filing, or when that traffic light finally turns green for us.
And unlike the pre-LLM era, these computers no longer follow very clear rules that anyone with a programming background can inspect and verify. The rules that computers follow now are non-existent.
These computers are becoming very sophisticated and very powerful, and very, very opaque. If you ask one of them, why it decided like this or why it decided like that, is going to come up with a very sophisticated response. The only problem with the response is that you don't know if it's actually true. In fact, the machine itself doesn't know if it's true and why it reasoned the way it did!
We’ve lived in an unreliable world forever. Why the fuss?
Now, putting the word computers aside, this is very much the world that we've always been living in—the world of humans. Humans are very sophisticated calculators that work in very opaque ways.
That's why psychology is still a battlefield with a reproducibility crisis that doesn't want to stop. So why suddenly are we having a problem with computers not being reliable?
While we've been dealing with unreliable humans for all our lives, the killer argument here is that AIs are not us. All of us here are humans, except for – hello – all the bots that are reading this article. (Yes, I mean you!)
And even though another human across the room may behave in what we deem illogical ways, we can kind of empathize with that human if we know enough about them. With most humans, enough empathy gets us to some degree of understanding of the reasons why they did something weird, or bad – and that counts even if it’s not very scientific but rather based on intuition alone.
Computers are a bit different from us. For example, if you feed a current-day LLM with enough additional context, its reasoning and math capabilities go down the drain. That's a bit like if you sent a ninth-grader human to some extra schooling in all subjects, and as a result the person didn’t get somewhat better in all subjects – it became surprisingly good at history, got worse at math, and became dumber overall.
That's normal AI behavior and it’s not intuitive at all. I think that this is part of the reason why we're scared of AI, because we don't really know in which ways it's unreliable – I just cited a well-known behavior, but there are so many more that we don’t even know about yet.
With humans, in contrast, we very intimately do understand how they work, however illogical they sometimes are. Plus, we've got thousands and thousands of years of experience to point back to, and institutions and systems that grew around, and because of, human unreliability.
Intimacy in the era of AI
Which brings me to intimacy. I'm a writer myself and I do make use of AI in various ways. But when I read an article that sounds mostly AI generated, I feel cheated. Because when I'm reading a human-written piece, it feels like I'm getting a front seat in that person's mind, and that’s what gives me a sense of intimacy. It reveals how that person thinks and feels, and that's exciting for me.
However, I'm not very interested in empathizing with a computer. I know it's a human creation, and I know it can fake some feelings, but really it's not the same experience as connecting, even through a screen, with another being that’s made of flesh and blood like myself. I want human connection, not computer connection.
In writing, we actually prize unreliability because it's exactly what makes us so human. Unreliability becomes creativity.
In contrast, if I reliably read the words “genuinely” and “honest” in the same paragraph, then I know that this was Claude’s making, and I feel cheated. If I see an em-dash too often, I know this was ChatGPT, and I feel cheated. Too many of these phrases break the unilateral contract of trust that I had imagined myself having with that particular author (or sometimes with myself, because I, too, am guilty of having generated some AI-powered swivel in the past)
AIs don’t understand relationships
This brings me to the next relationshippy thing, which is repair. Humans do make mistakes, many mistakes. But when a human that I care about makes a mistake, then I can always go up to that human and tell them my opinion on their behavior. They might rectify it or not – but the point is that there is a clear protocol as to how set errors straight.
With an AI, it's very different. If I tell the AI that it’s wrong, it will profusely apologize (sometimes even when it wasn’t actually wrong), and then just make that same mistake again! If I’d meet a human who systematically behaved this way, I’d label them something between a pathological people-pleaser and a psychopath.
AI doesn't understand what I really mean when I point out a mistake. And even if it does correct its mistakes, I don't know if it's just correcting those mistakes because it wants to make me happy, or because it's actually, genuinely (that’s my “genuinely,” not Claude’s) understood what I mean by making things right.
AI mistakes scale beyond comprehension
One of the biggest arguments for the AI reliability anxiety is scale. AI systems are so much faster than human brains ever could be, and they're getting faster still. With that come blind spots, and if those blind spots are everywhere all at once, then we might run into massive problems.
Humans have their blind spots – but they're usually distributed in different places. For example, one person might be really bad at tying their shoelaces. And another person might be really bad at not bumping into lampposts on the street. (That, by the way, is a real anecdote of my own couple’s life – your guess on who is who.)
Now, if we live in a world of AI-powered humanoids that are consistently bad at tying their shoelaces and consistently good at dodging lampposts, that creates a society in which everybody dodges lampposts expertly with untied shoelaces, and then trips and falls over their laces anyway…
Luckily, we live in a (humanly) diversified society where some people bump into lampposts and other people trip over their shoelaces. With AI, that diversity just doesn't exist, because there are maybe a dozen widely used models and not eight billion of them. And often don't know what those widespread failures are until it's too late, when they’re already everywhere. Which is scary.
In addition, humans can reverse their mistakes (at least the more intelligent ones). AIs make mistakes very fast, and then not know how to reverse a mistake. For example, if I'm writing a piece of code with AI, then that code may contain a bug, and when that bug doesn't get detected and I have to undo that bug, that takes so, so many hours of frustrated prompting. (I’m speaking from experience.) With a human author, they would spot the bug in the correct line of code and fix it with a couple edits. A similar logic applies to things non-code.
What happened with privacy?
When we share our problems with AI, we don’t really know where all our private information is going. If I complain to an AI about my partner not being able to tie their shoelaces, then could Sam Altman in theory know that a random guy in Paris is frustrated about this? Nobody really knows.
Of course, there are so many problems being confided to AI every minute these days that it’s impossible for anyone to keep up with this swamp of human quandaries. My little problems are probably being buried by everybody else’s.
But the point stands: if I complain about my big fat shoelace problems to my best friend, it stays with my best friend. Maybe they’ll tell another friend, in which case, obviously, I’m going to give them a hard time – but for the most part, my information doesn’t travel very far, and besides, my best friend might even come up with some nicer solutions than those of an AI. Plus, it’s much nicer to look at an empathetic face over a cup of tea rather to stare at another screen.
Will human knowledge go down the drain?
Another huge problem is this fear that we’re losing key capabilities. What happens when we outsource so many key tasks to AI that we forget how to do things manually anymore?
For example, it’s been a long while since I’ve actually authored a block of code all by myself. I’ve edited code, I’ve organized code, I’ve architected code, yes and yes and yes. But I haven’t really written a single new function from scratch for the past three years or so. And I’m sure that I’m not as good at this task as I was back in the days.
What happens when this forgetfulness sets in at scale? What if at some point the AI might also forget about it, or changes things around in illogical or irreversible or suboptimal ways? May we then suddenly find ourselves at a point beyond return where we’re sitting with a problem that we can’t solve, and AI can’t solve either? That’s a bit scary, and it may happen in many ways way beyond code.
Accountability in AI is opaque at best
I’ve saved the biggest point for last – contrary to good writing advice, I know.
Accountability ties into the forgiveness, see above, but it’s also much more than that. A doctor or an editor or a pilot or an official has some real stakes in their profession: a name and a license and an employer and professional obligations, and so on. They can make many mistakes, and they do, but they are ultimately held accountable in some way (even if it’s just their own sense of guilt, perhaps, in some cases).
That’s why malpractice lawsuits, professional governing bodies, and justice as such exist. With AI, it’s a different kettle of fish.
I had a minor surgery a few years back, and when I went under the knife, I basically entrusted that doctor with my life. It was a low-risk operation, but still I considered the possibility that I might not wake up on the other end of the procedure. I felt confident nevertheless, because I had trust in the surgeon. But had an AI operated on me – I would have had a very different, very queasy feeling about it.
This comes down to personal preference, obviously, but the point is that when it’s a human doctor and something goes terribly wrong, then my friends and family can sue that doctor and make sure they don’t cause more harm to other patients. If an AI accidentally kills me, there’s no such mechanism.
You see, if ChatGPT performs a bad surgery on me, Sam Altman is not going to jail. Maybe even the hospital operators that implemented that instance of ChatGPT don’t go to jail. Who actually goes to jail when such things go wrong? Nobody really knows.
Besides, a software is pretty difficult to put into jail. We can only pull the kill switch – and there’s very good reasons about why we shouldn’t be threatening AI with that. (The short version is: AI will get more and more incentivized to block us from actually being able to pull that kill switch. It wants to survive after all.)
The point here is that accountability exists in a human world and is very well scripted, and with AI we don’t even really know how to write that script. These AI-facing legal systems are still being developed. Even once it’s matured, perhaps AI law will always feel a little bit unintuitive and opaque to us because we’re humans and not AI, and accountability largely is based on our own feelings of morality.
The verdict: If anything, reliable AI is underhyped
To conclude, is a reliable AI overhyped? I don’t think so.
AI is getting more and more powerful, and we don’t really know where the risks truly are. We don’t know where the blind spots are until it’s too late. (At times, at least. The earlier we detect them, the better.)
We don’t know whether AIs checking their own work with other AIs will be enough of a strategy to mitigate against dramatic mistakes. And we also don’t quite know yet how to incentivize humans to properly check AI-generated work before it’s too late, because we don’t really know how to make such a task feel fun and purposeful yet.
There’s just a lot of question marks when it comes to all these developments, and that’s why I think that the current anxiety that we’re seeing around AI development is a good thing, actually, and juxtaposes the AI-enthusiasm in very constructive ways.
I’m not trying to spread doom and gloom here; I’m excited about AI and use it every day that I’m online. Nevertheless – if anything, we should invest more into guardrails against unruly AI, so that this powerful technology doesn’t bump us in all kinds of edges of this road.
What I know for sure (to say it like Oprah would) is that we have a steep and curvy road ahead of us with AI. And we don’t want to be riding such a road with a high-speed car, ready to be flung out of the next curve.
Slow and steady wins the race. In the era of the stochastic machine, we’d better run many experiments, many little adjustments, really iterate in a very diligent fashion – even if that means looking less “disruptive.” It might sound a bit unsexy if the next AI breakthrough then takes a few months longer to materialize. But for me, that sounds way more appealing than living in a world full of cyborgs who systematically bump into lampposts with perfectly tied shoelaces.
Meanwhile, at Wangari
One week after my summer break, a sweet recognition: Wangari Global is the #2 AI Deployment company on F6S for August 2026. We’ve ranked highly out of the 2 million F6S startups.
Thank you to the team (or AI?) at F6S for this recognition! It means a lot to us to know that many different stakeholders in this weird and exciting time value what we produce.
Reads of the Week
Navin Kabra has a surprisingly down-to-earth article on how to keep an AI in check that’s smarter than you. The trick is to notice that the problem has been solved with humans, and apply those techniques. Think: letting it check its own work, checking the AI’s track record in similar problems, asking it to explain itself. It won’t hedge against the longterm AI problems from this article, but for practical use it’s very good advice.
In a beautifully pointed and poetic way, Brosatsu writes that the only recession-proof career might just be the one as a self-employed artist. Many people daydream about such a thing, only to pursue a “safer” path which is now being automated with AI. I’m not saying that art isn’t being automated by AI at all — it is — but the argument carries merit nevertheless. To me, the trick is to find the art form that’s difficult for an AI to emanate (to date).
If we really want to know about how to make AI reliable, why not learn from the people actually building the AI? That’s how Nikki Siapno is tackling the problem. Exemplified by OpenAI’s data agent, she shows us that key to reliable AI systems these days are context, memory, and evals. The article seems technical but can be understood without a technical background — worth a read.
Great to hear you enjoyed my case study article on OpenAI's data agent, Ari. Thanks for sharing it.