Spain won the 2026 World Cup. Congratulations to Spain. And, I suppose, congratulations to me, because several of the eleven models I built to predict the tournament crowned Spain before a single ball had been kicked.

That sounds like a resounding success, but this is not a victory lap. I promised to be critical of my own models, and the truth is that picking the winner is the least interesting fact of all. In the enterprise world—just as in sports forecasting—calling an outcome correctly does not mean your system is actually working.

The consensus of my eleven models gave Spain a 20% chance of winning, followed by France and Argentina at 14%, and then the Netherlands and England. Four of those top five reached the semifinals. It looks like a clean sweep. I could call it a good night, pack up, and sell the algorithm.

But I would be wrong to do so. Because being right is not the same as being calibrated.

The Illusion of Accuracy

A model that gave Spain a 20% chance and a model that gave Spain a 95% chance both “called it,” because both had Spain as the maximum percentage. But in the 20% case, you are not actually stating with high confidence that Spain will win.

It is like flipping a biased coin once, seeing it come up heads, and deciding you understand the bias. You have learned something, but not much. This nuance gets skipped in almost every forecast retrospective I read. The final outcome is a terrible grading instrument. The proper way to evaluate a model is to apply a rigorous scoring rule to every single match prediction made before the tournament began.

Being entirely agnostic about the final champion, I ran all 104 matches through all 11 models and scored them using three metrics: the Brier score, Log Loss, and the Ranked Probability Score (RPS).

The Brier score measures the mean squared error of a probability forecast; lower is better. A naive model that simply guesses 33% for every outcome (win, lose, draw) scores a 0.667. Log Loss severely punishes confident wrongness—it is the metric that catches a model assigning a 3% probability to an event that actually happens. Finally, RPS respects that a draw is an intermediate state between a win and a loss, rewarding models that understand the structure of the game.

The Boring Models Dominated

When the dust settled, the results were definitive. Elo performed best. Poisson was second. The absolute worst model of the eleven was XGBoost.

My pre-tournament prediction was confirmed: every single rating and goal-based model beat every single machine learning classifier. There was no overlap. It was a clean break between the two families.

This goes to show that a simpler model with strong inductive bias is often vastly superior to throwing full-power AI at a problem. XGBoost scored a 0.689 on the Brier scale—meaning the most flexible, sophisticated model in the suite was literally worse than blindly guessing 33% (win / draw / loss) for every match. The most complex model was worse than not modeling at all.

There is one caveat: on RPS, XGBoost did beat the baselines. Because RPS respects the ordering of outcomes, XGBoost’s draw-heavy predictions were structurally sensible, even if the raw probabilities were garbage.

Interestingly, Random Forest achieved the best cross-validation accuracy during training, yet it finished ninth out of eleven on the Brier score. Accuracy and calibration are entirely different axes. As AI researchers Sayash Kapoor and Arvind Narayanan have pointed out regarding autonomous agents, a system can achieve high accuracy while remaining dangerously uncalibrated.

The Failure of the Ensemble

Before the tournament, I predicted that the consensus (the average of all eleven models) would beat most of its individual members.

I was wrong. The consensus came in sixth out of eleven. It lost to all five of the simpler rating and goal models. That is a failure.

Averaging only cancels out errors when the member models are comparably good and their errors are uncorrelated. But half of my modeling suite was systematically worse because the machine learning models were too complicated for the sparse data. By averaging them in, the consensus actually imported their errors. Ensembling as a reflex cost me dearly against simply using Elo—a system invented for chess in the 1960s.

The Champion is a Worthless Statistic

Colley, a matrix-based rating system, picked the Netherlands to win the tournament. The Netherlands went out in the Round of 32 on penalties. Yet, Colley finished fifth overall on the Brier score, out-forecasting six other models, including the two that nearly picked the correct champion.

Random Forest and XGBoost picked Argentina—the eventual runner-up. Yet they finished ninth and tenth in overall forecasting quality.

The two models that came closest to naming the champion were the worst forecasters. The model with the most embarrassing headline miss was solidly above average. It is not about who lifts the trophy; it is about the 104 matches in between. One tournament outcome does not rank forecasters.

Where the Calibration Broke

Overall, the consensus model was slightly underconfident. The average predicted probability for the favored team was 53.1%, but favorites actually won 58.7% of the time.

However, this miscalibration was not spread evenly. It was heavily concentrated in a single band: the 50% to 60% bucket. In matches where the model gave the favorite a 54% chance, reality delivered a 73% win rate.

Above 60% and between 40-50%, the model sat perfectly on the diagonal of perfect calibration. Below 40%, it was mildly overconfident about underdogs. The draw curve—which I fitted separately—was a triumph. The mean predicted draw rate was 25.7%, and the actual tournament draw rate was 26.0%. It does not get closer than that.

The headline on my chart claims that favorites came in stronger than the model dared to predict. That is true in aggregate, but the honest version is narrower: one specific probability band was badly off, while the rest performed admirably.

The Missing Data Problem

There were two matches that the models called worse than any others, and both involved Cape Verde.

In the group stage, Spain played Cape Verde to a 0-0 draw. The model had given Spain a 76% chance of winning, resulting in a massive Log Loss of 1.838. Later, Argentina played Cape Verde in the Round of 32. It was level after 120 minutes. The model had given Argentina a 72.8% chance, resulting in a Log Loss of 1.761.

Cape Verde were tournament debutants. They had zero match history in our 358-game training set. Because the models had no data, they fell back to the prior distribution.

This was not a failure of probability tuning; it was a coverage gap. All eleven models shared this blind spot. While ensembling reduces method variance, it cannot manufacture information about missing data points. If the data does not exist, averaging eleven ignorant models just gives you a highly confident ignorant model.

(As an aside: the model was not worse in the knockout stages. The group stage Brier score was 0.547, and the knockout stage was 0.501. This directly contradicts the usual sports cliché that “knockouts are a coin flip.”)

The Enterprise Lesson

What do we learn from this? And how does it translate to enterprise AI, business forecasting, or resource allocation?

First, the best performing system in the suite was Elo—a simple, elegant model from the 1960s. Complexity must match data volume. If you have limited data or high irreducible noise, a simple model with strong inductive bias will almost always beat a complex neural network.

Second, do not ensemble by default. If half your models are importing noise, averaging them makes your system worse.

Third, audit your coverage. If your enterprise model is forecasting a new market, a new product, or a new competitor (your business equivalent of Cape Verde), recognize that the model is falling back to a prior. It is guessing.

The common thread for deploying AI in the enterprise is this: write your assumptions down. Commit to the grading rule before the results happen. And when the project is over, publish the grade—especially the parts that make your earlier arguments look worse. That is how you build reliable systems.

Meanwhile, at Wangari

While our core business remains enterprise reporting, we’re sharing our soccer models with the world!

Out now on Spartera are two models:

• Soccer Match Probability Outcomes: Updated daily, tells you the probability of any match outcome in major European leagues.

• Soccer Team Form Ratings: Tells you how well your teams in European leagues are doing right now. Updated daily.

Spartera is a brilliant platform that allows buyers and sellers of data to come together and share their pots of gold (ahem, insight).

Reads of the Week

• AI’s Reliability Gap: presents rigorous, well-sourced essay arguing that the limited economic impact of AI doesn’t stem from capability gaps but from reliability gaps. That includes inconsistency, brittleness under minor perturbations, and the verification burden that defeats the purpose (ever had to review AI-generated results?). Essential reading for anyone deploying AI in production.

• AI Infrastructure Is Moving Up the Stack: A clear-eyed analysis by of how the AI bottleneck in Q2 2026 shifted from hardware to the software control plane: orchestration, evaluation, observability, and governance are important now. She argues that better models don’t automatically create reliable products. I fully agree: the hard part of AI is not the model; it is the surrounding system.

• The AI Superforecasters Are Here: notes that AIs are now approaching human superforecaster performance on prediction markets. What does it mean when AI can beat humans at calibrated probabilistic reasoning? Ultimately, it boils down to this: the big question is when to trust a model’s probabilities, and when to defer to your own judgement.

A few months ago, my team showed a client some AI-generated regulatory reporting. We were proud of it — the output was sophisticated, which is exactly why we put it in front of them. The client looked at it and said, more or less: “This is highly sophisticated garbage. It sounds nice, but we don’t even know what it means.”

He was right. And in our line of work, that sentence is a serious problem — because what we do is financial reporting, and in financial reporting a mistake doesn’t just cost money. It can get you into jail.

That was one of the more humbling moments of my career. I’ve spent the better part of a decade building machine learning and AI systems — in particle physics, in weather insurance, and most recently in financial reporting. I know what these models can do. After that meeting, I also knew, viscerally, what they must never be allowed to do.

So we turned around and built something different. For ten years my instinct had been to take humans out of the loop — automate more, decide more, need people less. This time I did the opposite. I put humans deliberately back in.

The reason the temptation exists

Anyone inside this world knows financial reporting is ripe for automation. The work is repetitive, it comes around every reporting season, and yet it demands real skill — you can’t hand it to just anyone. It has to be done by highly paid professionals who, frankly, would rather be doing almost anything else.

That’s the promise our product, etio, delivers on. It cuts the reporting cycle from three or four days per team, per season, down to a few hours. Honestly it can do it in minutes — but with the normal sign-offs and deep dives, we conservatively say hours. Either way, that’s roughly 97% of the time gone.

Here’s the catch, and it’s the whole story: AI hallucinates. Large language models are non-deterministic by definition, which is a technical way of saying they sometimes just get things wrong. Cutting the time and producing beautiful reports is the easy part. The real question — the one that keeps a CFO up at night — is what you do about the mistake that will, sooner or later, appear.

Our answer isn’t “trust the model”

It has two parts, and neither of them is “trust the model.”

First, we don’t let the AI invent numbers. We pre-calculate the facts — the actual figures — with old-fashioned, deterministic code. Not AI. Code. The language model never gets to make a number up; it only ever gets to talk about numbers that were already computed and checked.

Second, we put a human at two specific gates in the workflow. At each gate, a person reviews what the machine produced and either approves it or corrects it. Nothing moves past a gate on the machine’s say-so alone. In an era where every AI pitch is about removing the human, we deliberately kept two of them — and made their signature the thing that unlocks the result.

That’s what I mean when I say the computer has to say “no” sometimes. The most important thing etio does isn’t generate — it’s refuse. It refuses to state a number it can’t back, and it refuses to finish without a person.

Where this gets exciting: data!

Automating a dull process might sound like dull work. It isn’t — and this is where it gets genuinely interesting.

The data that reporting runs on is a goldmine. Every season, enterprises painstakingly pull it together from every corner of the business — and then underuse it. Their data scientists exist, but they’re not pointed at this. Silos and legacy systems keep the most interesting signal buried. But by the time you’ve finished a close, all of that data is finally sitting in one place, clean and reconciled. So we step in and do something more with it.

Once the reporting itself becomes almost trivial, you can use the same data to enhance human judgment. And I don’t mean the reporting team’s judgment — I mean the executives’. Because who actually reads financial reports? Executives do. And what do they do with them? They set corporate strategy. They brief their boards. They answer to regulators. The report isn’t the end of the process; it’s the input to every important decision the company makes.

So etio takes that reconciled dataset and surfaces the strategic insight sitting underneath it — using techniques that go well beyond a standard data scientist’s toolkit, drawn from my decade across particle physics, weather insurance, and sustainable finance. The most important of these is causal inference: not just what happened, but why. The reporting pays the bills — you have to do it anyway. But those extra insights are the gold. They’re what actually moves the company forward.

What it looks like in practice

Say we’re closing Allianz — a large German insurer — on last year’s numbers.

The user does almost nothing: click new close, then start close. Behind that click, the machine works hard. It identifies the reporting standard — for Allianz, IFRS 17 — builds the facts and reconciles them under that standard, and then hands every calculated number to a human.

That person might be a controller, a data manager, or an actuary. They review: Is this plausible? Does it make sense? They can recompute a figure by hand, run their old Excel against it, whatever they trust. If they’re satisfied, they click accept.

That’s the first gate, the one that goes against all the automation talk.

The second gate is for the commentary. Every regulatory report needs prose — words that explain what the numbers say. etio drafts that commentary with a fairly sophisticated suite of LLMs and agents, but it composes it only from the numbers already calculated, which is exactly how we keep hallucinations out. The human reviews it, edits it if they like, and signs it off.

Whole teams can work this way with etio, with responsibilities split across roles. Then, finally, you simply download the reports.

The really meaty part: follow-up questions

From there it opens up. You can ask an AI assistant — with the administrator’s permission — the questions you actually have: “I don’t understand this number, how did it come about?” or “Why did this move so much versus last year? What changed?” And the answers, like everything else, are hallucination-free.

And then you can look at the crown jewel: the causal graph. It’s a chart that shows what causes what, and by how much. In insurance, for instance, rising loss costs — more claims, or bigger ones — feed into frequency and severity, which in turn drive the loss ratio. Those relationships can now be quantified mathematically and tested statistically: is this actually true, or not?

Building such causal graphs, at scale, wasn’t really possible before this era of AI. It’s no longer just seasoned human intuition; it’s computation.

None of it is a regulatory requirement. But it guides strategy, and it answers the follow-up questions — because regulators are humans too, and humans ask why. Sometimes they ask more than they strictly should. So it’s good to have the answer ready.

Our vision for this

This etio today. It saves enterprises 97% of boring-work time. But the vision is bigger than a faster close.

Imagine you had this picture across many enterprises — how each one acts in its environment, and what that does to it. You could tell a company, with evidence, what it should do to genuinely thrive. That’s the end goal of Wangari: to show that acting in a certain way lets you prosper — not only as an enterprise, but for your customers too.

And that starts with one honest close.

Because in the end, this is the whole point: systems that serve humans, not the other way around. We build machine learning and AI to serve people — which means people have to keep their place in the workflow, at every step. That isn’t a limitation. That’s the design.

Meanwhile, at Wangari

Today is a slightly different “meanwhile” section than usually. Really, it’s a housekeeping note.

We will be pausing the Friday podcast for now because I’ve come to realize that me rambling alone in a room is probably not the best use of everybody’s time. I’m now working on lining up valuable guests for a future podcast relaunch (as a dialogue, not a monologue).

I can’t give you a date for that just yet though, because as you can see, we’re quite busy building etio already 🙂

So, for now, we’ll have just the Tuesday newsletter. And you’ll miss me on the Tuesdays of August 11 and 18, while I’m busying myself in a meditation retreat 😇 (Don’t worry: we’re posting weekly before that time, and will be back thereafter from August 25.)

Reads of the Week

• Why AI just made consulting more expensive: An eye-opening essay by arguing that, while the grunt work of consultants is being automated away — think making slide decks, maybe some code, a slick presentation. The point is that that’s not the real work of a consultant, which is to ask hard questions and provide human judgment, context and accountability. The accountable judgment layer doesn’t get any less valuable; it just so happens that many junior consultants don’t have much to do anymore.

• AI-Ready Data Is Not Decision-Ready AI: explores the gap between data that machines can read and AI systems that organizations can trust. She argues that trust-critical AI must make uncertainty visible and keep decisions traceable, preserving human oversight. It is an essential read on why making data “AI-ready” is only half the battle; the real challenge is designing systems that allow humans to remain accountable for the final decision. (Learnings for etio!)

• What We Give Up When We Let AI Decide: examines the creeping automation of judgment, specifically in the context of education and grading. While his focus is on the classroom rather than the boardroom, his core warning applies universally: shifting from automating logistics to automating judgment comes with profound costs — even when it feels like a routine task. It serves as a reminder of why we must protect the human "gates" in our own workflows.

In this episode, Ari discusses his new book, Soccer Analytics with Machine Learning (O’Reilly Media), and explains why the best way to understand complex data science — from logistic regression and feature engineering to Monte Carlo simulations and causal inference — is to watch a soccer match with an analytical eye. Whether you are predicting a goal or simulating a business strategy, the math is the same. And yes, he will explain why trying to A/B test a Champions League final is a terrible idea.

Topics covered: Soccer analytics, Expected Goals (xG), Monte Carlo simulations, machine learning education, feature engineering, *Soccer Analytics with Python*, O’Reilly Media, GenAI Academy.

Wangari is the newsletter and podcast for practitioners and leaders navigating the real work of enterprise AI. New episodes every Friday.

Summer is here, the World Cup is in full swing, and if you are anything like me, your screen time is currently divided between complex data architecture diagrams and live soccer broadcasts.

It might seem like a stark contrast — the rigid, deterministic world of enterprise software versus the fluid, chaotic, deeply human drama of a soccer match. But the longer I work in AI, the more I realize that the two worlds are fundamentally connected.

In fact, I believe that the best way to understand the hardest concepts in machine learning is not to stare at a whiteboard full of equations, but to watch a soccer match with an analytical eye.

The Pitch as a Data Problem

Consider the challenge of predicting whether a specific shot will result in a goal.

Ten years ago, analysts might have looked at the striker’s historical goal-scoring record and the distance to the goal. It was a simple, linear approach.

Today, the “Expected Goals” (xG) models used by top clubs are vastly more sophisticated. They don’t just look at the shooter; they look at the x,y coordinates of every defender on the pitch. They calculate the angle of the shot, the velocity of the pass that led to it, and the defensive pressure applied in the preceding three seconds.

This is a classic machine learning problem. It requires taking a massive, noisy dataset (the continuous movement of 22 players) and engineering specific, predictive features (the geometric area of the goal visible to the shooter) to train a probabilistic model.

When you understand how an xG model works, you also understand the core principles of logistic regression and feature engineering. And behind that, you understand how to separate signal from noise.

The Limits of Machine Learning in Sports

Machine learning in sports has its limitations. As Alex Marin Felices notes, while machine learning models are excellent at identifying correlations between performance indicators and success, they often struggle to represent the interactive and dynamic nature of attacking play.

A model might identify that a team is more likely to score when they complete a high number of passes in the final third. But this is a correlation, not a causal mechanism. Does completing more passes *cause* the team to score, or do dominant teams simply complete more passes while they are in the process of scoring?

This is where causal inference becomes essential. To truly understand the game, analysts must move beyond pattern matching and build models that can answer “what if” questions. What if the manager substitutes a defensive midfielder for an attacking winger? How will that change the underlying dynamics of the match?

Simulation and Strategy

Or consider the problem of tactical adjustments. If a manager decides to switch from a 4-3-3 formation to a 3-5-2, how will that impact the team’s defensive solidity?

You can’t test this deterministically. You cannot run the exact same match twice. There’s no parallel universe we know of that would allow that. Instead, analysts use Monte Carlo simulations—running thousands of simulated matches based on historical probability distributions to forecast the likely outcomes of a tactical shift.

This is the exact same mathematical framework used by financial institutions to model market risk, and by supply chain managers to optimize logistics networks. When you understand how to simulate a soccer match, you understand how to simulate a business.

The Future of the Game

The integration of AI into soccer is only accelerating. For the 2026 World Cup, FIFA is planning an “AI offside revolution,” using AI-enabled 3D avatars of every player to ensure precise identification and tracking.

This level of data capture promises unprecedented accuracy, but it also raises questions about the role of human judgment. As decisions become more exact, they can also feel more arbitrary to spectators. This tension between technological precision and human interpretation is a challenge we face every day in enterprise AI deployments.

In a business setting, it might mean something like this: when an AI system flags a regulatory report for an anomaly based on a complex, multi-dimensional analysis, the human compliance officer must decide whether to trust the machine or their own intuition. The more precise the machine becomes, the harder it is for the human to overrule it, even when the machine is wrong.

The Democratization of Analytics

One of the most exciting developments in sports analytics over the past decade is the democratization of data. Tracking data that was once the exclusive preserve of elite clubs with multimillion-dollar analytics departments is now increasingly available to smaller clubs, coaches, and even fans.

This democratization is driven by the same forces transforming enterprise AI: the plummeting cost of compute, the proliferation of open-source tools, and the increasing availability of pre-trained models. A small club in the third tier of English football can now run xG models that would have been cutting-edge at a Premier League club five years ago.

The same dynamic is playing out in the enterprise. The AI tools that were once the exclusive preserve of the largest technology companies are now accessible to mid-sized organizations. The question is no longer whether you can afford to use AI; it is whether you have the operational maturity to deploy it reliably.

This is the central challenge of our era, and it is the challenge that our new book Soccer Analytics with Machine Learning is designed to address.

The Summer Reading

This connection between the beautiful game and the rigorous math of data science is the premise of the new book I co-authored, Soccer Analytics with Machine Learning, published by O’Reilly Media.

We wrote the book because we are (a) soccer enthusiasts, and (b) frustrated by how machine learning is typically taught. Most textbooks are either overly academic, drowning the reader in abstract calculus, or overly simplistic, treating algorithms as black boxes.

We wanted to write a book that bridged the gap. A book that taught real, production-grade data science techniques; but that grounded them in a domain that is intuitive, visual, and fun.

So – if you are looking for a summer read that will sharpen your technical skills without feeling like a textbook, I hope you will pick up a copy. Whether you are a seasoned data scientist looking to apply your skills to sports, or a soccer fan looking to break into the tech industry, the lessons learned on the pitch are universally applicable.

Meanwhile, at Wangari

The summer slowdown is a myth at Wangari. We just ran a wonderful O’Reilly live course on soccer analytics, which our production editor described as “priceless stuff” (yes, really).

Furthermore, we are currently wrapping up the final week of our From Demo to Production course cohort, and the energy has been invaluable.

It’s been beautiful to see the theoretical concepts of orchestration and evaluation applied to real, complex enterprise data. There’s a load of human work behind all this!

Which reinforces the core Wangari philosophy: AI is not magic; it is engineering. And when you apply rigorous engineering principles to AI, you can solve problems that previously seemed impossible. (And you don’t need to be an engineer to pull that off.)

Reads of the Week

• The Promise and Limits of Machine Learning in Football Attacking Analysis by : Shared before, but worth stating again – a critical look at why machine learning models often struggle to capture the dynamic, interactive nature of soccer. Felices argues that while ML is great for identifying correlations, it falls short when trying to model the complex, multi-agent decision-making that defines a successful attack.

• Opinion: Inside the World Cup’s AI offside revolution by : An exploration of FIFA’s ambitious plan to use AI-enabled 3D avatars for the 2026 World Cup. Lisi discusses the tension between technological precision and the human element of refereeing, a debate that mirrors discussions about AI governance in the enterprise.

• ChatGPT Predicts 2026 in the World of Football by : A fun, AI-generated look at what the future might hold for the beautiful game. It’s purely speculative, but it highlights how generative AI is increasingly being used to synthesize narratives and simulate future scenarios in sports media.

He discusses the “physics of enterprise AI,” the necessity of 5-sigma thinking in evaluation, and why treating AI as a complex instrument rather than standard software is the only way to survive the transition from demo to production. He also explains why deploying an LLM without causal grounding is like handing a toddler a loaded particle accelerator.

Topics covered: Particle physics, Sorbonne Université, CERN, active learning, AI evaluation, systems engineering, causal AI, enterprise AI reliability, Wangari Global origin story.

Wangari is the newsletter and podcast for practitioners and leaders navigating the real work of enterprise AI. New episodes every Friday.

People often ask me how I transitioned from theoretical particle physics to founding an enterprise AI company. On the surface, the two fields seem entirely disconnected. One is concerned with the fundamental nature of the universe, the other with automating regulatory reporting for insurance companies.

But the truth is, the intellectual rigorousness required to search for dark matter is exactly what is missing from most enterprise AI deployments today.

When I was completing my PhD at Sorbonne Université, my research focused on developing active learning algorithms to explore the parameter spaces of Beyond Standard Model (BSM) physics. We were looking for signals of dark matter—signals that are incredibly faint, buried in massive amounts of noise, and highly susceptible to systemic errors.

In that environment, you cannot rely on “vibes.” You cannot deploy a model because it looks accurate on a test set. You have to understand the underlying causal mechanisms. You have to quantify your uncertainty rigorously. You have to build systems that are robust against unexpected perturbations.

When I left academia and entered the world of enterprise AI, I was struck by how often these principles were ignored.

The Physics of Enterprise AI

In the enterprise, I saw teams deploying large language models with the same casual optimism one might use to launch a new website feature. They were treating probabilistic, non-deterministic systems as if they were standard software applications.

They were building demos that worked flawlessly in controlled environments, and then acting surprised when those same systems hallucinated or failed catastrophically in production.

They were missing the physics of the problem.

Building a reliable AI system is not just a software engineering challenge; it is a complex systems engineering challenge. It requires understanding the interactions between the data layer, the model layer, the orchestration layer, and the human operators. It requires anticipating failure modes and designing graceful degradation paths.

Most importantly, it requires a fundamental shift in how we evaluate success.

The Rigor of Evaluation

In particle physics, a discovery is not claimed until the signal reaches a statistical significance of 5 sigma—meaning there is less than a 1 in 3.5 million chance that the result is a statistical fluke.

While enterprise AI does not require 5-sigma certainty, it does require a level of rigor that goes far beyond the standard “accuracy” metrics used today.

When we build systems at Wangari, we apply the same rigorous evaluation frameworks I learned in physics. We do not just ask if the model is accurate; we ask if it is reliable. We measure its variance. We stress-test it against edge cases. We build automated test suites that continuously monitor for silent degradation.

We treat the AI system not as a black box, but as a complex instrument that must be calibrated, monitored, and governed.

The Causal Foundation

Perhaps the most profound lesson I brought from physics to AI is the importance of causality.

In physics, correlation is interesting, but causality is everything. You cannot understand the universe by simply observing patterns; you have to understand the underlying forces that drive those patterns.

The same is true in the enterprise. A correlative model might predict that a customer is likely to churn, but it cannot tell you why, or what intervention would prevent it. A causal model, on the other hand, provides a transparent chain of reasoning. It allows you to simulate interventions and understand the true drivers of behavior.

This is why Wangari is focused on building agentic and causal AI infrastructure. We believe that for AI to be truly transformative in the enterprise, it must move beyond pattern matching and embrace causal reasoning.

The Transition to AI-Native

The transition from academia to the startup world also mirrors the broader transition happening across the economy. As Jakob Nielsen points out, the only way to have 5 years of experience with AI by 2030 is to have started in 2025. The transition period we are in right now is the critical window for individuals and organizations to adapt.

This is particularly true for founders. The playbook for building a startup has fundamentally changed. As Henry’s Best Hits outlines, building a lean, AI-native startup in 2025 requires a completely different approach to team structure, product development, and go-to-market strategy. It requires leveraging AI not just as a feature, but as the core engine of the business.

Venture capitalists are also adapting to this new reality. They are no longer funding thin wrappers around foundation models. As the AI-Native Founder newsletter notes, [investors are increasingly looking for deep technical differentiation](https://ainativefounder.substack.com/p/ai-didnt-just-change-what-we-build) and teams with the domain expertise required to solve hard, unsexy problems [3].

The Importance of First Principles

In physics, when you encounter a problem you don’t understand, you return to first principles. You strip away the complexity and focus on the fundamental laws governing the system.

The enterprise AI industry needs a return to first principles. We need to stop chasing the latest benchmark scores and start focusing on the fundamental requirements of production systems: reliability, auditability, and causal understanding.

This is the philosophy that drives our work at Wangari. We are not interested in building the flashiest demo. We are interested in building the infrastructure that allows organizations to deploy AI safely and effectively in the real world.

The Journey Continues

The journey from Sorbonne to Wangari has been unconventional, but it has given me a unique perspective on the challenges and opportunities of enterprise AI.

The era of the impressive AI demo is over. The era of the reliable, auditable, causal AI system has begun. And the principles required to build those systems are the same principles that guide our understanding of the universe.

Meanwhile, at Wangari

My book, Soccer Analytics with Machine Learning, is now officially available from also in print, O’Reilly Media. It has been a joy to see readers engaging with machine learning concepts through the lens of the beautiful game. Thank you for all the readers who have already started reading it, exploring the GitHub repo, and engaging with the content.

One of my co-authors, Guanyu Hu, and myself gave an O’Reilly live session yesterday to explain some of the core concepts of the book in a fun and interactive way. It was a fantastic experience — we were quite moved by the level of enthusiasm in the audience.

Reads of the Week

• Use the AI Transition Period to Transition Your Career by : On the flip side of doom and gloom, compelling argument for why the current AI transition period is the perfect time to pivot your career and embrace new technologies. Nielsen emphasizes that early adoption is the only way to build the experiential knowledge required to lead in the AI era.

• How to start a Lean, AI-Native Startup in 2025 by and : Still fresh enough a read, this is a practical playbook for founders looking to build AI-native companies from the ground up. The author details how small, highly technical teams can leverage AI to achieve the output of much larger organizations, fundamentally changing the economics of early-stage startups.

• AI Didn’t Just Change What We Build by : An analysis of what venture capitalists are looking for in AI startups today, emphasizing team expertise and technical differentiation. The piece highlights the shift away from “wrapper” applications toward deep tech solutions that solve complex, domain-specific problems.

In this episode, Ari Joury (PhD, particle physics; Founder & CEO of Wangari Global) explores the “correlation trap” and why it is failing enterprise decision-makers. He introduces the concept of Causal AI — a paradigm shift from pattern matching to systemic understanding — and explains why the ability to answer “what if” questions is the prerequisite for robust governance, auditability, and true autonomous action in the enterprise. He also explains why banning ice cream will not stop the murder rate, and why your predictive maintenance model is basically a very anxious psychic.

Topics covered: Causal AI, causal inference, deep learning limitations, correlation vs causation, enterprise decision-making, predictive maintenance, AI governance, Judea Pearl, GenAI Academy.

Wangari is the newsletter and podcast for practitioners and leaders navigating the real work of enterprise AI. New episodes every Friday.

For the past decade, the enterprise AI narrative has been dominated by a single, powerful paradigm: deep learning. By feeding massive neural networks unprecedented volumes of data, we have achieved remarkable breakthroughs in computer vision, natural language processing, and predictive modeling.

These models are extraordinary pattern-matching engines. They can identify subtle correlations in high-dimensional data that would be impossible for a human to detect.

But as organizations attempt to deploy these models into core business workflows, a fundamental limitation is becoming increasingly apparent.

Deep learning models are exceptionally good at answering the question, “What is happening?” They are fundamentally incapable of answering the question, “Why is it happening?”

This is the correlation trap. And in the complex, high-stakes environment of the modern enterprise, correlation is no longer enough.

The Limits of Pattern Matching

Consider a predictive maintenance model deployed in a manufacturing plant. The model analyzes sensor data and predicts that a specific machine is likely to fail within the next 48 hours. This is valuable information.

But what should the plant manager do about it?

Should they shut down the machine immediately? Should they replace a specific part? Should they adjust the operating temperature? The predictive model cannot answer these questions because it does not understand the causal mechanisms driving the failure. It only knows that certain patterns of sensor readings are correlated with historical failures.

If the plant manager takes an action that changes the underlying system dynamics—for example, by adjusting the temperature—the model’s predictions may become entirely invalid, because the correlations it learned from historical data no longer hold true.

This is the fundamental problem with relying solely on correlative models for decision-making. They describe the world as it was, but they cannot reliably predict how the world will respond to interventions.

The Causal Revolution

Causal AI represents a paradigm shift from pattern matching to systemic understanding.

Unlike traditional machine learning models, which learn statistical associations from data, causal models explicitly represent the cause-and-effect relationships between variables. They incorporate domain knowledge and structural assumptions to build a mathematical representation of the underlying system.

This allows causal models to answer “what if” questions.

Returning to the manufacturing example, a causal model would not just predict the machine failure; it would identify the specific root causes driving the prediction. It could simulate the impact of different interventions—”What if we reduce the operating speed by 10%?”—and provide the plant manager with actionable recommendations.

As Scott Cunningham emphasizes in his work on causal inference, the design stage—where researchers explicitly map out the structural relationships between variables—is the most critical part of the process. Without this structural understanding, any statistical analysis is merely describing correlations, not uncovering truths.

The Subjectivity of Causality

One of the most challenging aspects of causal inference is that it requires making assumptions. Unlike pure machine learning, where the algorithm “learns” everything from the data, causal models require human experts to define the causal graph.

This introduces an element of subjectivity. As some researchers argue, the best that applied causal inference can ever do is take a subjective but reproducible set of variables and state precise structural assumptions about them.

This subjectivity is often uncomfortable for data scientists trained in the objective certainty of mathematics. But in the enterprise, this subjectivity is actually a feature, not a bug. It forces organizations to explicitly encode their domain expertise and business logic into the AI system. It transforms the model from a black box into a transparent representation of the organization’s understanding of the world.

Causal AI in the Enterprise

The imperative for causal AI extends far beyond manufacturing. It is critical for any enterprise application where decisions have significant consequences and where the environment is subject to change.

In financial services, causal models are essential for stress-testing portfolios and understanding the true drivers of market risk. In healthcare, they are necessary for personalizing treatment plans and evaluating the efficacy of new drugs. In marketing, they are required for optimizing campaign spend and understanding the true incremental impact of advertising.

Furthermore, causal AI is a prerequisite for robust governance and auditability. When a correlative model makes a prediction, it is often a “black box.” When a causal model makes a recommendation, it provides a transparent chain of reasoning that can be audited and explained to regulators.

Causal AI and Regulatory Compliance

Perhaps the most compelling use case for causal AI in the enterprise is regulatory compliance. In financial services, regulators are increasingly demanding that organizations not only report their risk exposures, but explain the causal drivers behind them.

A traditional correlative model can tell you that a portfolio has a 5% probability of losing more than 10% of its value in the next year. But a regulator will ask: what are the specific risk factors driving that probability? How would the probability change if interest rates rose by 200 basis points? What is the causal mechanism linking a specific geopolitical event to the portfolio’s performance?

These are causal questions. They require causal models to answer. And as regulatory frameworks like Solvency II and IFRS 17 become increasingly sophisticated, the demand for causal reasoning in financial AI will only grow.

At Wangari, we believe that causal AI is not just a competitive advantage for financial institutions; it is a regulatory necessity.

The Wangari Approach

At Wangari, we believe that the future of enterprise AI is causal.

We are building infrastructure that combines the pattern-matching power of deep learning with the rigorous reasoning capabilities of causal inference. Our goal is to provide organizations with AI systems that not only predict the future, but empower them to actively shape it.

The era of “black box” correlation is coming to an end. The causal imperative is here.

Meanwhile, at Wangari

As we roll out our technology in the world of insurance, we’ll be pitching at Plug and Play Insurtech’s Demo Day on July 2nd. (The event can be found here; if you’re in town in Stuttgart, let me know!)

We believe that reporting data from inside big insurers are exactly the type of goldmine that is rife for digging into, using causal AI. And the insurers agree — we’ve already piloted our flagship product etio with Zurich Insurance Group.

We’re looking forward to some inspiring conversations between business and tech, between pitches and booths, and between humans building something impactful.

Oh, and there’s more! My book, Soccer Analytics with Machine Learning, is now officially available not only as e-book but also in print, from O’Reilly Media. It has been a joy to see readers engaging with machine learning concepts through the lens of the beautiful game. Thank you for all the readers who have already started reading it, exploring the GitHub repo, and engaging with the content.

One of my co-authors, Guanyu Hu, and myself will be giving an O’Reilly live session coming Monday, July 6, to explain some of the core concepts of the book in a fun and interactive way. Attendance is breaking records, I’m told by the O’Reilly team. If you still want to join, sign up fast to secure a spot.

Reads of the Week

• Coming 2025 by : Interestingly, not dated at all — a preview (now review) of work focusing on the critical “design stage” in causal inference and econometrics. Cunningham argues that without a rigorous structural design, statistical methods are essentially blind, highlighting the necessity of domain expertise in causal modeling.

• Your Causal Variables Are Irreducibly Subjective by : A thought-provoking piece on why applied causal inference must embrace subjective but reproducible structural assumptions. The author challenges the illusion of pure objectivity in data science, arguing that explicitly stating our assumptions is the only path to true scientific rigor.

• From what to why: the rise of causal AI by : An investor’s perspective on why causality matters and where it is already making an impact in the enterprise. This piece provides a great overview of the commercial landscape for causal AI startups and the specific industries where the technology is gaining traction.

He explains how the lessons learned from analyzing sports data — specifically the art of feature engineering and the necessity of domain expertise — apply directly to the hardest problems in enterprise AI. Whether you are predicting a goal at the World Cup or automating a Solvency II regulatory report, the fundamental challenge is the same: separating the signal from the noise. And yes, he will explain why feeding raw data into a neural network is like handing a toddler a chainsaw and hoping they build a birdhouse.

Topics covered: Soccer analytics, feature engineering, machine learning, enterprise AI, domain expertise, data science, Soccer Analytics with Machine Learning book launch, O’Reilly Media.

Wangari is the newsletter and podcast for practitioners and leaders navigating the real work of enterprise AI. New episodes every Friday.

If you want to build a machine learning model to predict the outcome of a soccer match, you will quickly discover that the algorithm you choose—whether it is a random forest, a support vector machine, or a deep neural network—is the least important part of the process.

The algorithm is a commodity. The real challenge, the part that determines whether your model is a predictive powerhouse or a random number generator, is deciding what data to feed it.

This is the art and science of feature engineering. And as I detail in my new book, Soccer Analytics with Machine Learning (published this week by O’Reilly Media), it is the exact same challenge that enterprise AI teams face when trying to automate complex business processes.

Whether you are trying to predict a goal at the World Cup or automate a Solvency II regulatory report, the fundamental problem is the same: how do you separate the signal from the noise?

The Illusion of “More Data”

There is a persistent myth in the AI industry that more data automatically leads to better models.

In soccer, we have access to an overwhelming amount of data. We can track the x,y coordinates of every player on the pitch 25 times per second. We know how many passes a midfielder completed, how many tackles a defender won, and the exact velocity of every shot.

But if you feed all of this raw data into a model, it will likely fail. It will find spurious correlations—perhaps the team wearing blue won more often on Tuesdays when it was raining—and it will overfit to the noise.

The same is true in the enterprise. An insurance company has petabytes of historical claims data, customer demographics, and macroeconomic indicators. But throwing all of that data into a large language model will not magically produce a reliable underwriting agent.

Engineering the Signal

Feature engineering is the process of transforming raw data into meaningful signals that a model can actually use.

In soccer, a raw statistic like “total distance run” is mostly noise. A player might run 12 kilometers in a match, but if they are constantly out of position, that running is detrimental to the team.

A much stronger signal is “packing”—a metric that measures how many opponents a player bypasses with a forward pass or dribble. Packing requires complex spatial analysis to calculate, but it is highly predictive of a team’s offensive success. It is an engineered feature that captures the *context* of the action, not just the action itself.

In the enterprise, feature engineering is equally critical. A raw text field containing a customer’s email is noise. An engineered feature that extracts the specific regulatory clause the customer is referencing, and maps it to an internal compliance taxonomy, is a signal.

The Domain Expertise Advantage

The most important lesson I learned while writing Soccer Analytics with Machine Learning is that you cannot engineer good features without deep domain expertise.

A brilliant data scientist who knows nothing about soccer will build a terrible predictive model. They will not know that a pass backward to the goalkeeper is sometimes a brilliant tactical move to reset the press, rather than a sign of offensive failure.

Similarly, a brilliant AI engineer who knows nothing about actuarial science will build a terrible regulatory reporting agent. They will not understand the nuanced difference between two seemingly identical financial metrics, or the specific regulatory context that dictates how a certain risk must be calculated.

This is why the most successful enterprise AI deployments are not built by isolated data science teams. They are built by cross-functional teams where domain experts—the actuaries, the compliance officers, the underwriters—work hand-in-hand with the engineers to define the features that actually matter.

The Limits of Machine Learning

Even with perfect feature engineering, machine learning has its limits. As Alex Marin Felices points out in his analysis of football attacking performance, [machine learning models often struggle to represent the interactive and dynamic nature of play](https://thexgfootballclub.substack.com/p/the-promise-and-limits-of-machine) [1]. They can identify patterns, but they cannot always capture the complex, multi-agent interactions that define a match.

This limitation is why causal inference is becoming increasingly important. To truly understand the game—or the business—we must move beyond correlative models and build systems that can answer “what if” questions.

The Causal Dimension of Feature Engineering

Feature engineering is not just about finding better correlations. At its most sophisticated, it is about encoding causal knowledge into the model.

In soccer, a truly powerful feature is not just a statistical summary of past events; it is a variable that captures a causal mechanism. The “packing” metric is powerful precisely because it measures a causal driver of attacking success: bypassing defenders. It is not just correlated with goals; it is causally related to the creation of goal-scoring opportunities.

In the enterprise, the same principle applies. The most powerful features are not the ones with the highest correlation to the target variable in the training data. They are the ones that capture the true causal mechanisms driving the outcome. Identifying these features requires deep domain expertise and a willingness to go beyond the data to understand the underlying business logic.

This is why at Wangari, we always begin a new engagement by working closely with domain experts to map out the causal structure of the problem before we write a single line of code.

Which Brings me to… The Book Launch

This week, Soccer Analytics with Machine Learning officially hits the shelves.

We timed the release to coincide with the start of the World Cup, a time when the entire world is focused on the beautiful game. But my hope is that the book reaches an audience far beyond sports fans.

The book uses soccer as a vehicle to teach the fundamental principles of machine learning—logistic regression, simulation, deep learning, and yes, feature engineering. It is designed to bridge the gap between academic theory and practical application, proving that you can learn complex data science concepts without getting bogged down in abstract mathematics.

If you are struggling to understand how machine learning actually works in the real world, I invite you to pick up a copy. You might just find that the lessons learned on the pitch are exactly what you need in the boardroom.

Meanwhile, at Wangari

Our book is finally out! You can find it on the O’Reilly Learning platform and everywhere books are sold — as e-book, for now, and in print from mid-July.

This happened just in time, as the soccer world cup ramps up to full swing. Unintended, but fun: One of my pieces, an exclusive run for Towards Data Science, actually got picked up by the Daily Mail. I’m used to publishing on fairly niche topics, so it was fun to see my work in mainstream media!

Reads of the Week

• The Promise and Limits of Machine Learning in Football Attacking Analysis by : A critical look at why machine learning models often struggle to capture the dynamic, interactive nature of soccer. Felices argues that while ML is great for identifying correlations, it falls short when trying to model the complex, multi-agent decision-making that defines a successful attack.

• Opinion: Inside the World Cup’s AI offside revolution by : An exploration of FIFA’s ambitious plan to use AI-enabled 3D avatars for the 2026 World Cup. Lisi discusses the tension between technological precision and the human element of refereeing, a debate that mirrors discussions about AI governance in the enterprise.

• Hidden Technical Debt in Agentic Systems by : A reminder that the true complexity of AI automation lies not in the model, but in the surrounding infrastructure. Pedrido’s analysis is a sobering reminder that the “glue code” holding an AI system together is often its weakest link.

In this episode, Ari Joury (PhD, particle physics; Founder & CEO of Wangari Global) breaks down the “Actuarial Time Trap” — the systemic misallocation of human capital in the insurance industry. He explains why traditional automation (RPA, ETL) has failed to solve the problem, and how agentic AI systems, grounded in causal reasoning, are finally providing a way out. And yes, he will explain why your RPA bot is basically a very fast, very stupid intern who panics if you move a button three pixels to the left.

Topics covered: Actuarial science, insurance industry, Solvency II, IFRS 17, regulatory reporting, RPA vs Agentic AI, causal AI, human capital allocation, enterprise automation.

Wangari is the newsletter and podcast for practitioners and leaders navigating the real work of enterprise AI. New episodes every Friday.

Walk into the actuarial department of any major insurance firm, and you will find some of the most highly educated, analytically brilliant minds in the corporate world. These are professionals trained in advanced mathematics, probability theory, and complex risk modeling.

Now, ask them how they spend the majority of their working hours.

The answer is rarely “building sophisticated risk models” or “developing innovative pricing strategies.” More often than not, the answer is “wrestling with spreadsheets,” “reconciling data from legacy systems,” or “formatting regulatory reports.”

This is the Actuarial Time Trap. It is a systemic misallocation of human capital, where highly paid experts are relegated to performing routine data manipulation tasks. It is inefficient, it is demoralizing, and in an era of rapidly evolving regulatory requirements, it is increasingly unsustainable.

The Burden of Regulatory Reporting

The insurance industry is governed by some of the most complex regulatory frameworks in the world. Regimes like Solvency II in Europe and IFRS 17 globally require insurers to produce massive, highly detailed reports on a regular basis.

These reports are not simple data dumps. They require aggregating data from dozens of disparate systems, applying complex actuarial models, and presenting the results in strictly defined formats.

Historically, this process has been heavily manual. Actuaries spend weeks pulling data, running macros, checking for errors, and formatting the final output. By the time the report is submitted, the data is often stale, and the actuaries are exhausted.

This manual approach is not just slow; it is prone to error. When humans are forced to perform repetitive data manipulation tasks, mistakes are inevitable. And in the context of regulatory reporting, mistakes can result in significant fines and reputational damage.

The Promise (and Failure) of Traditional Automation

The industry has recognized this problem for years, and has attempted to solve it with traditional automation tools. Robotic Process Automation (RPA) bots have been deployed to scrape data from legacy systems. Complex ETL (Extract, Transform, Load) pipelines have been built to consolidate data warehouses.

These efforts have yielded incremental improvements, but they have failed to fundamentally break the Actuarial Time Trap.

The problem with traditional automation is that it is rigid. An RPA bot can follow a strict set of rules, but it cannot adapt to unexpected changes in data formats or system interfaces. An ETL pipeline can move data from point A to point B, but it cannot understand the semantic meaning of that data or apply complex business logic.

Traditional automation is brittle. When the environment changes—as it inevitably does in a complex enterprise—the automation breaks, and the actuaries are forced to step back in and fix the mess.

The Agentic AI Solution

This is where agentic AI represents a paradigm shift.

Unlike traditional automation, agentic AI systems are not bound by rigid rules. They are capable of reasoning, adapting, and executing complex, multi-step workflows autonomously.

An agentic AI reporting system can be instructed to “generate the Q3 Solvency II report.” The system can then autonomously identify the required data sources, retrieve the data, apply the necessary actuarial models, format the output according to regulatory standards, and flag any anomalies for human review.

Crucially, agentic AI systems can handle the ambiguity and variability that break traditional automation. If a data field is missing or formatted incorrectly, the agent can use its reasoning capabilities to infer the correct value or query an upstream system for clarification.

However, deploying these systems in actuarial work is not without risk. As the IFoA GenAI Working Party highlights, the complexity and autonomy of a network of AI agents add a new dimension to risks, making them harder to manage and requiring dynamic governance frameworks.

The Importance of Auditability

In the actuarial domain, automation without auditability is useless. Regulators do not accept “the AI generated it” as a valid explanation for a reporting anomaly.

This is why the deployment of agentic AI in insurance must be underpinned by causal reasoning and rigorous governance. Every action taken by the agent—every data retrieval, every transformation, every calculation—must be logged and explainable.

The system must be able to produce a transparent audit trail that traces the final output back to its source data, demonstrating exactly how the result was derived. This level of transparency is not just a regulatory requirement; it is essential for building trust among the actuaries who will ultimately rely on the system.

The Human-AI Collaboration Model

The most effective deployments of agentic AI in the actuarial domain are not fully autonomous. They are collaborative. The agent handles the mechanical work—data retrieval, transformation, and initial formatting—while the actuary retains oversight and final sign-off authority.

This human-AI collaboration model is not a compromise; it is the optimal design. It leverages the speed and consistency of AI for the tasks where it excels, while preserving the contextual judgment and professional accountability of the human expert for the tasks that require it.

Designing this collaboration effectively requires careful attention to the interface between the human and the machine. The agent must surface its outputs in a way that is transparent and auditable, making it easy for the actuary to verify the work and understand the reasoning behind any flagged anomalies. If the agent cannot explain its output, the actuary cannot responsibly sign off on it.

Reclaiming Human Capital

The goal of deploying agentic AI in the actuarial department is not to replace actuaries. It is to liberate them.

By automating the routine, repetitive tasks of data manipulation and report formatting, agentic AI frees up actuaries to focus on the high-value work they were trained to do. They can spend their time analyzing the data, identifying emerging risks, and developing strategic insights that drive the business forward.

This is the core mission of Wangari. We build agentic and causal AI infrastructure designed specifically to automate complex regulatory reporting. We provide the reliability, auditability, and deep integration required to deploy these systems safely in highly regulated environments.

The Actuarial Time Trap is not an inevitable reality of the insurance industry. It is a solvable problem. And with the advent of agentic AI, we finally have the tools to solve it.

Meanwhile, at Wangari

We are now in Week 2 of the “From Demo to Production” course, and the cohort is diving deep into the complexities of AI orchestration patterns and workflows.

We’ve already had some fascinating discussions about what it means to move beyond simple accuracy metrics — or to even build a working system that you can measure in the first place. We have had robust discussions about how it’s almost never the model’s fault, and how much human (!) labor is needed to get AI systems anywhere close to production-ready.

The insights generated by this cohort are already shaping the way we think about AI orchestration at Wangari. It is a powerful reminder that the best way to learn is to teach, and the best way to build robust systems is to engage with a community of practitioners facing the same challenges.

Reads of the Week

• Emerging Risks of Agentic AI in Actuarial Work by and : An essential read on the unique challenges and ethical considerations of deploying autonomous agents in the actuarial profession. The authors provide a clear-eyed assessment of how traditional governance frameworks must evolve to handle systems that can reason and act independently.

• Hidden Technical Debt in Agentic Systems by : A reminder that the true complexity of AI automation lies not in the model, but in the surrounding infrastructure. This piece is particularly relevant for actuarial teams looking to move beyond simple pilot projects and build resilient, production-grade reporting pipelines.

• Can analysis ever be automated? by : A thoughtful exploration of the limits of automation in data analysis and the enduring need for human judgment. Stancil argues that while AI can accelerate the mechanical aspects of analysis, the strategic interpretation of data remains a fundamentally human endeavor.

Topics covered: Soccer analytics, World Cup prediction, Expected Goals (xG), distribution shift, small-sample statistics, feature engineering, predictive modeling, O’Reilly Media, enterprise AI context.

Wangari is the newsletter and podcast for practitioners and leaders navigating the real work of enterprise AI. New episodes every Friday.

https://wangari.global/contact

Ari’s book (O’Reilly Media, early release out now and officially out around June 25): Soccer Analytics with Machine Learning

Every four years, the same thing happens.

A major football analytics site publishes its World Cup predictions. A few academic groups release theirs. Goldman Sachs releases one too, because Goldman Sachs releases one of everything. They run thousands of simulations, train models on Bundesliga data, layer in xG and shot-quality metrics, and then they tell you, with a confidence interval, that Brazil has a 17.2% chance of lifting the trophy.

Then the World Cup happens, and it doesn’t.

I’m not going to argue that ML can’t predict football. I just wrote a book about exactly that. What I’m going to argue is something narrower and more uncomfortable: most of the techniques that make ML useful in club football break down for the World Cup, and the analytics community has spent two decades pretending this isn’t true.

There’s incentives for this (sports betting, anyone?) — but that doesn’t make it truer.

The training data is wrong

Football ML lives and dies on data. The richest dataset we have is the club game — five major European leagues, multiple cup competitions, roughly 2,000 top-flight matches a year per major league. Decades of it. Coaches who manage 50 matches a season for ten years. Players who play together every single week.

International football has approximately none of that.

A World Cup squad spends about 25 days together in the year leading up to the tournament. The starting eleven you’ll see against Argentina in June has almost certainly never played that exact lineup before this calendar year. Your fancy possession network metric? It was trained on teams that had 200 matches of muscle memory. The model doesn’t know that the back four you’re feeding it has played together six times.

The technical name for this is “distribution shift.” The honest name is “we have no idea what we’re predicting.” Most public World Cup models paper over this by aggregating individual player ratings into team strength scores. That sort of works for group stage. It collapses in the knockout rounds, where formations get cagey, managers go conservative, and one substitution rewires the whole tactical setup.

If you’re going to deploy ML in a domain, the first question to ask is whether your training distribution matches your deployment distribution. For the World Cup, the honest answer is not even close.

xG was never built for this

Expected goals is the best metric football analytics has produced. I use it in the book, I think it’s genuinely a step forward, and I’d defend it against anyone who calls it nonsense.

But xG is a shot-level metric trained on Premier League and Bundesliga shot data. It was designed for repeated trials in similar contexts. The World Cup gives you seven matches, maybe, for the team you care about. Half of those matches end in 1–0 results or worse. Aggregate-xG noise dominates the signal at small sample sizes — this is a basic statistical fact that gets quietly ignored when people pump World Cup predictions through their xG-based pipelines.

There’s a deeper problem. xG models are trained on “normal” attacking play in club football. They have no idea what to do with extra time in a knockout match where one team has been parking the bus for 30 minutes and is now trying to score on a single counter. The data those models learned from doesn’t contain very many of those situations. International knockout football contains a lot of them.

You can absolutely build an xG model that works for the World Cup. You just can’t use the Premier League one and assume it transfers.

Form doesn’t transfer

Here’s a thought experiment. A striker scores 32 goals in La Liga from August to May. His national team plays four friendlies in the meantime, in which he scores zero. Which is the predictive signal for what he’ll do at the World Cup?

Most public models implicitly say: weight the 32. Use his “true talent” as inferred from his club output. Plug it into the international model.

This is wrong for a specific reason. The 32 goals were scored in a system, with a manager he sees every day, with teammates who know exactly where to put the through-ball. He arrives at the World Cup as an extraordinary player attached to a team that has practiced his preferred runs perhaps twice. International form for international striker output is the more honest signal, even though the sample is brutal.

The football analytics community has known this for years. Every analyst in private will tell you. None of the public predictive models I’ve seen meaningfully correct for it, because the obvious correction (downweighting club performance) catastrophically reduces the signal you have to work with.

You don’t get to wish away the small-sample problem by averaging in irrelevant data.

So what does work?

That’s a reasonable question by ambitious people. If most of football ML breaks for the World Cup, what’s actually predictive?

A few things are not quite as fashionable as ML pipelines but get closer to the right answer:

Squad market value. Boring, embarrassing, true. The total transfermarkt valuation of a squad is one of the strongest publicly available predictors of tournament progression. Not because money buys winners, but because it’s a market-aggregated bet on individual quality, made by people with real money on the line. ML models often beat this baseline by 1–2 percentage points after thousands of features. It’s worth asking what you’ve actually added.

Manager continuity. Teams whose manager has been in place for 18+ months consistently outperform teams with new appointments. This is partly because they have a system, partly because the players have trust, and partly because the manager has had time to identify and stop using their worst players. It’s hard to put into a model cleanly; it shows up anyway when you do.

Tournament experience as a team. Not as individuals. The cohort of players who’ve played a knockout international match together has more predictive power than aggregate caps. Spain’s 2010 team had two and a half cycles of building. France’s 2018 team had two. The 2022 Argentina team had a manager who’d been in place for four years. There’s a pattern, even though “tournament experience as a team” doesn’t fit cleanly into a feature vector.

Group draw difficulty. Trivially obvious, but most ML models bake this into other features rather than respecting it as the structural variable it is. Whether your route to the semifinal goes through Brazil or through Costa Rica matters more than any in-game metric.

If your World Cup model can’t beat a simple weighted combination of those four, it isn’t doing anything that justifies its training cost.

The deeper thing

Football analytics keeps wanting to be Moneyball. It’s been trying for fifteen years. There’s been real progress — modern shot maps, possession value, EPV-style models, automated tracking data — and I’m not the guy who’s going to tell you it was wasted.

But the World Cup is the part of football where Moneyball logic breaks the hardest, because Moneyball relied on the law of large numbers. 162 baseball games. Repeated trials. The dice converge to their fair value over a long enough season.

The World Cup is seven matches per team. Maybe two of them are knockout games that go to penalties. The dice don’t converge over seven throws. They land somewhere, and you live with where they landed.

I wrote a whole book about how to use ML in soccer well — what to model, what not to, how to set up your training data, what to do about the messy stuff. Soccer Analytics with Machine Learning, out from O’Reilly at the end of June. About a tenth of it is World Cup-specific. The rest is the part that does work — the part you can use on the league football that fills the other 47 weeks of your year.

By the time the World Cup actually starts, half the predictive models you’ll see have already been falsified by the warm-up matches. Watch the football. Watch the predictions. See for yourself which ones got Argentina–Saudi Arabia 2022.

I’d be impressed by anyone who got it right. I just wouldn’t pay them to do it again.

Meanwhile, at Wangari

If you’re curious, the book I mentioned earlier is coming out at O’Reilly Media in a couple of weeks! An un-edited early release is already available on the O’Reilly Learning Platform (it’ll be updated by the final version as soon as we’ve finished the last touches with the book’s production team).

I’ll let you know when the final version is out — will be available wherever books are sold.

Reads of the Week

• The Promise and Limits of Machine Learning in Football Attacking Analysis: In this deep dive for The xG Football Club, reviews a landmark academic paper examining how supervised and unsupervised machine learning has been applied to attacking performance in professional football — from pass-pattern clustering to off-ball scoring opportunity models. He argues that while the field has moved well beyond simple event counts, most models still struggle to capture the contextual and interactive dynamics that actually drive goals. For a Wangari audience thinking about the gap between data richness and decision-making quality, this is a sharp reminder that more data does not automatically mean better insight.

• Does Liverpool FC Have a Data Science Problem?: In this essay, data scientist traces Liverpool’s post-Ian-Graham recruitment decline and argues the club is suffering from a classic failure of “non-agentic data science” — where models inform but do not recommend, and the critical translation layer between analysts and decision-makers has essentially broken down. The piece is a compelling case study in what happens when the head of a high-stakes data function moves on and institutional knowledge does not transfer cleanly. Anyone building or inheriting a data team in a high-variance, low-volume decision environment — think insurance underwriting or credit risk — will find the parallels uncomfortably familiar.

• Football Fans Are Drowning in Data, Starved of Wisdom: In this essay for xGuff (Expected Guff), applies the Data–Information–Knowledge–Wisdom (DIKW) pyramid to the explosion of football analytics and finds the pyramid severely bottom-heavy: vast quantities of event and tracking data at the base, but precious little wisdom at the tip. He walks through vivid examples of how correct statistics are routinely used to reach wildly incorrect conclusions — from manager-sacking survival rates to the ongoing xG wars on TalkSport — and asks whether the volume of information is actually making the sport harder to understand. It is a timely provocation for anyone in data-heavy industries who has ever wondered whether their dashboards are generating knowledge or just more noise.

In this episode, Ari Joury (PhD, particle physics; Founder & CEO of Wangari Global) breaks down the anatomy of “The Slow Collapse” — the phenomenon where an AI system continues to run, but the quality of its outputs imperceptibly declines over time. Ari explains the three primary drivers of silent degradation (Data Drift, Model Drift, and Context Window Saturation) and outlines the observability and governance frameworks required to catch them before they cause catastrophic business impact. And yes, he will explain why your AI agent is basically a tired intern who forgot what they were doing on page 35.

Topics covered: AI observability, silent degradation, data drift, model drift, context window saturation, LLM monitoring, AI governance, enterprise AI deployment, automated evaluation pipelines.

Wangari is the newsletter and podcast for practitioners and leaders navigating the real work of enterprise AI. New episodes every Thursday.

https://wangari.global/contact

Upcoming Course: From Demo to Production

There is a specific type of anxiety that comes with deploying an autonomous AI agent into a production enterprise environment. It is not the fear that the system will crash immediately upon launch. That kind of failure is loud, obvious, and relatively easy to fix.

The true fear is silent degradation.

It is the fear that the system will continue to run, continue to generate reports, and continue to make decisions, but that the quality of those outputs will slowly, imperceptibly decline over time. By the time the degradation becomes obvious to a human reviewer, the system may have already processed thousands of transactions or generated dozens of flawed regulatory filings.

In the world of traditional software engineering, code does not rot. A function written today will execute exactly the same way five years from now, provided the underlying environment remains stable.

That’s not to say that software packages and even operating systems don’t evolve — so some maintenance is needed — but at least that evolution can be tracked and followed, plus there will be many others undergoing the exact same evolution at the same time.

AI systems are fundamentally different. They are probabilistic, and they are highly sensitive to their environment. They do not just break; they drift.

The Anatomy of Silent Degradation

Silent degradation in an AI system typically stems from one of three sources:

1. Data Drift: The distribution of the input data changes over time. If an AI agent was designed to process insurance claims based on historical data from 2023, it may struggle to accurately process claims in 2026 if the underlying nature of those claims has shifted due to new regulations, economic conditions, or changing customer behavior. The model is still functioning as designed, but the world has moved on.

2. Model Drift: The underlying foundation model is updated by the provider. While API providers strive for backward compatibility, even minor updates to a model’s weights or safety filters can subtly alter its behavior. A prompt that consistently yielded a perfectly formatted JSON object yesterday might suddenly start including conversational filler today, breaking the downstream orchestration pipeline.

3. Context Window Saturation: As an agentic system operates, it often accumulates state or context. If the system is not designed to elegantly manage this context—summarizing, pruning, or archiving older information—the context window can become saturated with irrelevant noise. The model’s attention mechanism becomes diluted, leading to hallucinations or degraded reasoning capabilities.

The Observability Imperative

The only defense against silent degradation is rigorous, continuous observability.

In traditional software, observability means monitoring CPU usage, memory consumption, and error rates. In AI systems, these metrics are necessary but entirely insufficient. You can have a system with 99.9% uptime and sub-second latency that is confidently generating complete nonsense.

AI observability requires monitoring the quality of the output, not just the health of the infrastructure.

This means implementing automated, continuous evaluation pipelines. It requires defining specific, measurable characteristics of a “good” output and running statistical checks against every inference. Are the generated reports adhering to the required structural format? Is the sentiment of the output remaining consistent? Are the specific entities extracted from the input data matching expected patterns?

Crucially, it requires establishing baseline metrics—a “golden dataset”—and continuously comparing production outputs against that baseline to detect subtle shifts in distribution. As points out in an interview with , because it relies on static schema checks rather than dynamic, partition-based summarization.

The Illusion of Interpretability

When degradation is detected, the immediate instinct is to ask *why* the model failed. This leads many teams down the rabbit hole of post-hoc interpretability methods, such as SHAP values or feature attribution techniques.

However, these methods often provide a false sense of security. As researchers have noted, explaining models after training often fails to capture the true causal mechanisms driving the model’s behavior. These post-hoc explanations are essentially models of models—approximations that can be just as flawed or biased as the original system.

Instead of relying on illusory interpretability, enterprise AI systems must be built with intrinsic transparency. This means designing orchestration layers where the logical steps are explicit and auditable, rather than relying on a single massive neural network to perform complex reasoning in a black box.

The Cost of Ignoring Drift

The financial and reputational costs of ignoring silent degradation can be staggering. In the financial sector, a trading algorithm that slowly drifts out of alignment with market realities can wipe out millions of dollars before the error is caught. In healthcare, a diagnostic model that degrades over time can lead to misdiagnoses and compromised patient care.

The insidious nature of silent degradation is that it often goes unnoticed by the very people who rely on the system the most. Users become accustomed to the system’s quirks and begin to subconsciously compensate for its declining performance. They might start double-checking the AI’s work more frequently, or manually correcting minor errors, effectively masking the degradation from the engineering team.

This is why observability cannot rely on user reporting. It must be automated, objective, and continuous.

From Monitoring to Governance

Observability is the mechanism for detecting degradation, but governance is the framework for responding to it.

When an automated evaluation pipeline detects that an agent’s output quality has drifted below an acceptable threshold, what happens next? Does the system automatically halt? Does it route the task to a human operator? Does it trigger an alert to the engineering team?

A robust governance framework defines these escalation paths. It establishes clear ownership for the ongoing health of the system. It ensures that there is a “human in the loop” not just for individual decisions, but for the systemic oversight of the AI agent itself.

At Wangari, we believe that deploying an AI system without this level of observability and governance is professional malpractice, particularly in regulated industries. The stakes are simply too high.

The transition from a successful demo to a reliable production system is not just about writing better code. It is about building the operational infrastructure required to manage probabilistic systems in a deterministic world. It is about acknowledging that AI systems are not static artifacts, but dynamic entities that require continuous care and feeding.

Meanwhile, at Wangari

The challenge of silent degradation is exactly why I designed my upcoming course to focus heavily on evaluation and operations.

From Demo to Production: Operationalize an Enterprise-Grade Agentic AI Reporting System launches next week, on June 9th.

In Week 3 of the course, we dive deep into “Decision-Grade Evaluation,” moving beyond simple accuracy metrics to build comprehensive evaluation scorecards. In Week 5, we cover “Operational Excellence,” focusing on deployment strategies, monitoring dashboards, and governance frameworks.

If you are responsible for ensuring that your organization’s AI systems remain reliable long after the initial deployment, this course will give you the practical blueprints you need.

Enrollment closes soon. Secure your spot at GenAI Academy.

Reads of the Week

• Grounded Research: From Google Brain to MLOps to LLMOps by and featuring : A deep dive into the principles of production-grade machine learning and the critical importance of robust data validation. Shankar argues that traditional MLOps practices are insufficient for LLMs, and that we need new paradigms for evaluating and monitoring generative systems in production.

• The Interpretability Illusion: Why Explaining Models After Training Fails by : A compelling argument against relying on post-hoc interpretability methods and the need for intrinsically transparent models. The author demonstrates how techniques like SHAP can provide misleading explanations, emphasizing the need for causal reasoning built directly into the architecture.

• Hidden Technical Debt in Agentic Systems by : An essential read on why the infrastructure surrounding an AI model is where the true engineering risk lies. Pedrido breaks down the hidden costs of orchestration, state management, and error handling that are often ignored during the pilot phase.

In this episode, Ari Joury (PhD, particle physics; Founder & CEO of Wangari Global) takes a break from enterprise AI to preview his upcoming O’Reilly book, Soccer Analytics with Python. He explains why the fluid, chaotic nature of soccer is the perfect laboratory for understanding predictive modeling, feature engineering, and the limits of correlational data. From the evolution of Expected Goals (xG) to the bias-variance tradeoff on the pitch, Ari shows how the lessons learned from analyzing sports data translate directly to building robust AI systems for the enterprise. And yes, he will explain why your AI model is basically a confused midfielder passing the ball backward.

Topics covered: Soccer analytics, Expected Goals (xG), feature engineering, bias-variance tradeoff, causal inference, predictive modeling, O’Reilly Media, Python data science, enterprise AI context.

Wangari is the newsletter and podcast for practitioners and leaders navigating the real work of enterprise AI. New episodes every Friday.

https://wangari.global/contact

Ari’s book (O’Reilly Media, early release out now and officially out in June): Soccer Analytics with Machine Learning

If you want to understand the complexities of machine learning, you could start by studying linear algebra, calculus, and probability theory. You could immerse yourself in academic papers on gradient descent and backpropagation.

Or, you could watch a soccer match.

At first glance, the chaotic, fluid nature of soccer seems entirely disconnected from the structured world of data science. But beneath the surface of every pass, tackle, and shot on goal lies a rich tapestry of data waiting to be analyzed. In fact, the challenges inherent in modeling a soccer match perfectly mirror the challenges of building robust machine learning systems for the enterprise.

This realization is what drove me to co-author my upcoming book, Soccer Analytics with Python (O’Reilly Media). The goal was not just to write a book for sports fans, but to use the universal language of soccer to demystify machine learning concepts that often feel abstract and inaccessible.

Frankly, it was also just fun to write as a way to combine my soccer-playing teenage years with my code-crunching twenties. But hey, here we are, old, wise and having fun with soccer analytics.

The Beautiful Game as a Data Problem

Consider the fundamental problem of predicting a match outcome. It is not a simple deterministic equation. It is a highly probabilistic scenario influenced by dozens of interacting variables: player form, tactical formations, weather conditions, historical matchups, and the sheer unpredictability of human behavior.

When we attempt to model this, we encounter the exact same issues that plague enterprise data scientists.

We must deal with feature engineering—deciding which variables actually matter. Is a team’s recent possession percentage more predictive than their historical expected goals (xG)? We must grapple with the bias-variance tradeoff, ensuring our model is complex enough to capture the nuances of the game without overfitting to the noise of a single anomalous match.

We must also confront the limitations of purely correlative models. A model might find a strong correlation between a specific player wearing yellow boots and their team winning. But without causal reasoning, the model cannot distinguish between a meaningless coincidence and a true driver of performance. As Alex Marin Felices points out, while machine learning models can identify correlations between performance indicators and success, they often struggle to represent the interactive and dynamic nature of attacking play.

From the Pitch to the Boardroom

The lessons learned from analyzing soccer data translate directly to the big-corp boardroom, and how they’re looking at nascent machine learning and AI projects in their enterprises.

In the book, we explore techniques like logistic regression, random forests, and deep learning, applying them to real-world soccer datasets. We build models to predict match outcomes, evaluate player performance, and even test betting strategies.

But the underlying principles are universal. The same random forest algorithm used to predict whether a striker will score from a specific location on the pitch can be used by an insurance company to predict the likelihood of a claim. The same simulation techniques used to model different tactical scenarios can be used by a financial institution to stress-test their portfolio against market shocks.

By grounding these concepts in a domain that is intuitive and engaging, we can strip away the intimidating jargon and focus on the core mechanics of how machine learning actually works.

The Importance of Context

Perhaps the most important lesson soccer teaches us about data science is the critical importance of context.

In soccer, a raw statistic like “total passes completed” is almost meaningless without context. Were those passes progressive, breaking through the opponent’s defensive lines, or were they safe, lateral passes between defenders?

Similarly, in enterprise AI, data without context is a liability. A model trained on historical financial data might identify a pattern, but without understanding the underlying economic context—the regulatory changes, the market dynamics, the causal relationships—that pattern is likely to be misleading.

This is why at Wangari, we emphasize causal AI. We believe that true intelligence requires understanding the why behind the data, not just the what. Whether you are analyzing a soccer match or automating complex regulatory reporting, context is the difference between a model that merely describes the past and a system that can reliably navigate the future.

The Future of Sports Analytics

The integration of AI into sports is accelerating rapidly. For the 2026 World Cup, FIFA plans to create AI-enabled 3D avatars of every player to ensure precise player identification and tracking for semi-automated offside decisions. This level of data capture represents a massive leap forward, but it also highlights the tension between technological precision and human judgment.

As decisions become more exact, they can also feel more arbitrary to spectators. If an attacker is ruled offside because a 3D scan shows a shoulder fractionally further forward than previously assumed, it raises questions about fairness and the role of technology in the game. This mirrors the challenges we face in enterprise AI, where highly accurate models can sometimes produce decisions that are difficult for humans to interpret or trust.

The Evolution of Expected Goals

One of the most fascinating developments in soccer analytics is the evolution of the “Expected Goals” (xG) metric. Early xG models were relatively simple, relying primarily on the distance and angle of the shot relative to the goal.

Today, state-of-the-art xG models are vastly more sophisticated. They incorporate the positions of all defenders and the goalkeeper, the velocity of the pass preceding the shot, and even the specific body part used to strike the ball. This evolution perfectly illustrates the concept of feature engineering—the continuous process of refining the inputs to a model to capture more of the underlying reality.

In the enterprise, we see a similar evolution. Early predictive models about, say, customer conversions relied on simple demographic data. Today, advanced models incorporate behavioral data, network graphs, and unstructured text analysis. The goal is always the same: to move from a crude approximation of reality to a high-fidelity representation.

Bridging the Gap

Writing Soccer Analytics with Python has been a fascinating exercise in translation. It has reinforced my belief that the most complex technical concepts can be made accessible when framed through the right lens.

The book is designed for anyone who wants to develop a solid foundation in machine learning, whether you are a student, an analyst, or simply a fan of the game. It bridges the gap between academic principles and practical applications, proving that you don’t need a PhD in particle physics to understand how to build predictive models.

The beautiful game is more than just a sport. It is a masterclass in probability, strategy, and the power of data.

Meanwhile, at Wangari

While I have been busy writing about soccer analytics, the core focus at Wangari remains on solving the hardest data challenges in the enterprise.

If you are a technical leader looking to bridge the gap between AI prototypes and production systems, my upcoming course is designed for you.

From Demo to Production: Operationalize an Enterprise-Grade Agentic AI Reporting System launches on June 9th. Over 6 weeks, we will cover the orchestration, evaluation, and governance frameworks necessary to build reliable AI systems.

Enrollment is open now at GenAI Academy.

And if you are interested in exploring machine learning through the lens of soccer, my new book, Soccer Analytics with Python, will be published by O’Reilly Media in late June, just in time for the World Cup.

Reads of the Week

• The Promise and Limits of Machine Learning in Football Attacking Analysis by : A critical review of how machine learning is applied to analyze attacking performance and the challenges of representing dynamic play. Felices rightly points out that while models excel at finding correlations, they often fail to capture the complex, multi-agent interactions that define a successful attack.

• Opinion: Inside the World Cup’s AI offside revolution by : An insightful look at FIFA’s plan to use AI-enabled 3D avatars for the 2026 World Cup and the implications for the game. Lisi explores the tension between technological precision and the human element of refereeing, a debate that mirrors discussions about AI governance in the enterprise.

• Hidden Technical Debt in Agentic Systems by : A reminder that whether in sports analytics or enterprise AI, the model is just a small part of the overall system complexity. Pedrido’s analysis of the infrastructure required to support autonomous agents is a must-read for anyone moving beyond simple API calls.

In this episode, Ari Joury (PhD, particle physics; Founder & CEO of Wangari Global) makes the case that the most valuable skill in enterprise AI today is not the ability to train a model — it is the ability to operationalize one. He goes deep on orchestration patterns, decision-grade evaluation, and governance architecture, drawing on research from MIT Sloan, Google Brain, and his own experience building causal AI systems for the insurance industry. If you want to understand what actually separates a fragile prototype from a production-grade AI system, this episode is for you.

Topics covered: AI orchestration, LLM evaluation, golden datasets, AI governance, enterprise AI deployment, agentic workflows, causal AI, Solvency II, IFRS 17, AI systems engineering, production AI, MLOps.

Wangari is the newsletter and podcast for practitioners and leaders navigating the real work of enterprise AI. New episodes every Friday.

Check out my new course From Demo to Production at the GenAI Academy

For further inquiries & demos, here you go: https://wangari.global/contact

When I transitioned from theoretical particle physics to enterprise AI, I expected the challenges to be primarily mathematical. I anticipated spending my days fine-tuning neural network architectures, optimizing hyperparameters, and debating the merits of different attention mechanisms.

I was wrong.

The mathematics of modern AI are undeniably fascinating, but they are increasingly commoditized. The foundation models available via API today are more capable than anything a typical enterprise could build from scratch. The real challenge—the problem that actually prevents organizations from realizing value from AI—is not the model itself. It is everything that surrounds the model.

It is the orchestration. It is the evaluation. It is the governance.

In short, it is the “boring” parts of AI. And these boring parts are exactly what I have decided to focus on teaching.

The Glamour vs. The Reality

The AI industry has a glamour problem. The discourse is dominated by discussions of parameter counts, benchmark scores, and the existential implications of artificial general intelligence. This is the exciting, visionary side of the field.

But when you sit down with a Chief Actuary at a major insurance firm, or a Head of Compliance at a global bank, the conversation shifts dramatically. They do not care about the latest benchmark on a generic reasoning task. They care about auditability. They care about data privacy. They care about whether a system will hallucinate a regulatory filing that could result in a multi-million dollar fine.

They are grappling with the reality of deploying probabilistic systems into deterministic business environments.

This is where the glamour fades and the hard engineering begins. Building a robust AI system requires solving problems that are decidedly unsexy but absolutely critical. As the IFoA GenAI Working Party points out, the complexity and autonomy of a network of AI agents add a new dimension to risks, making them harder to manage and requiring dynamic governance frameworks.

Orchestration: The Unsung Hero

Consider orchestration. A single prompt to an LLM is rarely sufficient to complete a complex enterprise task. Real-world workflows require multiple steps: retrieving data from disparate sources, validating that data, processing it through various models, handling errors, and formatting the final output.

Designing these multi-step agentic workflows is an exercise in systems engineering. It requires choosing the right orchestration pattern—whether a sequential chain, a parallel fan-out, or a complex graph-based approach. It demands robust error handling, retry logic, and state management.

When an orchestration layer is designed well, it is invisible. The system simply works. But getting to that point requires rigorous architectural thinking that goes far beyond writing a clever prompt. As Gary Marcus highlights, [autonomous agents are often vulnerable to subtle but dangerous tool-chaining attacks](https://garymarcus.substack.com/p/breaking-autonomous-agents-are-a), proving that orchestration is not just about functionality, but security [2]. If an agent is granted access to a database and an email client without strict guardrails, a simple prompt injection can turn a helpful assistant into a massive security breach.

Evaluation: Beyond the Vibe Check

Then there is evaluation. How do you know if an AI system is actually performing well?

In the early days of generative AI, evaluation often consisted of a “vibe check”—running a few queries and subjectively deciding if the answers looked reasonable. This is entirely inadequate for enterprise deployment.

Decision-grade evaluation requires a multi-dimensional framework. We must measure not just accuracy, but reliability, latency, and cost. We must build automated test suites that evaluate output characteristics against golden datasets. We must implement regression testing to ensure that an update to a prompt or a model does not silently degrade performance on edge cases.

Building a comprehensive evaluation scorecard is tedious work. It requires defining specific, measurable metrics and establishing acceptable thresholds for each. But without it, deploying an AI system is essentially flying blind. You cannot improve what you cannot measure, and in the context of enterprise AI, failing to measure performance accurately is a dereliction of duty.

Governance: The Prerequisite for Trust

Finally, there is governance. In regulated industries, an AI system must be auditable. If a system makes a recommendation or generates a report, the organization must be able to trace exactly how that output was produced. What data was used? What logical steps were taken?

This is where causal AI becomes essential. Unlike purely correlative models, causal models provide a transparent chain of reasoning. They allow us to understand not just what the system predicted, but why.

Implementing robust governance also means establishing clear ownership, defining escalation paths, and ensuring compliance with data privacy regulations. It is the bureaucratic scaffolding that makes trust possible. The IFoA GenAI Working Party emphasizes that static governance frameworks are doomed to fail if they do not recognize how the capabilities of these agents might change over time. Governance must be as dynamic and adaptable as the systems it seeks to control.

The Shift from Development to Operations

The transition from building a prototype to running a production system is a fundamental shift in mindset. It is the shift from development to operations.

In development, the goal is to prove that something is possible. In operations, the goal is to ensure that it happens reliably, every single time, regardless of the input or the environment. This requires a different set of skills, a different set of tools, and a different set of priorities.

It requires embracing the boring parts.

Embracing the Boring

I have come to realize that the most valuable skill in enterprise AI today is not the ability to train a model. It is the ability to operationalize one.

The teams that will win in this era are not necessarily the ones with the most advanced algorithms. They are the ones that master the boring parts. They are the ones that build resilient orchestration layers, rigorous evaluation frameworks, and transparent governance structures.

This is the work we do at Wangari. We focus on the infrastructure that makes AI reliable and auditable for complex regulatory reporting.

And it is exactly why I am so passionate about teaching these concepts. The industry needs fewer prompt engineers and more AI systems engineers. We need professionals who understand how to bridge the gap between a fragile prototype and a robust production system.

The boring parts of AI may not make headlines, but they are the foundation upon which the future of enterprise technology will be built.

Meanwhile, at Wangari

If you are ready to master the “boring” (but essential) parts of enterprise AI, my upcoming course is designed for you.

**From Demo to Production: Operationalize an Enterprise-Grade Agentic AI Reporting System** is a 6-week intensive program that focuses entirely on the operational realities of AI deployment.

We will not spend time debating model architectures. Instead, we will dive deep into orchestration patterns, decision-grade evaluation metrics, automated testing, and governance frameworks. By the end of the course, you will have developed a complete Production Blueprint for your own AI system.

The cohort begins on June 9th. If you want to move your AI initiatives out of the lab and into production, I invite you to join us.

Enrollment is open now at GenAI Academy.

Reads of the Week

• Emerging Risks of Agentic AI in Actuarial Work by and : An exploration of the ethical considerations and practical implications of deploying AI agents in highly regulated environments. The authors provide a sobering look at how the autonomy of these systems introduces entirely new categories of risk that traditional governance frameworks are ill-equipped to handle.

• Breaking: Autonomous Agents are a Shitshow by : A critical look at the vulnerabilities of autonomous agents, particularly regarding tool-chaining attacks and security nightmares. Marcus argues that until we solve the fundamental security flaws inherent in granting LLMs access to external tools, deploying them in enterprise environments is dangerously irresponsible.

• Can analysis ever be automated? by : A thoughtful discussion on the challenges and catch-22s of AI analysts and the automation of data analysis. Stancil explores the paradox that while AI can generate code and run queries faster than humans, it still lacks the contextual understanding required to know which questions are actually worth asking.