A few weeks ago, I had the pleasure of taking the stage at the AI Summit London to take part in the “Beyond Agents” panel. I was joined by some fantastic leaders in the space, including Prachetas Bhatnagar, from the Tony Blair Institute for Global Change, Angharad Williams, from Lloyds Banking Group, Anupama Hatti, from NHS Blood and Transplant, and Kerry Sheehan, from the UK Government. It was esteemed company, people at the leading edge of AI implementation, but, crucially, from organisations in either highly regulated or public sector industries. 

In these spaces, the question isn’t just ‘does AI work?’, it is ‘can we trust it? Can we govern it? Can we scale it?”  The thing is, these should be the burning questions for all organisations, regardless of sector. Normally, it’s the less regulated industries that lead the way. They build fast and break things. That’s fine for the pilot, but when it comes to real production, failure can’t be part of the plan. For regulated sectors, it should never be part of the plan.

AI adoption is not just about giving people access to powerful tools. AI is powerful, but power without governance creates risk, inconsistency, and wasted investment.

Accountability Isn’t Optional

This was a key point of the panel discussion. It centered on a simple but crucial truth: you can’t outsource accountability to an algorithm. 

The way I like to put this into perspective is this: putting AI fully in charge of critical business decisions is like letting a very smart teenager run a house.

They may be capable. Fast. Even right about things more often than expected.

But they do not carry the mortgage. They do not face the legal consequences. They do not live with the long-term impact of every decision.

AI is the same. It can help us think faster, see patterns earlier, and act with more precision. But responsibility cannot be delegated to the system. Humans must remain accountable for the outcome.

This is why observability matters so much. If organisations are serious about scaling AI, they need to understand what systems are doing. Not just what, but when, where, how, and why. During my time at AWS, there was one aspect of the Amazon culture I really enjoyed. I’d always hear leaders saying: “We hire builders and we let them build. : Trust, but verify.” That mindset feels more relevant than ever.

I think what enterprises are really searching for with AI isn’t more intelligence, it’s more confidence. They want to be confident that systems are behaving as they should, that the decisions can be explained, and that there’s an audit trail with no blind spots or siloes.  

Without that visibility, “Shadow AI” becomes inevitable. And adding AI to a broken process doesn’t fix the process; it just creates a broken process on a massive scale.

The FinOps Reality Check

Another theme that came up repeatedly on the panel was the growing need for financial discipline and control around AI. The COO of Uber recently revealed that his company burned through its entire 2026 AI budget in four months due to uncontrolled AI use. This is so rife currently that the industry has coined the term, ‘Tokenmaxxing’, and it has been dominating headlines in recent months. 

We’ve started hearing terms like “AI FinOps” for a reason. Too many organisations are still treating AI as an experimental budget line rather than an operational capability that needs measurable outcomes.

The conversation often focuses on token costs, but the bigger expense is usually elsewhere: cleaning data, maintaining good governance, and having a flexible AI architecture that means matching the right model to the right job. In many cases, this could mean running open-source models on-prem for predictable, high-volume workloads. 

In many ways, the conversation we’re having around AI FinOps is the same one that happened around the cloud a decade ago – the same one that gave birth to the idea of FinOps to begin with. Back then, organisations discovered that unlimited scalability was only valuable if it came with visibility, accountability, and cost control. AI is now reaching a similar inflection point. 

In other words, the conversation is shifting from capability to economics.

And that’s a sign of a market beginning to mature.

The New Strategic Choice

This FinOps question is leading organizations down a different path. And it’s the right path to be on. A year ago, the big question in enterprise AI was: Which model should we choose?

Today, that feels increasingly like the wrong question. The pace of innovation is too fast. Models are constantly improving, and new open-source alternatives are constantly emerging. 

Again, I’m reminded of the cloud maturity journey. As the picture changes so fast, as costs shift, and capabilities evolve, enterprises can’t afford to be locked in to one vendor. More importantly (particularly in the AI age), you can’t afford to be locked out of changing to another model or provider, as the opportunity cost could be just as damaging. 

So, beyond the choice of individual models, the bigger, more important question is whether your architecture allows you to adapt as the landscape changes. If your strategy depends on a single model, vendor, or deployment approach remaining dominant, you’re making a risky bet.

The organisations that will thrive over the next decade won’t necessarily be those with access to the most advanced model. There will be those that retain the flexibility to evolve as the market evolves.

The Question We’re Really Trying to Answer

The more I reflect on the discussion, the more I think we’re asking the wrong question when we focus exclusively on AI capability.

The technology is improving at an extraordinary rate. That much is clear. The harder challenge is building organisations that can absorb that change safely, responsibly, and at scale.

We often assume that wider AI adoption inevitably increases organisational risk. At Jeen, we’re convinced the opposite is true. With the right controls, visibility, and governance in place, scale can actually reduce risk because every new workflow inherits established guardrails rather than creating entirely new vulnerabilities.

That’s the challenge facing enterprises today. Not whether AI is ready. But whether we can build the confidence, control, and operational foundations needed to realise its potential.