E-Paper
Governing agentic intelligence in regulated financial environments
This article is authored by Sandeep Khuperkar, founder & CEO, Data Science Wizards.
Something fundamental is changing in how intelligent systems relate to the businesses that deploy them, and most conversations about AI adoption have not yet caught up with it.
For the better part of a decade, AI inside financial institutions has worked in a fairly fixed division of labour: A model scores a risk, ranks a recommendation, flags an anomaly, and a human evaluates what to do next. That division of labour is now evolving, not disappearing. Agentic AI increasingly carries out more of the procedural sequence itself gathering information, applying rules, proposing or initiating an action while the human role shifts accordingly: Less time spent on every routine step, more time spent exercising judgement at the points that genuinely need it. The model is taking on more of the execution. The person alongside it is taking on a sharper, more accountable form of oversight. Both roles are becoming more valuable in this shift, not less necessary.
This is a bigger shift than most roadmaps currently treat it as. It is not simply more automation, and it is certainly not the removal of people from the process. It is a change in how intelligence and judgement divide their labour inside a business—and getting that division right, deliberately, is what requires rethinking both the technology that runs these systems and the business arrangements that govern them. The two can no longer be designed separately, which is the conviction this piece is really about.
That this shift is already being recognised well beyond technology circles is worth noting briefly. India’s central bank, for instance, recently moved to draft model risk guidance for financial institutions that explicitly assumes AI systems can act with consequence requiring override capability, documented human oversight, and accountability that sits with the institution regardless of where a model was built or sourced. It is one of several recent signals, alongside emerging frameworks in other markets, that serious institutions are no longer treating agentic AI as an experiment to be governed later. The detail of any one such framework matters less here than what its existence confirms: this shift is real, and it is already underway.
From a pure systems standpoint, an actor needs different guardrails than a tool. A recommendation engine that is occasionally wrong creates rework. An agent that is occasionally wrong has already acted sometimes on a customer, a transaction, or a ledger before anyone can intervene.
The architectural answer to this is not exotic, but it is unforgiving of shortcuts. Systems built so that AI never has direct write access to core systems proposing actions through a governed layer rather than executing them unilaterally make it possible to pause or reverse a capability immediately, without retraining a model or rewriting code under pressure. The override becomes a property of how the system is built, not a feature bolted on after the first incident.
There is an elegant precedent for this in a discipline far older than AI. Indian banking has operated for decades on a simple control principle: The person who initiates a transaction is never the person who approves it. One role proposes, a separate, named role authorises. Applied to agentic AI, the same logic extends naturally the system proposes an action as the maker, and a named human remains the checker, approving or adjusting before anything reaches a customer or system of record. It is a pattern institutions and auditors already trust, simply extended to a new kind of actor and it is exactly what human oversight should mean in practice: a named, accountable role embedded directly in how the system works, carrying real authority over what the system is allowed to do.
Here is where the technology question and the business question become inseparable, and where I think most organisations are still under-prepared.
The traditional shape of a technology engagement has been transactional: a provider builds or licences something, the customer deploys it, and responsibility for what happens next becomes a matter of interpretation. That shape is unworkable for agentic AI, where the system is no longer just informing a decision the business is responsible for it is taking the action itself.
The more durable shape I have come to believe in starts with something almost embarrassingly simple: A clear, jointly authored Statement of Business Purpose, agreed before a single line of the system is built, stating explicitly what the AI is meant to achieve and how that will be judged over time. It turns “is this AI any good” from a subjective argument after the fact into a question both parties already agreed how to answer. Outcome-based accountability follows naturally from there value tied to whether the outcome actually materialises, not merely to whether a system was delivered on schedule.
If the Statement of Business Purpose defines what success means, governance is what makes that commitment trustworthy in practice. And the most reliable place for governance to live is inside the system itself rather than in a document that sits beside it. Call it governance as code: policies expressed directly in how a system is built, so they are enforced continuously rather than reviewed periodically.
This changes what compliant means in practice. Every AI-proposed action, logged with its reasoning and outcome at the moment it happens, becomes both an operational record a team uses to improve the system and an audit trail a regulator or board can request later. When models are sourced jointly or from outside partners, the same discipline extends naturally into how those relationships are structured, with oversight and review rights built in from the start.
None of this requires unusual technology. It requires treating a Statement of Business Purpose, outcome accountability, and governance as code as three parts of one design conversation not three workstreams owned by three different teams that rarely speak to each other until an incident forces them to.
The current generation of agentic systems mostly act within a single workflow one agent, one process, one set of guardrails. That will not hold for long. The next phase already visible on the horizon involves multiple agents coordinating across functions one system verifying a document, another assessing risk, another orchestrating the handoff between them with humans positioned deliberately at the junctions that matter most, rather than at every individual step. When that happens, a Statement of Business Purpose will need to describe how an entire constellation of agents behaves together, not just one model’s job, and governance as code will need to operate at the level of the workflow, not only the individual system.
This also means the human role will keep evolving, not shrinking. Much as risk and compliance became distinct, senior disciplines inside financial institutions over the past few decades, something similar is likely to happen with AI accountability a recognised function, with real seniority attached, responsible for exactly the kind of oversight described here. Institutions and AI partners that start building that muscle now, even informally, will be considerably better placed than those who wait for it to become a job description written under pressure.
None of this is a reason to slow down. It is a reason to make sure the architecture, the engagement model, and the people responsible for both grow in the same direction, at the same time.
The organisations that lead the agentic AI era will not be the ones with the most advanced models. They will be the ones whose leadership stopped treating AI architecture and AI commercial structure as separate conversations, owned by separate functions, on separate timelines.
That is, ultimately, a leadership posture more than a technical one. It asks a CEO or founder to hold two ideas as genuinely complementary, not competing: that a system should be free to act with real autonomy to be useful, and that the business should be able to explain, name, and stand behind every action it takes. Done well, these reinforce each other named accountability is precisely what makes autonomy trustworthy enough to extend further. The organisations that design for both together, rather than treating one as a brake on the other, are the ones agentic AI will actually reward.
To address a regulator’s concerns and mitigate the risks associated with it, while designing the AI/ML models and implementing them for any business purpose, every organisation should have an in-built governance and compliance about the secured behaviour and outcome of AI models working together by way of having an AI operating system or an horizontal system layer with the built in governance, data and cyber security, industry specific configurable guardrails and worldwide approved certifications like ISO 27001, SOC2, ISO42001, HIPPA, GDPR authorising the same.
(The views expressed are personal)
This article is authored by Sandeep Khuperkar, founder & CEO, Data Science Wizards.
