As AI gains autonomy, enterprises must retain authority

This article is authored by Ashish Chandra, CEO & founder, GFF AI.

A company would rarely give a new employee permission to change supplier details, approve payments and settle disputes simply because the employee performed brilliantly in an interview. It would define a role, set limits and establish who answers for the consequences. Enterprises should bring the same discipline to Artificial Intelligence (AI) as they begin allowing it to act on their behalf.

Artificial Intelligence

The appeal of autonomous AI is easy to understand. A system that can pursue a task across several applications could remove hours of coordination: collecting documents, checking an order, arranging a delivery or resolving a routine query. Agentic AI extends the familiar interface of questions and answers into sequences of actions. Each additional permission, however, creates another point at which a machine’s interpretation can become an organisation’s commitment.

This makes autonomy a question of institutional authority. A system may be capable of negotiating a discount without being authorised to change a contract. It may correctly identify a customer’s request while lacking permission to disclose the records needed to fulfil it. Competence and permission must be evaluated separately, even when the technology makes combining them effortless.

Consider a hypothetical procurement agent asked to keep a factory supplied at the lowest possible cost. It finds a cheaper vendor, adjusts an order and expedites payment. Each step could appear sensible against its objective. Yet the vendor may require an additional quality check, the revised quantity may exceed storage capacity, or the payment may bypass an established approval. The agent could meet its assigned target while creating costs elsewhere in the business.

Management, therefore, has to specify what a successful outcome includes. Price, service quality and acceptable exceptions cannot remain assumptions that experienced employees are expected to understand. Delegating a poorly defined objective to a faster system gives that ambiguity greater reach.

A natural response is to promise human oversight. But oversight needs an operating design. An employee presented with hundreds of approval requests, little context and a demanding productivity target may have scant opportunity to challenge the system. A person’s presence in a workflow offers limited protection unless that person has the information, time and authority to intervene.

Review should concentrate on consequential transitions. Comparing suppliers may proceed automatically; committing to an unfamiliar supplier may require approval. Preparing a payment and releasing it should carry different permissions. Changing a customer’s contact details should be treated differently from changing who can access the account. The boundary should reflect the possible harm and the difficulty of correcting it.

Engineering guidance already recognises the need for limits. Anthropic’s discussion of effective agents describes human checkpoints and stopping conditions, and warns that errors can compound as agents operate. For enterprise leaders, the implication is that permission to continue should depend on what the system encounters, rather than merely on permission granted when a task began.

Those boundaries need enforcement in the applications through which the AI acts. An instruction to respect a spending limit should be backed by a payment system that rejects an unauthorised transaction. Access should be confined to the task, with limits on the total activity an agent can initiate. Otherwise, many individually permissible actions could accumulate into an exposure nobody intended to approve.

Authority must also have an identifiable owner. Someone in the business should be responsible for approving the agent’s remit, reviewing its performance and withdrawing its access when necessary. NIST’s AI Risk Management Framework emphasises documented responsibilities and executive accountability. An enterprise putting agents into operation should be able to identify the person who can suspend a failing process before another customer is affected.

Accountability also requires evidence. The organisation should be able to reconstruct which records informed an action, which permissions applied and what changed as a result. A fluent explanation generated afterwards is insufficient on its own. Operational records must allow an independent reviewer to establish what happened. Recovery deserves equal attention: stopping further actions does not automatically reverse a payment, retrieve disclosed information or repair a broken commitment.

For Indian businesses, the customer’s ability to seek correction should be part of this design from the outset. Imagine a small supplier trying to challenge an automated payment hold, or a customer whose request has been misunderstood across languages. A company may count the interaction as efficiently processed while the person affected remains trapped in it. Every consequential automated process should offer a clear route to someone empowered to reconsider the outcome.

This has implications for how productivity is measured. Tasks completed and minutes saved tell only part of the story. Leaders should also examine avoidable rework, mistaken refusals, unresolved complaints and the effort customers must expend to correct an error. A process that looks cheaper inside the enterprise may simply have transferred its administrative burden to everyone outside it.

There is a legitimate concern that elaborate controls could consume the benefits of automation. The answer is proportionate delegation. Begin with a bounded task whose outcomes can be checked and whose mistakes can be contained. Extend authority when evidence supports it, and reassess that authority when the task, data or operating environment changes. Routine activity should flow freely within established limits; consequential exceptions should receive attention.

The responsibility begins with leadership. Boards and executives choosing autonomous systems are also choosing how much discretion to delegate, to whom and on what terms. They must be able to explain those choices to the people who bear their consequences.

An enterprise earns the freedom to automate more when it can demonstrate that its authority remains intact. When a customer asks why an AI system acted against their interests, the organisation should have both an answer and a way to put things right.

(The views expressed are personal)

This article is authored by Ashish Chandra, CEO & founder, GFF AI.