People researching AI have flagged that it could exceed the sum of human intelligence within about five years. Organisations at the forefront of AI development have spoken of a transition towards superintelligence, and a future in which powerful AI systems could possess intellectual capabilities matching or exceeding those of Nobel Prize winners. These are predictions, not certainties. But they raise a question healthcare cannot afford to postpone: What happens when the system helping make a healthcare decision may be more capable than the person expected to interpret it? In most industries, this is a question of productivity. In health care, however, it is a question of responsibility where the consequences directly affect the quality of care, the accuracy of clinical decisions and ultimately patient outcomes.
An AI system can eventually become better than a doctor at recognising patterns in an image or better than a researcher at identifying potential drug candidates. But superior analytical capability alone does not give an AI system clinical judgement, professional accountability or responsibility for what happens to a patient.
A common argument against concerns about AI risks is that doctors will retain ultimate decision-making authority. However, this overlooks whether doctors will have sufficient time, access to relevant information, and the technical expertise needed to identify errors in increasingly sophisticated AI systems. As these systems become more complex, the assumption that human oversight alone can ensure safety is becoming harder to sustain.
Healthcare needs an additional layer of expertise: A person who understands both the capabilities and limitations of AI and the clinical significance of its outputs. They would examine the evidence, identify inconsistencies and uncertainties and determine whether the results can be passed on to a treating doctor.
Imagine a future in which every AI system deployed in health care comes with a simple, continuously updated trust passport which is a living, auditable record that makes trust transparent and verifiable. It would tell clinicians, hospitals, regulators and patients what an AI system is designed to do, the populations on which it has been tested, its known limitations and biases, the safeguards protecting patient data, who is accountable for its use and how its performance is continuously monitored.
{{/usCountry}}Imagine a future in which every AI system deployed in health care comes with a simple, continuously updated trust passport which is a living, auditable record that makes trust transparent and verifiable. It would tell clinicians, hospitals, regulators and patients what an AI system is designed to do, the populations on which it has been tested, its known limitations and biases, the safeguards protecting patient data, who is accountable for its use and how its performance is continuously monitored.
{{/usCountry}}Beyond documenting these essentials, the trust passport could record model versions, validation evidence, permitted uses, privacy controls and review requirements. It could capture whether an AI-generated output was accepted, modified, or rejected, by whom it was done and on what grounds. This makes human oversight significantly critical.
In essence, the trust passport would serve as a living record of an AI system’s capabilities, limitations and accountability, helping distinguish between a technically impressive system, and the one that is fit for a specific healthcare application.
Consider cancer care, where AI can help detect the likelihood of getting diagnosed with cancer or interpret radiological images to diagnose specific tumors. However, these insights need to be interpreted in the context of the patient’s history, other investigations, and the consequences of an incorrect prediction.
In HIV care, AI could help identify patterns in treatment response, flag risks or analyse emerging resistance patterns. Before such insights inform a treatment decision, a trained professional must assess their quality, relevance and clinical limitations.
Health care will increasingly need medical professionals who can translate between AI-generated intelligence and clinical judgement. Potential roles include clinical AI validation specialists, healthcare AI interpreters and AI safety officers with domain specific expertise.
The trust passport could help define their competencies, establish when independent review is required and specify the evidence; the sign-off needed before the output reaches the treating doctor. Rather than overburdening health care professionals, AI is likely to create new opportunities focused on governance, oversight, validation and patient safety, helping health care organisations adopt emerging technologies responsibly while maintaining trust and quality of care.
The patient should not have to surrender control of their data simply because AI needs to learn from it. Federated architecture provides the foundation for privacy-preserving data exchange, while federated learning builds on this foundation by enabling AI models to learn from data distributed across multiple institutions without requiring sensitive patient records to be centralised. Together, they allow health care to generate insights from diverse datasets while maintaining patient privacy, data control and trust
Privacy-preserving technologies can reduce these risks, but no single technique is a complete solution. Pseudonymisation replaces direct identifiers such as a patient’s name or hospital number with a code to reduce the direct exposure of personal information which allows healthcare data to be used for research and AI model development while reducing privacy risks, although re-identification may still be possible.
Similarly, tokenisation replaces sensitive patient information with substitute values (tokens) that can be mapped back only through a protected system, so that the original details are not directly exposed. While both can reduce unnecessary exposure of patient information, neither guarantees anonymity. Even if organisations use such technologies to form part of a broader set of technical safeguards and support responsible data processing, it does not replace applicable legal obligations and must still comply with the Digital Personal Data Protection (DPDP) framework.
This approach complements the federated architecture promoted by Ayushman Bharat Digital Mission, which is intended to support interoperable digital health services while enabling health information to be exchanged through appropriate consent and access mechanisms.
All this matters to the trust passport because a model’s performance in one hospital or population pool does not automatically establish that it will perform equally well elsewhere. The passport should record where a system has been validated, the populations and settings covered by the evidence, and any important limitations on its transferability.
India has an opportunity to bring these elements together through a common trust passport framework for health care AI. Such an approach could support collaboration in drug discovery, predictive analysis and personalised care while reducing the need to centralise raw patient records. By connecting privacy safeguards with model validation, qualified human interpretation and auditable accountability, India could explore a distinct national approach to trustworthy health care AI.
The health care sector offers examples of how responsibilities around healthcare AI can be defined. Leading AI engines, for instance, assist with clinical documentation, while clinicians remain responsible for reviewing and approving material before it becomes part of the medical record.
In India, health care providers have been investing in AI applications intended to support clinical and administrative workflows. These examples illustrate the need for a structured, auditable framework that combines evidence, safeguards, expertise and accountability needed for responsible AI in health care. The trust passport can provide the framework with demonstrable evidence and clear responsibilities.
As AI capabilities advance, meaningful accountability is bound to become the next frontier of intervention. It requires data, qualified human judgement and responsibility.
And India is uniquely positioned to move beyond powerful health care AI to establish a model of responsible adoption in which privacy-preserving innovation, verifiable evidence, qualified human interpretation and clinical accountability come together through a common trust passport.
Because, if AI does become more capable than humans, then health care must ensure that accountability advances alongside intelligence.
(The views expressed are personal)
This article is authored by Pradeep Gupta, partner, Deloitte India.