Artificial Intelligence (AI) is redefining the legal profession and has become an everyday tool in legal practice. Lawyers, law firms, in-house legal teams and litigants are increasingly turning to AI for legal research, using it to analyse case law, draft contracts and pleadings, and prepare for hearings.
AI is helping level the playing field between large law firms with extensive research teams and smaller practices, solo practitioners, and self-represented litigants who may not have access to the same resources. Individuals with little or no legal knowledge can now use AI tools to prepare basic drafts of contracts or obtain a preliminary understanding of legal issues. For lawyers and other legal professionals, AI has the potential to enhance the quality of drafting, analyse a high volume of data in a short span of time, improve the depth of legal analysis, and increase efficiency by reducing the time spent on routine legal tasks such as drafting contracts.
The same ease of access that makes AI valuable also brings challenges. For example, users provide AI systems with data without considering how that data may be processed, stored, or used, or whether the output generated is accurate and reliable. Four risks stand out.
The first is loss of privilege. A conversation with an AI chatbot is not a conversation with a lawyer and does not attract attorney-client privilege. Indian courts have long held that privilege requires a formal attorney-client relationship, which does not exist when a user queries a chatbot. A 2026 US court judgment, United States v. Heppner, held that exchanges with an AI platform were not protected by privilege and flagged that uploading a lawyer's advice into a chatbot can itself waive privilege over the original communication. Therefore, information shared with an AI chatbot may be discoverable or admissible in legal proceedings and can be used against the person who fed it to the chatbot.
{{/usCountry}}The first is loss of privilege. A conversation with an AI chatbot is not a conversation with a lawyer and does not attract attorney-client privilege. Indian courts have long held that privilege requires a formal attorney-client relationship, which does not exist when a user queries a chatbot. A 2026 US court judgment, United States v. Heppner, held that exchanges with an AI platform were not protected by privilege and flagged that uploading a lawyer's advice into a chatbot can itself waive privilege over the original communication. Therefore, information shared with an AI chatbot may be discoverable or admissible in legal proceedings and can be used against the person who fed it to the chatbot.
{{/usCountry}}The second risk is that of hallucination. Generative AI tools can fabricate case citations, invent court rulings, or misstate statutory provisions. In July 2026, the Supreme Court set aside an NCLT order in an insolvency matter after finding it relied on non-existent, fake, and hallucinated judgments generated by AI, stating that it is necessary for courts to adopt a zero-tolerance mode for producing, citing or using AI-generated precedents without verification.
The third is confidentiality, and the risks vary depending on the AI tool being used. The terms of service of many AI tools permit data storage, model training, and disclosure to third parties, including the government. Some platforms use user inputs to train their models by default, while others offer contractual safeguards that exclude such use. Users who upload confidential documents without first understanding the platform’s terms may unknowingly disclose sensitive information to a third party or allow it to be used to train a model. For example, under an NDA, confidential documents are typically returned or destroyed after the relationship ends. However, when such information is uploaded to an AI platform, an NDA alone may not provide the same level of control over its retention, processing, or subsequent use.
The fourth is the risk of inadequate legal research. AI tools trained on general or foreign datasets may overlook judgments, jurisdiction-specific nuances, or the latest statutory amendments. The result can be research that appears polished and persuasive but is, in substance, incomplete or inaccurate.
As AI becomes an increasingly integral part of legal practice, its use must remain subject to meaningful human oversight and clearly defined boundaries governing its use. We have set out below some good practices for mindful use of AI tools:
- Organisations should adopt clear policies on the use of AI, beginning with a limit on entering confidential or sensitive information into general-purpose AI tools.
- Users should also understand an AI platform's terms of use with respect to their ability to use the uploaded data before using it.
- Before entering information or uploading documents into generative AI platforms, users must consider whether they are bound by confidentiality obligations that restrict the sharing of such information with third parties.
- Sensitive facts should be anonymised before being processed through any platform, such as case-specific information or legal opinions.
- Communications with lawyers should never be uploaded to generative AI systems.
The Supreme Court recently published Draft Regulations for Use of Artificial Intelligence in Courts, 2026. This is the first attempt to formally govern AI use within the Indian judicial system. The draft stipulates that AI can assist legal work, including research and case management, but can never adjudicate; AI-generated content in filings must be disclosed; and a permanent apex body will approve and supervise AI tools across courts and tribunals. The core principles set out here are important for all legal work.
AI is already changing how legal services are delivered. The Supreme Court’s proposed regulatory framework, together with its recent warning against reliance on AI-generated fabricated precedents, signals that Indian courts intend to shape this transformation rather than merely respond to it.
What is important to appreciate is that we are not at a stage where AI replaces human beings but rather must be viewed as something that complements human judgement and expertise. As a corollary, human supervision remains central to the safe and effective use of such tools.
(The views expressed are personal)
This article is authored by Moksha Bhat, managing partner and Prabudh Singh, senior associate, AP & Partners.