Bombay High Court Voids Tax Order Backed by Fictional AI Case Law

MUMBAI – In a significant judgment that underscores the growing pains of integrating artificial intelligence into the legal and administrative spheres, the Bombay High Court has quashed a substantial income tax assessment, admonishing the Assessing Officer for relying on "completely non-existent" case laws generated by an AI system. The ruling in KMG Wires Private Limited v. The National Faceless Assessment Centre serves as a critical judicial warning against the unverified and "blind" use of AI in quasi-judicial proceedings, reinforcing the indispensability of human diligence and verification.

The Division Bench, comprising Justices B.P. Colabawalla and Amit S. Jamsandekar, set aside an assessment order that had inflated the petitioner's taxable income from the returned Rs. 3.09 crores to an assessed Rs. 27.91 crores. The Court found the order was vitiated not only by its reliance on fictional jurisprudence but also by fundamental breaches of procedural fairness and natural justice.

The Core Issue: AI's "Hallucinations" in a Quasi-Judicial Order

The case centered on an assessment for the Assessment Year 2023-24, conducted under the faceless assessment regime governed by Section 144B of the Income Tax Act, 1961. In justifying a significant addition to the assessee's income, the Assessing Officer (AO) cited several judicial precedents. However, upon investigation by the petitioner's counsel, these cited judgments were found to be entirely fabricated—a phenomenon commonly known in the tech world as AI "hallucination," where a large language model confidently presents false information as fact.

The Court did not mince words in its critique of this practice. "The judicial decisions relied upon are completely non-existent," the bench observed. "In other words, there are no such decisions at all which are sought to be relied upon by the assessing officer. It is for the assessing officer to show from where such decisions were fetched."

This lapse formed the crux of the Court's cautionary message to administrative and tax authorities. The bench noted the allure of AI-driven efficiency but stressed the non-negotiable requirement for human oversight:

"In this era of Artificial Intelligence ('AI'), one tends to place much reliance on the results thrown open by the system. However, when one is exercising quasi-judicial functions, it goes without saying that such results [which are thrown open by AI] are not to be blindly relied upon, but the same should be duly cross-verified before using them. Otherwise, mistakes like the present one creep in."

Compounding Procedural Errors

Beyond the glaring issue of AI-generated citations, the High Court found the assessment order to be riddled with other serious procedural deficiencies that violated the principles of natural justice.

The primary addition of Rs. 2.15 crores, related to purchases from Dhanlaxmi Metal Industries, was made on the premise that the said party failed to respond to a notice under Section 133(6) of the Act. The petitioner, KMG Wires, presented evidence to the contrary, demonstrating that Dhanlaxmi Metal Industries had not only replied but had also provided "voluminous details/evidence" confirming the transactions. The Court concluded that the AO had made the addition "in ignorance and without considering the reply filed," a clear failure to apply a judicious mind to the evidence on record.

Furthermore, the assessee argued that other additions, including a "peak balance," were made without any discernible basis or calculation shared with them. The Court validated this grievance, stating, "It is also one of the grievances of the assessee that they are clueless as to how the figures are arrived at, as no basis or working was ever shown to the assessee, nor was any Show Cause Notice issued before making the addition... Even this grievance of the assessee is justified."

The department’s attempt to remedy the situation by passing a rectification order to remove the erroneous case law references was deemed insufficient by the Court, as it did not cure the underlying procedural illegalities and the substantive lack of a fair hearing.

Legal Implications and a Path Forward

The KMG Wires decision is a landmark moment at the intersection of law, technology, and administrative justice. It provides crucial judicial guidance on the responsible adoption of AI tools, particularly in contexts where individual rights and financial liabilities are at stake.

• Reinforcement of Human Accountability : The judgment firmly places the onus of verification on the human user. An AI tool can be an assistant, but the quasi-judicial authority remains the ultimate decision-maker, fully accountable for the accuracy and validity of every component of their order, including cited authorities.

• Due Process in the Digital Age : The ruling highlights that technological advancements like the faceless assessment scheme cannot dilute the fundamental principles of natural justice. An assessee has an indefeasible right to know the case against them, inspect the evidence and precedents being relied upon, and be given a fair opportunity to rebut them. Fictional citations make this impossible.

• A Call for Institutional Safeguards : This incident signals a need for tax authorities and other government bodies to develop clear protocols for the use of AI. These guidelines should mandate cross-verification of all AI-generated outputs, especially legal citations, and provide training to officers on the known limitations and potential pitfalls of these powerful technologies.

In quashing the assessment order, the demand notice, and the consequential penalty notice, the Bombay High Court remanded the matter to the Assessing Officer for a de novo hearing. The Court's directions for the fresh proceedings emphasized procedural rectitude: the AO must issue a new Show Cause Notice clearly detailing the grounds for any proposed additions. Crucially, the bench mandated, "If any decisions are relied upon, then the Petitioner will be put to adequate notice of not less than 7 days, to counter such judgments."

This case will undoubtedly be cited as a cautionary tale for legal professionals, judges, and administrative officers alike, serving as a powerful reminder that in the pursuit of technological efficiency, the foundational pillars of justice—accuracy, fairness, and human judgment—must never be compromised.