Who bears liability when government AI systems cause citizen hardship in India's new digital mission?

The push toward a digitized state is gathering unprecedented momentum. Recently, the Indian government approved a substantial allocation of ₹10,371.92 crore for its national Artificial Intelligence (AI) Mission. This move is designed to fundamentally augment the nation’s AI infrastructure and reinvigorate state capacity, aiming to place the country at the forefront of the global technological revolution. Yet, lurking beneath the ambitious metrics of infrastructure growth and computing power is a structural void: the complete absence of a legal framework to compensate citizens when these automated systems inevitably fail. While billions are earmarked for innovation, the current administrative architecture lacks even a single rupee dedicated to identifying, resolving, or compensating for the harms caused by algorithmic errors.

The Reality of the Black-Box Phenomenon

At the core of the dilemma lies the "black-box" phenomenon—a state of affairs where the input data of a system and the resulting verdict are visible, but the logic—or reasoning—connecting the two is intentionally or technically obscured. When a citizen interacts with a state-run digital portal, they are rarely provided with a transparent explanation of how the system arrived at a decision regarding their eligibility, identity, or welfare status.

As observers note, “The department administering the scheme has never explained how the PPP database concluded that he was dead.” This lack of transparency is not merely a technical limitation; it is a profound legal barrier. When the State adopts automated tools, it inherits the fiduciary and constitutional duty to ensure that its actions are not arbitrary. However, the use of black-box algorithms often insulates the administration from accountability, as the complexity of the code serves as a convenient shield against scrutiny.

Citizen Hardship: The Cost of Automation

The abstract projections of the AI Mission stand in stark contrast to the lived realities of individuals who find themselves on the wrong side of a machine-generated decision. Take the case of an individual known as Chand, who suffered the devastating consequence of such an error: losing six months of pension eligibility because an automated database incorrectly flagged him as deceased.

In this incident, the system’s error was absolute, yet the remedy was non-existent. Over the course of six months, the individual—whose livelihood was dependent on state support—was left in a administrative vacuum. There was no clear channel for redress, no explanation of the criteria that triggered the error, and certainly no state-led mechanism to offer compensation for the period of neglect. Episodes like these clarify a chilling reality: while the government is eager to scale up technological efficiency, it is failing to scale up its mechanisms for justice and administrative accountability.

The Question of State Liability

If the State persists in building systems that wield influence over citizen's lives without sufficient mechanisms for review, the question of liability becomes paramount. Who is liable to pay them back when the machine errs? Currently, the law remains murky. In traditional administrative law, redressal is predicated on human decision-making that can be reviewed through internal departmental appeals or judicial intervention. However, when the decision is purely algorithmic, the ability of the judiciary to conduct a meaningful review is severely hampered by the obfuscation of the system’s internal calculations.

The legal community must grapple with the fact that these technological systems are functioning as quasi-judicial authorities. They verify identity, determine access to resources, and categorize individuals in ways that have immediate legal weight. If the State is the deployment vehicle for these tools, the principle of Respondeat Superior —or even broader concepts of state liability in tort—must be re-evaluated to hold the administration accountable for the harmful outputs of its automated agents.

Broader Legal Implications for the State

The lack of transparency in automated governance invites a challenge to the principles of Natural Justice. In any legal system, a participant in a government-run program is entitled to understand the grounds upon which an adverse decision is made. Without this, the administrative process becomes an exercise in arbitrary authority. The "black box" is essentially an opaque decision-making process, which by design excludes the victim from a fair hearing.

Furthermore, these systems threaten to erode the trust between the citizen and the state. If the AI Mission continues to prioritize infrastructure over ethical oversight, the legal system will likely see an uptick in litigation regarding automated discrimination and systemic failure. For legal professionals, this represents a shifting landscape where traditional notions of discovery and evidence are challenged. How does one cross-examine an algorithm? How does one compel a government department to disclose the source code or the nature of training data that has stripped a citizen of their constitutional right to social security?

Impact on Legal Practice and Professional Duty

For the legal practice, the rise of AI-driven administration presents a twofold challenge. First, there is the immediate need for effective administrative law advocacy to force departments to justify machine-generated outputs. Practitioners may need to pivot towards "algorithmic auditing" within the courtroom, demanding that the state demystify its internal data processes during judicial reviews.

Second, the legal system must push for legislative intervention. There is a pressing need for a comprehensive framework that mandates "explainable AI" (XAI) in all public-facing schemes. Until the government mandates that any system controlling the distribution of rights must be able to articulate its reasoning in human language, the risk of systemic injustice will remain. Legal professionals have a critical role to play in lobbying for the appointment of human-in-the-loop oversight committees and statutory grievance redressal mechanisms before a single new technology is implemented.

Conclusion

The government’s AI Mission, while theoretically intended to uplift state capacity, cannot be viewed merely as an engineering project. It is, at its essence, an administrative transformation that requires a corresponding legal transformation. As the state moves toward digital automation, it must accept the inherent responsibility that comes with that power. If the mission survives purely as an unaccountable digital architecture, it risks creating a society where citizens are perpetually at the mercy of inaccessible data points. Investing ₹10,371.92 crore in machinery without investing a single digit in accountability is not merely an oversight—it is a dangerous failure of policy that the legal community must confront with urgency and precision.