Introduction

The United States federal government is rapidly expanding its use of artificial intelligence (AI). According to the Government Accountability Office, the overall number of AI applications in federal agencies doubled between 2023 and 2024, and the deployment of generative AI tools surged nine times during the same period. This acceleration promises greater efficiency in public services but also raises serious concerns about systematic errors that could affect millions of citizens.

Body

One striking example of AI‑related risk emerged in Arkansas. In 2016, the state’s Department of Health Services contracted a software firm to build an algorithm for assessing the needs of disabled Medicaid recipients. The system produced harmful outcomes: it reduced home‑care benefits for an amputee who had no foot problems and generally disadvantaged nearly half of the state’s Medicaid participants. Similar algorithmic failures have been reported in Idaho, Texas, Michigan, Wisconsin, and even at the Department of Homeland Security, illustrating a pattern of unintended consequences when AI replaces human judgment without sufficient safeguards.

Legal scholars have long grappled with the “arbitrary and capricious” standard that courts apply to agency actions under the Administrative Procedure Act. The rise of AI complicates this standard because decision‑making processes often become a “black box.” Even when agencies request explanations from AI models, the provided rationale may not reflect the true factors influencing the outcome. Judges therefore face a daunting task: how to assess the reasonableness of decisions when the underlying logic is hidden or incomprehensible?

Adding to the complexity, the Trump administration reportedly signaled an intention to integrate AI into the rulemaking process. This move could embed AI‑generated information directly into regulatory decisions, further amplifying the need for transparent oversight mechanisms. Without clear procedural guidelines, agencies risk relying on opaque models that can perpetuate bias or produce erroneous results, undermining public confidence.

The Internal Revenue Service (IRS) illustrates another dimension of the challenge. While the agency is expanding its AI capabilities to improve tax processing and enforcement, recent reports indicate that the IRS is not yet prepared to manage the technology effectively. This gap between ambition and readiness mirrors the broader federal trend: rapid adoption outpacing governance.

Conclusion

The surge in federal AI adoption offers undeniable benefits, yet the accompanying rise in systematic error risk cannot be ignored. High‑profile failures, legal uncertainties, and readiness gaps across agencies underscore the urgent need for robust oversight, transparent model documentation, and rigorous testing before AI tools influence public policy. By establishing clear standards and ensuring accountability, the government can harness AI’s potential while protecting the citizens it serves.