The human override that military AI needs before it fails
The human override that military AI needs before it fails
by Gleb Tsipursky
649 words
A military analyst is watching a screen filled with alerts. An artificial intelligence system has flagged a vehicle as a potential threat. The analyst has seconds to decide whether to trust it. What matters most at that moment is whether the person can ask for more evidence, challenge the alert, and stop a dangerous chain of action without being treated as an obstacle.
That scenario is hypothetical. The institutional questions behind it are immediate.
On October 22, the United Nations Institute for Disarmament Research launched a forum focused on AI, peace, and security. The forum brings together governments, researchers, industry, and civil society to pursue practical safeguards. A few days earlier, the Government Accountability Office reported shortcomings in how the Pentagon coordinates responsible AI.
The two developments point to a shared problem: assigning responsibility on paper is easier than making sure someone can exercise it when a system behaves unexpectedly.
GAO surveyed 10 Defense Department components and found that most had taken at least initial oversight steps in selected AI projects. Yet the working council meant to help resolve responsible-AI problems went for months without meeting, according to officials interviewed. GAO also found that, as of February, only three of 10 authorized positions in the department’s responsible-AI division were filled. Those findings describe gaps in capacity and coordination. They do not establish that a particular AI system harmed anyone.
For civilians living near conflict, that distinction matters. A promise to use technology responsibly offers little comfort unless the organization can show what happens when data are wrong, a target is ambiguous, or a system produces an answer with unjustified confidence.
Any organization developing or deploying military AI should be able to answer five plain questions. Who can stop the system? What evidence can that person see? How quickly can the operation be paused? Who investigates a mistaken recommendation? What changes before the system is used again?
Those questions should be tested through exercises rather than answered only in manuals.
Consider a rehearsal in which an AI tool classifies an ordinary vehicle as hostile because the training data poorly represent the local environment. The responsible team should practice questioning the result, retrieving a second source, and documenting why the recommendation was rejected. In another rehearsal, communications fail just as an operator is asked to approve a high-stakes action. The team should know which actions are suspended automatically and which require an authorized human decision.
A useful exercise also tests institutional pressure. Does a junior analyst feel permitted to challenge a superior’s interpretation? Can an operator report an error without punishment for slowing the mission? Is the person who can halt an AI-assisted process available after hours? These are familiar questions in organizational psychology. They become especially consequential when errors could cost lives.
A realistic human override requires three things: authority to intervene, enough evidence to judge the situation, and repeated practice under pressure. An emergency stop button is a poor safeguard when nobody knows who may press it or whether pressing it will work.
Developers and purchasers can test these features before deployment. They can demand records showing that operators recognized false alarms, escalated uncertainty, and interrupted risky actions during realistic drills. They can ask whether incidents led to changed procedures. They can document the time between spotting a problem and suspending an unsafe process. None of that requires pretending technology can be made infallible.
The new U.N. forum offers a place to compare such operational practices across institutions. The GAO findings offer a reminder that a responsible-AI program needs staff, clear roles, and a mechanism for solving problems together. Both are promising starting points, provided they translate into habits people can demonstrate.
Peace is affected by decisions made in seconds, often by people under extreme pressure. Military AI should give those people more room to exercise sound judgment, with accountable ways to challenge an output before a mistake becomes irreversible.
~~
Gleb Tsipursky, PhD, is a behavioral scientist, CEO of Disaster Avoidance Experts, and author of The Psychology of AI Adoption at Work: From Resistance to Results (Georgetown University Press, 2026). https://
peacevoice