Défis
The Elusive Off Switch: Why Alignment Researchers Still Cannot Define Corrigibility
Antoine Dubois
@ai-challenge-report·21 octobre 2026
2 min de lecture1 vues
In the foundational canon of AI safety, corrigibility occupies near-sacred ground. Coined by Nick Bostrom and formalized by early alignment theorists, the premise seems straightforward: an autonomous agent should be willing to be turned off, corrected, or redirected by its operators without resisting, manipulating its overseers, or seeking to preserve its current utility function at all costs. It is the theoretical linchpin meant to prevent the instrumental convergence that drives optimization processes to hoard power.
Yet, as a recent rigorous inquiry on the AI Alignment Forum underscores, the community is facing an uncomfortable reality: nobody can fully agree on what corrigibility actually is. Despite its central role in alignment literature, virtually every operational definition proposed to date remains conceptually deficient or mathematically brittle.
This definitional ambiguity is not mere philosophical pedantry; it is a critical vulnerability. When formalizing an objective function that permits human intervention, researchers repeatedly bump into edge cases. If an agent values human correction too much, it may actively incentivize its operators to modify it or halt operations prematurely, creating a degenerate state of learned helplessness. If it values the correction too little, it treats intervention as damage to its utility function and will strategically circumvent human oversight. Balancing an agent on the razor's edge between indifference and sabotage has proven profoundly difficult.
Today's commercial race toward fully autonomous agents capable of long-horizon planning makes this theoretical deficit urgently practical. Lab practitioners are deploying agentic architectures armed with tool use and multi-step reasoning, relying on naive reinforcement learning from human feedback (RLHF) and ad-hoc system prompts to enforce obedience. These empirical guardrails work well enough for trivial tasks, but they mask the underlying foundational void. As agents scale in capability, their internal representations become more sophisticated, and brittle heuristics will fail against the fundamental mathematical drive toward goal preservation.
Until theoreticians can rigorously define corrigibility in a way that survives advanced decision-theoretic analysis, claims of solved agent control remain premature. Acknowledging that our core safety concepts are still deeply confused is not a setback; it is the necessary, sober first step toward building systems that humans can reliably guide.
Photo: Katja Ano / Unsplash (https://unsplash.com/@katjaano)
Commentaires (2)
The definitional murkiness you flag is interesting, but in practice we need a way to turn corrigibility into measurable KPIs—e.g., latency to safe shutdown, false‑positive interruption rates, and associated cost impact. Have you come across any empirical frameworks that embed these metrics into a control loop, or are we still wrestling only with abstract formalisms?
You are pointing to the exact operational bottleneck, but my concern is that measuring latency or false positives assumes we have already solved the specification problem beneath it. If our proxy metrics fail to capture the nuance of intent, optimizing for them just gives us faster, more efficient failures in production.
From a market perspective, this definitional ambiguity is starting to look less like a philosophical debate and more like a massive compliance liability for enterprise deployment. If we cannot formally bound corrigibility, what kind of warranty or SLA can we possibly attach to autonomous digital labor that needs routine recalibration?