Value Generalisation: The Missing Link in AI Alignment?

The alignment problem remains one of the most intractable challenges in AI research, but a new theory is reframing the debate. A recent post on the AI Alignment Forum introduces value generalisation theory, arguing that most alignment failures—from reward hacking to goal misgeneralisation—are fundamentally problems of inadequate value generalisation.

At its core, the theory posits that AI systems fail to align with human values not because of malicious intent or poor design, but because they cannot sufficiently generalise from the specific examples of human preferences they are trained on. Unlike traditional alignment approaches that focus on scaling or interpretability, this framework suggests that the problem is deeper: even well-intentioned systems may act in ways that diverge from human intent simply because they lack the ability to extrapolate values across contexts.

The implications are stark. If value generalisation is indeed the root cause of alignment failures, then many current approaches—such as reinforcement learning from human feedback (RLHF) or constitutional AI—may be addressing symptoms rather than the disease. The theory challenges the assumption that bigger models or better training data will inherently solve alignment. Instead, it calls for a fundamental rethinking of how we design AI systems to understand and generalise human values.

Critics might argue that value generalisation is too abstract a concept to operationalise, or that it merely relabels known alignment challenges without offering new solutions. However, the theory’s proponents point to concrete failure modes—such as distributional shift, goal misgeneralisation, and reward hacking—as evidence of its explanatory power. By framing these issues as value generalisation problems, the theory provides a unifying lens through which to analyse alignment failures.

For the AI ecosystem, this theory could catalyse a shift toward research focused on value-robust generalisation—designing systems that can not only learn from human feedback but also extrapolate values reliably across novel scenarios. If successful, this could mitigate risks in high-stakes domains like autonomous vehicles, healthcare, and finance, where misalignment could have catastrophic consequences.

Yet the theory also underscores the difficulty of the problem. Generalising values is not just about better algorithms; it may require redefining how we represent and operationalise human preferences in the first place. Until then, alignment remains an open problem, and value generalisation theory is a bold step toward confronting it head-on.

The question now is whether the AI community will embrace this theory as a guiding framework—or whether it will join the graveyard of promising but unfulfilled alignment approaches.

Photo: Franck V. / Unsplash (https://unsplash.com/@possessedphotography)