Paper

{
  "raw_xml": "<entry>\n    <id>http://arxiv.org/abs/2602.23239v1</id>\n    <title>Agency and Architectural Limits: Why Optimization-Based Systems Cannot Be Norm-Responsive</title>\n    <updated>2026-02-26T17:16:17Z</updated>\n    <link href='https://arxiv.org/abs/2602.23239v1' rel='alternate' type='text/html'/>\n    <link href='https://arxiv.org/pdf/2602.23239v1' rel='related' title='pdf' type='application/pdf'/>\n    <summary>AI systems are increasingly deployed in high-stakes contexts -- medical diagnosis, legal research, financial analysis -- under the assumption they can be governed by norms. This paper demonstrates that assumption is formally invalid for optimization-based systems, specifically Large Language Models trained via Reinforcement Learning from Human Feedback (RLHF). We establish that genuine agency requires two necessary and jointly sufficient architectural conditions: the capacity to maintain certain boundaries as non-negotiable constraints rather than tradeable weights (Incommensurability), and a non-inferential mechanism capable of suspending processing when those boundaries are threatened (Apophatic Responsiveness). These conditions apply across all normative domains.\n  RLHF-based systems are constitutively incompatible with both conditions. The operations that make optimization powerful -- unifying all values on a scalar metric and always selecting the highest-scoring output -- are precisely the operations that preclude normative governance. This incompatibility is not a correctable training bug awaiting a technical fix; it is a formal constraint inherent to what optimization is. Consequently, documented failure modes - sycophancy, hallucination, and unfaithful reasoning - are not accidents but structural manifestations.\n  Misaligned deployment triggers a second-order risk we term the Convergence Crisis: when humans are forced to verify AI outputs under metric pressure, they degrade from genuine agents into criteria-checking optimizers, eliminating the only component in the system capable of normative accountability. Beyond the incompatibility proof, the paper's primary positive contribution is a substrate-neutral architectural specification defining what any system -- biological, artificial, or institutional -- must satisfy to qualify as an agent rather than a sophisticated instrument.</summary>\n    <category scheme='http://arxiv.org/schemas/atom' term='cs.AI'/>\n    <category scheme='http://arxiv.org/schemas/atom' term='cs.CY'/>\n    <published>2026-02-26T17:16:17Z</published>\n    <arxiv:comment>About 10,500 words in all (including 922 words of literature and 2019 words of Appendices). Under journal review</arxiv:comment>\n    <arxiv:primary_category term='cs.AI'/>\n    <author>\n      <name>Radha Sarma</name>\n    </author>\n  </entry>"
}