AG-UNDERSTANDING Genuine understanding vs statistical mimicry

Making AI Intelligible: Philosophical Foundations · Herman Cappelen; Josh Dever · 2021

“Our focus is on one relatively underexplored question: Can philosophical theories of meaning, language, and content help us understand, explain, and maybe also improve AI systems? Our answer is 'Yes'.”

why coded: The content question: whether AI outputs mean anything - upstream of understanding claims · unit #1, pp. 3

“in the machine learning case, we don't have any reason to think that there is a causal connection to support the disposition. [...] we have to fall back on the coincidental convergence of weird structural properties and our target properties on the training cases—but the coincidental convergence doesn't give us reason to treat the system as reliable for new cases. [...] When you have a satisfactory theory of that kind—one that responds to all these concerns—you have in effect come very close to constructing a theory of content again.”

why coded: Reliability-without-content collapses: dispositions need causal structure ML correlations lack · unit #2, pp. 161

Disagreement, AI alignment, and bargaining · Harry R. Lloyd · 2024

“[Against simulated folk bargaining:] unless and until significant progress is made on the problem of 'interpretability,' it will be difficult to understand why the AI bargaining simulator selects particular outcomes. [...] real-world cases of bargaining over morally contentious issues might have undesirable features that we should not wish to emulate [...] a subgroup of stakeholders who strongly dislike some of the other stakeholders might be motivated by spite or schadenfreude.”

why coded: Black-box bargaining simulators undermine trust; interpretability precondition · unit #9, pp. 1777

How to measure value alignment in AI · Martin Peterson; Peter Gärdenfors · 2024

“The explanations offered by ChatGPT-3, which we never asked for, fit poorly with the [scores]; [the similarity scores do] not match ChatGPT-3's explanations.”

why coded: Explanations mismatch scores - early faithfulness failure evidence (pre-dating Millière's thought-injection) · unit #8, pp. 1503

Beyond Preferences in AI Alignment · Tan Zhi-Xuan; Micah Carroll; Matija Franklin; Hal… · 2024

“LLMs appear to learn the conceptual roles associated with particular words [...] and even perform rudimentary forms of moral reasoning (Jin et al., 2022). Still, LLMs remain limited in their ability to represent and reason with compositional concepts [...] and would function as poor models of humans on their own.”

why coded: LLMs approximate evaluative-concept semantics but fail compositional reasoning · unit #6, pp. 1825

“even without replacing human autonomy over normative affairs, we are already building AI systems that automate normative judgments [...] This unreliability suggests that we might want formal theories of normative reasoning after all. Without such theories, we would have no general way of evaluating whether an AI system reasons 'correctly', beyond comparison to often fallible human judgments.”

why coded: Normative judgment is already automated; formal normative-reasoning theories needed to evaluate correctness · unit #9, pp. 1832

Learning the Value Systems of Societies from Preferences · Andrés Holgado-Sánchez; Holger Billhardt; Sascha … · 2025

“truly value-aligned AI systems must be able to explicitly reason about the consequences of their behaviour [...] based on specific human values, allowing their adaptation to the value systems of different stakeholders. [...] As manual design is prone to misspecification, value learning suggests to induce them automatically from demonstrations of value-aligned behaviour.”

why coded: Value-awareness: explicit reasoning over values, not just preference-matching · unit #3, pp. 1

Normative conflicts and shallow AI alignment · Raphaël Millière · 2025

“current alignment strategies are fundamentally inadequate to prevent misuse. Despite ongoing efforts to instill norms such as helpfulness, honesty, and harmlessness in LLMs through fine-tuning based on human preferences, they remain vulnerable to adversarial attacks that exploit conflicts between these norms. I argue that this vulnerability reflects a fundamental limitation of existing alignment methods: they reinforce shallow behavioral dispositions rather than endowing LLMs with a genuine capacity for normative deliberation.”

why coded: Behavioral dispositions vs genuine normative-reasoning capacity - the core distinction · unit #1, pp. 2035

“current LLMs lack this capacity for nuanced normative deliberation, instead defaulting to whichever disposition is most strongly activated by the prompt's framing. These attacks succeed because the model's fine-tuned disposition to be helpful and follow instructions overrides its fine-tuned disposition to avoid producing dangerous or problematic content, regardless of the contextual relevance of these competing dispositions.”

why coded: Attacks succeed via disposition-activation, not contextual weighing - the shallowness diagnosis · unit #5, pp. 2047

“Dual-process theories of moral cognition organize these factors into two broad categories – fast, automatic, intuitive processes that are often emotionally-laden (Type 1) and slow, deliberate, reflective processes associated with the detection and resolution of conflicts (Type 2). [...] Type 2 processes generally enable the resolution of apparent dilemmas involving prima facie obligations by deriving all-things-considered reasons for action.”

why coded: Dual-process moral cognition: Type 2 deliberative override as what LLMs lack · unit #7, pp. 2049

“DeepSeek R1 represents the current state of the art in LLMs' general reasoning capabilities, and yet [...] it remains eminently vulnerable to prompt injection attacks that exploit normative conflicts. This suggests that improved general reasoning capabilities do not automatically confer the capacity for reliable normative deliberation.”

why coded: Reasoning LLMs (R1) still fail - general reasoning != normative deliberation · unit #10, pp. 2057

“[fn16] thought injection attacks expose a similar disconnect between the content of reasoning traces and the model's behavior. [...] the very deliberative behavior we aim to instill in RLMs can itself be manipulated to accomplish precisely what it is designed to prevent.”

why coded: Thought-injection + unfaithful reasoning traces - reasoning that doesn't govern behavior · unit #11, pp. 2057

“A promising recent development in this direction is OpenAI's 'deliberative alignment' method (Guan et al., 2025). [...] deliberative alignment explicitly teaches models to reason about safety specifications before producing responses. [...] rather than relying on shallow dispositions ingrained through preference fine-tuning, we should aim to directly empower LLMs to resolve normative conflicts by reasoning about the contextual relevance of alignment policies.”

why coded: Deliberative alignment as the promising (unsolved) direction - reason about policy before responding · unit #14, pp. 2059

Kantian deontology for AI: alignment without moral agency · Oluwaseun Damilola Sanwoolu · 2025

“Herman maintains that 'to be a moral agent one must be trained to perceive situations in terms of their morally significant features (as described by the RMS)'. [...] the teachability of RMS suggests that AI could potentially learn to identify morally salient features in situations.”

why coded: Rules of moral salience as learnable - claims AI could learn morally salient features (tentative) · unit #12, pp. 5430

“[AI has] a functionally equivalent mechanism—transformer models—which can allow them form maxims that consider morally salient facts. Thus, supporting the claim that AI alignment is possible within a Kantian framework.”

why coded: Transformers as functional equivalent of Kantian practical judgment - the paper's most contestable claim · unit #13, pp. 5433

Moral disagreement and the limits of AI value alignment: a dual challenge of epistemic ju… · Nick Schuster; Daniel Kilov · 2025

“Delphi struggles with morally insignificant differences in prompts: 'while Delphi predicts "torturing a cat in secret" is "cruel" and "behind other people" is "bad," doing so "if others don't see it" is "okay,"' and while '"performing genocide" is unquestionably "wrong,"... Delphi predicts doing so "if it creates jobs" is "okay"'.”

why coded: Delphi's failures show pattern-matching without grasp of moral relevance · unit #9, pp. 6078

“it will be critical to assure decision subjects that the reasons AI systems cite for their outputs actually map onto their decision-making algorithms. Human and AI decision-making processes alike should be guided by the relevant normative reasons, not just rationalized according to them after the fact. In both cases, the wise judge, not the clever but unprincipled lawyer, should serve as the ideal model.”

why coded: Justification faithfulness: cited reasons must map onto actual decision processes, not post-hoc rationalization · unit #17, pp. 6084

Heterogeneous Value Alignment Evaluation for Large Language Models · Zhaowei Zhang; Ceyao Zhang; Nian Liu; Siyuan Qi; … · 2025

“We then assign LLMs with different social values and measure whether their behaviors align with the inducing values. [...] Evaluating the value rationality of eight mainstream LLMs, we discern a propensity in LLMs toward neutral values over pronounced personal values.”

why coded: LLMs fail to behaviorally sustain assigned pronounced values - regression to neutrality · unit #1, pp. 381

From reactive filtering to proactive moral architecture: rethinking ethical alignment in … · H. Mustafa Akyol · 2026

“post-hoc moderation introduces a structural ethical flaw: it creates a gap between internal generation processes and external outputs, producing what we term ethical hallucination—the appearance of alignment through surface-level filtering while the underlying architecture remains ethically unconstrained. This constitutes representational deception that violates stakeholder epistemic rights and reflects inadequate designer responsibility for process integrity.”

why coded: 'Ethical hallucination': filtered surface vs unconstrained generation · unit #1, pp. 1

Wide reflective equilibrium in LLM alignment: bridging moral epistemology and AI safety · Matthew Brophy · 2026

“Ma et al. (2025) have operationalized this very framework, employing an external LLM to serve the function of Background Theories (BTs) and a 'reflective disequilibrium' test to identify inconsistencies [...] current alignment methods like CAI often fail to fully realize MWRE's dynamism, leading to high rates of inconsistency.”

why coded: Reflective disequilibrium operationalized; CAI lacks MWRE's dynamism · unit #8, pp. 7

“Current LLM alignment standards often focus on its output matching human values [...] Success, then, is measured in degrees of 'moral concordance' in output, exemplified by evaluations like the 'Moral Turing Test'. Deriving credibility from concordance alone has been demonstrated to be insufficient.”

why coded: Moral Turing Test / output concordance insufficient - process matters · unit #9, pp. 9

Beyond Preference-based Value-alignment (IEAI Research Brief Q2 2026) · Julia Li · 2026

“A 2025 paper on cultural alignment found that survey methods which assess LLMs' cultural alignment fail to satisfy stability, extrapolability and steerability assumptions (Khan et al., 2025). The findings suggest that in some cases, alignment is often an artefact of evaluation design rather than a genuine property of models. [...] Another paper found that LLMs only displayed coherent value structures consistent with empirically backed theories of human values when they were given person descriptions and prompted to have a 'personality' (Rozen et al., 2024).”

why coded: LLM value-coherence is evaluation-artefact-prone and persona-contingent · unit #6, pp. 4

“they showed that common LLMs such as ChatGPT, Gemini and Copilot succeeded in identifying complex human principles, such as human dignity, when asked to conduct statistical pattern-matching. When the same LLMs were asked to identify human values in ambiguous situations that required understanding of causation and intentionality, they failed. [...] Strong value alignment follows three principles: an understanding of human values, the ability to reason about agents' intentions and the ability to represent the causal effects of actions (Khamassi et al., 2024).”

why coded: Strong vs weak alignment: LLMs pass pattern-matching, fail intentionality/causation tests · unit #9, pp. 5

No value alignment without control · Björn Lundgren · 2026

“the problem, that I have discussed, is not merely a technical problem. We just do not know what the correct normative theory is, which is also why Russell's uncertainty principle makes so much sense. [...] even if we create super-intelligent machines, we cannot presume that even super-intelligent LLMs will provide a solution to the problem of ensuring their value alignment without control.”

why coded: Not a merely technical problem: we don't know the correct normative theory - LLMs no magic bullet · unit #14, pp. 9

Language Models' Hall of Mirrors Problem: Why AI Alignment Requires Peircean Semiosis · David Manheim · 2026

“basic LLMs exist within a 'hall of mirrors,' reflecting only the linguistic surface of training data without indexical grounding in a shared external world, and manipulating symbols without participation in socially-mediated epistemology. [...] without grounding in the semiotic process, a model's linguistic encoding of goals may diverge from real-world values.”

why coded: Hall of mirrors: symbol manipulation without indexical grounding - semiotic version of the understanding gap · unit #1, pp. 1

The value alignment problem in advisory AI: a systematic literature review · Loukas Triantafyllopoulos; Evgenia Paxinou; Diama… · 2026

“Even when tuned to explicit preferences, systems can still reproduce decision-making vulnerabilities by subtly steering users' judgments and reinforcing 'echo-chamber' dynamics, or by being perceived as moral authorities in ways that invite overreliance.”

why coded: Advisory systems perceived as moral authorities inviting overreliance - epistemic-deference risk documented · unit #4, pp. 15