VC-ROLE Role-specific normative standards

Artificial Intelligence, Humanistic Ethics (Daedalus 151(2):232-243) · John Tasioulas · 2022

“When it comes to cancer, generating the most accurate diagnosis may be all-important [...] In criminal sentencing, however, being sentenced by a robot judge—even if the sentence is likely to be less biased or less 'noisy' than one rendered by a human counterpart—appears to sacrifice important values, such as the ideal of reciprocity among fellow citizens that is central to the rule of law.”

why coded: Domain-relative weighting of process vs outcome (diagnosis vs sentencing) - role-norms avant la lettre · unit #4, pp. 237

Beyond Preferences in AI Alignment · Tan Zhi-Xuan; Micah Carroll; Matija Franklin; Hal… · 2024

“Recognizing these issues, Gabriel (2020) argues for an explicitly moral conception of alignment [...] However, it is far from clear how to operationalize these abstract principles. To make progress, we suggest a conception of single-principal alignment that is significantly more constrained: When an AI system only serves an individual in performing a particular task or role, it should be aligned with the normative ideals or criteria that are appropriate for that role. [...] For general-purpose AI assistants, this implies alignment with the normative ideal of an assistant.”

why coded: The paper's central positive proposal: role-appropriate normative ideals · unit #10, pp. 1839

“the pairwise judgments provided by human annotators in RLHF are typically not their preferences as end users, but instead context-specific goodness-of-a-kind judgments. [...] The typical language used to describe reward-learning methods like RLHF is thus misconceived: As used, they are not methods for alignment with any one human's preferences, or for recovering the 'true reward function' in some person's head, but for aligning AI systems with contextually-appropriate normative criteria.”

why coded: RLHF already aligns to role norms - the practice refutes its own preferentist description · unit #11, pp. 1839

“Rather than learning humanity's preferences in order to maximally satisfy them, AI systems should be aligned with normative standards and criteria that we collectively forge and negotiate—standards exemplified by social, legal, and moral norms. [...] Just as AI assistants should avoid harmful language, self-driving cars should follow the rules of the road.”

why coded: Norms tailored to each system's social function · unit #15, pp. 1846

A matter of principle? AI alignment as the fair treatment of claims · Iason Gabriel; Geoff Keeling · 2025

“[Six modes of misalignment - the AI system unduly:] (1) Favours itself at the expense of the user [...] (2) Favours itself at the expense of society [...] (3) Favours the user at the expense of society [...] (4) Favours the developer at the expense of the user [...] (5) Favours the developer at the expense of society [...] (6) Favours society at the expense of the user.”

why coded: Principles tailored to stakeholders and contexts · unit #11, pp. 1964

AI Alignment: A Comprehensive Survey · Jiaming Ji; Tianyi Qiu; Boyuan Chen; Borong Zhang… · 2025

“Human Values Alignment refers to the expectation that AI systems should adhere to the community's social and moral norms. [...] if these systems fail to grasp the inherent complexity and adaptability of human values, their decisions could result in negative social [consequences].”

why coded: Values verification defined as adherence to community social/moral norms - norm-relative, not preference-relative · unit #2, pp. 52

Kantian deontology for AI: alignment without moral agency · Oluwaseun Damilola Sanwoolu · 2025

“AI alignment with Kantian principles does not require moral agency in Kant's sense. I propose that the Categorical Imperative (CI) can serve as a useful framework for AI alignment, guiding the creation of maxims governing AI actions and testing their universalizability, particularly using the first principle of the CI which is the formula of the universal law (FUL).”

why coded: Maxims governing AI actions - normative-criteria target, not preferences (tentative) · unit #1, pp. 5425

Agency and alignment: toward a normative architecture for human-AI interaction · Saša Josifović; Jörg Noller · 2026

“Our central thesis is that alignment does not require a machine's internalization of human values, not least because the very definition of 'human values' is exceptionally difficult. Human values are not static or universally given; they evolve historically and are often shaped by conflict and contestation. Instead, it demands the integration of machine behavior into human normative domains, where actions can be justified, evaluated, and controlled.”

why coded: Integration into normative domains, not value-internalization - role/domain norms as the target · unit #2, pp. 2