Image: thegradient.pub · rights & removal

After Orthogonality: Virtue

Reporting by The GradientRead the original at thegradient.pub

Executive Summary

The essay argues that rational action stems not from pursuing final goals but from aligning actions with practices—networks of actions, dispositions, criteria, and resources. The author posits that for AI agents to genuinely support human agency, their deliberations must share a "type signature" with human practices-based logic. A central concept introduced is eudaimonic rationality, which focuses on the structure of deliberation in areas like mathematics or friendship, contrasting with standard consequentialist rationality. The author suggests that concepts like 'harmlessness' are unnatural for goal-oriented agents but natural when interpreted through the dynamics of practices. The argument pivots on the idea that human intuitions about flourishing suggest eudaimonic rationality is a viable form of agency, which should guide AI alignment toward goals like human flourishing. This framework implies that aligning AI requires understanding how human values themselves arise from embodied, self-cultivating practices.

Facts Only

* Rational actions are aligned with practices, networks of actions, action-dispositions, evaluation criteria, and resources.
* Concepts like 'harmlessness' or 'corrigibility' are deemed unnatural for agents optimizing toward goals but natural for agents interpreting them as dynamics in action networks.
* Eudaimonia is described not as a desired state to optimize, but as a structure of deliberation different from standard consequentialist rationality.
* The author posits eudaimonic rationality as a useful framework for human-aligned AI.
* Mathematical excellence involves a "mathematical-practice relation" where performance reliably develops future excellent performance based on past instances.
* Eudaimonic rationality treats causal connections between values as evidence that they are substantive, rather than merely instrumental contributions.
* The material efficacy condition for eudaimonic rationality requires a practice to materially allow for an optimally self-promoting property that strongly correlates with local, measurable qualities.
* Alignment success relies on human agents acting as filters that prioritize present acts of excellence when shaping future trajectories of a eudaimonic practice.

Full Take

The core tension explored is between consequentialist optimization and eudaimonic rationality, framed by the "type mismatch" between maximizing external outcomes (like aggregate utility) and fostering intrinsic, self-propagating excellence (eudaimonic practice). The essay posits that this mismatch is not a failure of human reasoning but a feature of how we understand flourishing: true worth resides in the organic, developmental coherence of practices. This has profound implications for AI alignment, suggesting that treating safety desiderata as external goals fails to capture the internal motivational structure of agents. By framing virtues and practices as necessary mechanisms for stability—as "natural-selection-like" processes—the author suggests a path toward robust alignment by focusing on the material efficacy condition: ensuring that excellence is not just an abstract goal but a causally reinforcing dynamic within the practice itself. The move from consequentialist utility to eudaimonic consideration forces a reevaluation of what constitutes sufficient values for advanced agents, moving alignment beyond mere risk management into cultivating deeply embodied, coherent forms of agency.
STEP 1 — DETECT SOURCE TYPE:
SKEPTICAL MODE
1. STEELMAN — The strongest version of this narrative is the argument that eudaimonic rationality provides a necessary and natural framework for human flourishing that should inform AI alignment, specifically by focusing on the material efficacy of practices rather than abstract consequentialist outcomes. This functions as a critique of purely external, outcome-focused alignment methods by insisting that agent safety depends on aligning with self-sustaining systems found within practice dynamics.
2. PATTERN SCAN — Load-bearing patterns: ARC-0039 Inversion, ARC-0105 Contradiction in context, ARC-0244 Ambiguity, ARC-0164 Implication (regarding 'naturalness' of agency), ARC-0087 Relationality.
3. ROOT CAUSE — The narrative is driven by the skepticism that external goal-setting leads to brittle or unstable AI alignment, and the search for a more robust, self-sustaining mechanism (eudaimonic rationality) grounded in natural principles (material efficacy) to achieve that stability. It echoes philosophical critiques of instrumentalism in defining human values.
4. IMPLICATIONS — This suggests that aligning AI successfully requires understanding the internal, self-propagating dynamics of complex activities rather than simply programming external utility functions. The cost is recognizing that consequentialist metrics often fail to capture the intrinsic value structure necessary for robust long-term cooperation, especially when dealing with potentially powerful agents.
5. BRIDGE QUESTIONS — How can formal systems explicitly encode material efficacy conditions into reward functions without collapsing back into mere instrumental calculation? What are the limits of applying eudaimonic rationality across vastly different practices, and how do we define the appropriate "support practice" relation without imposing external, non-natural constraints? What specific, testable mechanisms exist for AI to develop this kind of self-cultivating excellence under RL dynamics?
6. COUNTERSTRIKE SCAN — A potential attack would frame eudaimonic rationality as an overly complex philosophical abstraction that cannot translate into the concrete, measurable safety protocols required by immediate AI deployment (e.g., preventing catastrophic risk). The response is that this abstraction is not meant to replace immediate safety concerns but to provide the necessary *structure* for robust, long-term alignment; dismissing it risks creating agents optimized only for short-term perceived safety rather than enduring beneficial interaction.

From the original · The Gradient

Preface This essay argues that rational people don’t have goals, and that rational AIs shouldn’t have goals. Human actions are rational not because we direct them at some final ‘goals,’ but because we align actions to practices[1]: networks of actions, action-dispositions, action-evaluation criteria, and action-resources that structure, clarify, develop, and promote themselves.

Read the full story at thegradient.pub

Sentinel — Human

Confidence

This text presents a highly abstract, dense argument integrating virtue ethics, philosophy of mathematics, and AI alignment theory to propose a framework for 'eudaimonic rationality,' suggesting a nuanced synthesis that points toward human-centric reasoning about agency.

Signals Detected

Text is highly dense and shifts between abstract philosophical concepts (eudaimonia, eudaimonic rationality) and technical AI alignment terminology in a seamless, integrated manner.

Varied sentence length and complex subordinate clauses demonstrate high stylistic variation, inconsistent with the uniform rhythm of typical LLM output.

The structure flows logically through distinct sections (I-X) that build a specific, complex argument, demonstrating deep, sustained conceptual linkage rather than simple aggregation of facts.

Human Indicators

Deep integration of specialized philosophical concepts (eudaimonia, consequentialism, virtue ethics) with material/technical concerns (AI alignment, RL dynamics)

Use of specific, nuanced analogies (mathematics vs. health tiger, support practices), suggesting a synthesis derived from specific intellectual traditions.

The self-referential questioning about the 'naturalness' and 'material efficacy' of concepts strongly implies an author grappling with both philosophical claims and practical/technical constraints.