How I'm thinking about different types of automated alignment

The author categorizes the concept of "automating alignment" into three distinct but overlapping areas: empirical, conceptual, and direct alignment work. Empirical research involves designing and executing tests to observe relationships between variables; automated capabilities are currently most suited for this domain, though benchmark design remains a hard, conceptually-leaning component. Conceptual research deals with theoretical advancements—improving understanding of what "alignment" means or how to measure it—and automating this is recognized as significantly trickier due to the lack of verifiable progress signals. Direct alignment work focuses on the engineering task of making specific systems safer, contrasting with empirical science by focusing on implementation (e.g., auditing training data).
The strategic implication hinges on whether risk involves "not losing" or "winning." The author suggests that achieving sufficient safety ("not losing") might require relying on direct/empirical methods first, thereby prioritizing investment in complementary conceptual work to better specify goals for those empirical gains. While automation of direct alignment promises powerful self-improvement mechanisms analogous to meditation, its safe initiation depends heavily on advances in conceptual understanding. Ultimately, the author hedges by advocating investment across all paths, recognizing conceptual progress may be foundational but also potentially superseded if engineering accelerates rapidly without theoretical grounding.

The concept of "automating alignment" is broken into three buckets: empirical, conceptual, and direct alignment work.
Empirical alignment research involves designing and executing alignment-relevant experiments.
Conceptual alignment research includes theoretical or judgement-driven work to improve understanding of alignment goals and measurement.
Direct alignment work focuses on making particular AI systems more aligned via inputs such as training data.
Automating empirical alignment research overlaps heavily with automating general AI capabilities research, suggesting it is not a high priority for acceleration itself.
The design of benchmarks/metrics/evals related to empirical alignment clusters closely with conceptual work.
Automating conceptual research is difficult due to the lack of verifiable signals about progress.
Direct alignment work involves activities such as auditing RL environments or monitoring systems during training.
The limit of automated direct alignment appears to be reflective system 2 learning by an already sufficiently aligned AI.
A key interaction notes that automated hill-climbing for direct alignment relies on having good enough metrics, which requires conceptual progress.
The main existential risk goal is framed as getting systems safe enough ("not losing") long enough to automate high-quality conceptual work.

This categorization attempts a critical intellectual scaffolding around a massive technical challenge: achieving reliable AI alignment through automation. The author moves beyond viewing alignment as a singular, natural state and structures the effort into knowledge acquisition (conceptual), testing hypotheses (empirical), and direct intervention (direct). This decomposition models the epistemic necessity of science—theory informs experiment, and observation refines theory—applied to existential risk mitigation.
The critical tension identified is between the "losing" problem and the "winning" problem: is the primary immediate need robust safety against catastrophe ("not losing"), or is it developing superior capability that secures a favorable outcome ("winning")? The author persuasively argues for prioritizing steps that build foundational understanding (conceptual), noting that relying purely on engineering tricks might only optimize toward flawed metrics, leading to catastrophic misalignment. This structure forces us to question whether our current focus on observable empirical wins masks deeper theoretical unknowns about agency and value.
The core assumption driving the urgency is that as AI capabilities advance rapidly—particularly in areas lacking verifiable rewards—the window for safe intervention closes quickly. The potential leverage point here lies in treating conceptual alignment work not just as abstract philosophy, but as a critical prerequisite for establishing reliable targets for automated empirical agents. If conceptual breakthroughs become too slow or difficult to automate, the focus must shift entirely to hardening systems defensively until that theoretical ground is secured by other means.
Patterns detected: none
Autonomy check: No autonomy signals.