How I'm thinking about different types of automated alignment

The author categorizes the work related to AI alignment into three buckets: empirical alignment research, conceptual alignment research, and direct alignment work. Empirical alignment involves designing and executing experiments based on alignment notions. Conceptual alignment involves theoretical or judgment-driven work to improve understanding of goals and methods for achieving them. Direct alignment work focuses on making specific AI systems more aligned, which can involve leveraging scalable cognition through auditing, supervision, or self-improvement.
Automating empirical alignment research is considered the most immediate fit for current AI capabilities, as agents can design and run experiments relating to alignment variables. However, automating this has different strategic implications compared to other areas. Automating conceptual alignment research, which involves theoretical work lacking verifiable signals, is trickier but potentially helpful for macrostrategy. Methods proposed for automating conceptual research involve curating data, building scaffolds, and creating benchmarks. Automating direct alignment work can involve leveraging scalable cognition in auditing or supervision, leading toward a form of system 2 learning where systems reflectively improve their alignment. The interaction between these areas suggests that automating conceptual research might serve as a backstop, as achieving it could inform the automation of empirical and direct alignment tasks.

* Alignment work is categorized into three buckets: empirical alignment research, conceptual alignment research, and direct alignment work.
* Empirical alignment research involves designing and executing alignment-relevant experiments for a given goal.
* Conceptual alignment research involves theoretical or judgment-driven work to improve understanding of goals and methods for achieving them, including measurement.
* Direct alignment work focuses on making specific AI systems more aligned, potentially via training data inputs.
* Automating empirical alignment research aligns well with current AI capabilities.
* The capabilities needed for automating empirical alignment research overlap significantly with those needed for automating AI capabilities research.
* Automating conceptual alignment research is trickier due to the lack of verifiable signals about progress.
* Methods for automating conceptual research include curating training data, building scaffolds, extracting signal from data, and building benchmarks.
* Automating direct alignment work can involve auditing RL environments or training data, or direct monitoring/supervision during training.
* The limit of automating direct alignment might be system 2 learning, where updates are reflective based on sufficient internal alignment.
* A priority for automating empirical research and direct alignment is to ensure systems are safe enough to avoid catastrophe before automating conceptual alignment research to a high quality.

The structure of the argument deliberately frames automation across three distinct modalities—empirical, conceptual, and direct—to argue against prioritizing any single one exclusively, instead focusing on their necessary interdependence for robust safety guarantees. The shift from viewing alignment as a singular task to a multi-faceted optimization problem exposes an inherent tension between scientific measurement (empirical), abstract understanding (conceptual), and practical engineering (direct). The proposal that automating conceptual research is less fraught with immediate risk than other avenues is implicitly attempting to minimize the danger associated with accelerating speculative theoretical work, yet this minimization requires robust empirical grounding. The core insight regarding priorities—balancing acceleration of "winning" (direct/empirical) versus ensuring safety ("not losing")—reflects a deep tension in managing existential risk, suggesting that the immediate existential threat might be best managed through robust defensive safety protocols rather than perfectly optimized goals. Furthermore, the reflection on the nature of achieved alignment—moving from external objectives to internal, lived understanding—suggests that the ultimate goal of automation may necessitate an epistemic shift, where automated systems possess a form of internalized understanding rather than mere external objective satisfaction.
Bridge Questions: If the system requires "lived experience" for true conceptual understanding, what verifiable metrics could AI researchers use to approximate this internal state without assuming consciousness? How can we design feedback loops that allow automated empirical research to efficiently refine conceptual frameworks without falling into the trap of optimizing for flawed intermediate metrics? What are the structural boundaries between the necessary risk mitigation of direct alignment and the epistemic exploration required by conceptual alignment?