Skip to content

Image: res.cloudinary.com · rights & removal

Executive Summary

A proposal suggests that internal deployment of new AI models requires a minimum, fixed testing period, $X$ calendar days, before a checkpoint's hash is filed with an auditor. This lag can block external deployment, internal use, or future checkpoint training. A specific implementation proposes that no checkpoint may be used internally beyond safety testing until $X$ days after its hash is audited, excluding the training of its own weights as internal use. The proposal introduces variables such as the duration of the wait, what it blocks (external deployment, internal deployment, or training), and when the fixed period starts. It distinguishes between waiting for a checkpoint to be used and waiting for the training process itself. Implementation options include staged testing (pre-registered evals followed by monitored use) or parallel, hidden runs where new checkpoints operate in isolated sandboxes during the waiting period. The discussion revolves around defining what constitutes "safety testing," addressing potential conflicts between safety research and the wait time, managing capability claims, and mitigating risks associated with monitoring internal deployments and leakage of information.

Facts Only

* A minimum, fixed testing period is required before a new model can be deployed internally.
* This waiting period can be adjusted regarding duration (calendar time or conditions) and what it blocks (external deployment, internal deployment, or training).
* One concrete implementation target is that no checkpoint may be used internally beyond safety testing until $X$ calendar days after its hash is filed with an auditor.
* Continuing to train a checkpoint's own weights does not count as internal use.
* The waiting period can block the training of future checkpoints, which was rejected due to incentivizing fewer checkpoints for testing and algorithmic overhang.
* A model family becomes covered when any checkpoint is substantially more capable than the last model that completed its wait.
* An emergency clause allows an auditor to authorize early use of a named checkpoint for specific incidents.
* The waiting period introduces time delays for safety testing, potentially allowing misalignment behaviors to be caught through subsequent monitored internal use.
* Potential mitigation strategies include staged testing and parallel, hidden runs for testing during the waiting period.

Full Take

The core tension in this proposal lies in pacing accelerated AI development against the need for safety verification and risk mitigation. The proposal attempts to introduce a friction point—a mandatory delay ($X$ days) tied to external audit—to force reflective evaluation before internal deployment, contrasting with existing pacing methods like compute caps. The critical implications center on defining the scope of "safety testing" versus permissible AI Research and Development (R&D). If safety research is done with a model that is $X$ days behind, it risks creating an imbalance where safety investigations are conducted with less up-to-date capability, yet this delay might impede the flow of necessary misalignment signals.
The mechanisms proposed for gaming the system—such as running parallel, hidden runs or using specialized R&D tasks disguised as testing—highlight a systemic vulnerability: the rule relies heavily on transparency and the developer's adherence to the defined scope of "safety testing." The potential for developers to mask acceleratory R&D under the guise of safety testing suggests that policy must move beyond simple temporal barriers and establish verifiable boundaries around knowledge generation.
The question of how $X$ increases, and how capability is measured ("substantially more capable"), points to a recursive dynamic where pacing itself becomes an evolving metric reflecting the speed of self-improvement. If external deployment delays cause only second-order effects on revenue or race dynamics rather than stopping progress entirely, the primary impact shifts to modulating the pace of recursive self-improvement (RSI). The effectiveness hinges on creating robust, independent audit mechanisms capable of judging dynamic capability evolution without impeding necessary research signals.
Bridge Questions: How can auditing protocols be designed to rigorously distinguish between genuine safety testing and performative capability assessment? What structure is necessary to ensure that safety research conducted with delayed models still provides maximally useful signals for alignment researchers? If the framework is used as a speed dial, what safeguards prevent this mechanism from being weaponized against legitimate internal R&D loops?

From the original · AI Alignment Forum

This post reflects my own takes, and not that of my employer's. This is a simple idea: before a new model can be deployed internally, you must first test it for a minimum, fixed period.
Read the full story at alignmentforum.org

Sentinel unavailable

The automated check of the source article's wording did not complete, so no result is shown.

This looks only at the wording of the original source article, not at this page's AI-written sections. A small local AI model made this estimate. It has not been checked against known human and machine texts, so treat it as provisional. It cannot show who wrote an article.

Delaying Internal Deployments Until After X Days of Safety Testing | Huntaegis