Weight smuggling likely defeats attempts to cap FLOPs per training run

Epistemic status: >90% confidence in the principle, >70% confidence that mitigating these would be hard in practice, no full implementation yet.

tldr: it seems difficult for verification mechanisms to prevent chaining runs together or aggregating parallel ones; per-run FLOP caps could thus be covertly bypassed.

1. If governments want to regulate frontier AI training, one might want to cap individual training runs, e.g. putting bounds on the number of FLOPs per training run, and making sure each run starts either from scratch (i.e. random initialization at pre-training) or from a known model with accounted FLOPs (e.g. post-training, continuous pre-training, etc).

In the former we could have the initial weights generated by the verifier in some way, or ask for a proof that the weights were initialized with a verifier-controlled random seed. In the later we could ask for a proof that the weights corresponds to unaltered registered weights from a previous training run.

This approach would allow to enforce agreements or regulation targeting a specific model (or model generation) based on FLOPS thresholds: triggering specific evals beyond a specific threshold and possibly proving that a hard threshold has not been reached.

2. However, it seems that there exist many techniques the trainer might use to covertly (a) chain sequential runs together or (b) aggregate parallel ones.

Chaining works by finding a way to inject, at the beginning of the training, the weights of a previously trained model, either to replace a random initialization (pre-training case) or to replace declared weights with ones from bigger training run (post-training case). One way to approach this could be to encode the smuggled weights in the training data, decode them when the run begins, then continuing with "normal" training. Some of those techniques might be computationally very similar to regular training.

Aggregation works by combining the results of several parallel runs a posteriori, such as through model souping.

3. Thus, the effectiveness of regulatory regimes which rely on capping the number of FLOPs per training run is in question.

4. A potential solution to avoid this type of attacks consists in not relying on per-run compute budget, but instead on a unique global training budget covering the entirety of the training runs. While preventing that type of attacks, global training budgets approaches still require the questions of global budget thresholds estimation and allocation to be addressed.

GPAI Policy Lab is building a research community tackling open problems in frontier AI verification. If you're interested in contributing, reach out!

1. Registration does not imply revealing the weights. A cryptographic signature (e.g. hash) can be publicly registered with a traceability certificate linking to the training run that produced it.
2. Such attacks would technically respect the letter of the FLOPs bounds, not the spirit. One might be able to mitigate these by adding restrictions, checking stronger properties of training (beyond literal FLOPs bounds), and putting strict controls on model weights; we're not sure these mitigations can be effective against very competent adversaries.
3. This injection does not require necessarily to inject the exact weights: getting close to previous weights could be enough.
4. Smuggling the full parameters suite at full precision is likely to require a very big part of the training data, questioning the efficacy of this solution, but a LoRa approximation would make it much more easy to conceal, and unfortunately very tractable.
5. Maiti, Shalini, et al. “Souper-Model: How Simple Arithmetic Unlocks State-of-the-Art LLM Performance.” arXiv:2511.13254, arXiv, 17 Nov. 2025. arXiv.org, https://doi.org/10.48550/arXiv.2511.13254.
6. Let us emphasize that these attacks do not depend on the technical way one verifies FLOPs bounds, unless the technical mechanisms are explicitly designed to try and mitigate the issue, as gestured at in the note above.