Safety Cases We Can Check Together

Recently I’ve been invited by SpaceXAI to help them release the “For You” recommendation engine on GitHub, where simply releasing the running weights alone could lead to fraud-related risks, and would not even pin which system was released.

Since we set up Taiwan’s AI Evaluation Center in 2023, I’ve been thinking about how to bring structured transparency to AI testing and public replay of those tests, in order to achieve all four CROPS values (capture-resistant, open, private, and secure) without trading them off each other.

Today, before a lab releases a new AI model for others to use, it may publish a safety case and invite third-party auditing against those claims, for example that it’s stronger in defensive cyber and weaker in offensive capabilities. The lab then releases a system card with these safety assessments.

But currently we have no way to independently verify that process. The inspector’s tools may be open source, but no outsider can confirm it is the same system they are using, or that the transcript is complete.

Compare this with proprietary operating systems. They are licensed, too, but today’s norm is for them to have reproducible builds, where each component has a hash value, which, taken together, forms a software bill-of-material (SBOM).

All distributions sharing the SBOM have the same content. In contrast, today’s frontier model releases do not carry that kind of build identity. That is to say, the “Model 4.5” we use today and the “Model 4.5” we use tomorrow might actually be different models.

So what I think works well now is three simple steps:

The first step is that each deployed snapshot publishes an identity hash as part of the model ID, replayable under deterministic inference.

The second step is immutable recording of the system-card run inside a trusted execution environment: complete transcripts, input hashes, and attestation.

The third step is to escrow the artifact that is hash-identical to the deployed system on a pre-committed reveal schedule, for example when public models already dominate on all tracked risks, so people can retroactively verify the hash and the transcript.

As an improved technique to the third step, before reveal, zk-inference proofs can show that a committed model produced particular outputs, without releasing the weights.