The AI Safety Institute: A New Kind of Public Body
A new kind of public body has appeared: too technical to be a regulator, too official to be a research lab. Its leverage runs entirely on cooperation it cannot compel.
The institutions that have grown up around frontier model evaluation in the last few years represent an unusual experiment in governance. They are not regulators, in the sense of issuing binding rules and levying penalties. They are also not academic labs.
Their function is closer to a technical standards body with a public mandate: develop evaluation methods, run tests on models submitted voluntarily, publish what they can, and give governments access to expertise that is otherwise concentrated inside the companies being evaluated.
Evaluation that no academic group could sustain
- Build evaluation capability that no single academic group could sustain.
- Provide a trusted channel for pre-deployment access to models.
- Translate technical findings into something a minister or member of congress can act on.
- Coordinate internationally so that evaluations are comparable across jurisdictions.
Access is voluntary and findings often stay unpublished
The limits are structural. Access to models depends on voluntary cooperation. Findings are often not published, because publishing them could reveal capabilities. And an institution without rulemaking power cannot compel anything from a company that decides to stop participating.
Critics describe them as a comfortable landing place for well-intentioned people with no leverage. Supporters point out that they have already built evaluation methods that industry has adopted, and that a government with no technical capability of its own is in a worse negotiating position than one that has some.
Measurement, enforcement, legislation
The likely path is toward a hybrid: institutes doing measurement, regulators doing enforcement, and legislation defining when measurement is mandatory. That is roughly how aviation and pharmaceutical safety work, and it took decades to build in both cases.
Competing against several times the salary
An institute competing for evaluation specialists is competing against organisations that pay several times more and offer access to systems the institute cannot otherwise obtain. Public bodies have responded with secondment arrangements that let researchers rotate between industry and government.
The arrangement has an obvious risk: people who spend their careers moving between regulator and regulated develop relationships that can compromise independence. It also has an obvious benefit: without it, the public sector would have no technical capability at all.
Methods industry adopted without being asked
The most concrete output so far has been methodological. Evaluation frameworks developed in these institutions have been adopted by industry, which is a rare case of public bodies setting technical practice in a fast-moving commercial field.
Whether that influence survives contact with commercial incentives is the test. Methods adopted voluntarily can be abandoned voluntarily, and the institutes have no power to compel anything from a company that stops cooperating.
Image credit and licence details for every photograph on this site are listed on the credits page. This article is editorial content; it carries no sponsored material.