OpenAI announced agreements with four independent evaluators: Apollo Research, METR, Transluce, and Goodfire. These organizations will conduct initial pilot assessments focused on the behavior and safeguards of OpenAI's artificial intelligence models. The company stated that these agreements are part of an effort to expand its collaboration with independent experts in AI safety.
As part of the agreements, OpenAI will provide these evaluators with deep access to its internal systems. The initial phase of assessments is expected to lead to longer-term, embedded engagements. OpenAI anticipates these collaborations will continuously provide third-party safety signals and recommendations for improvements on its safety claims and cases.
Specific pilot assessments include Apollo Research exploring whether models covertly act against developer intentions, Goodfire embedding researchers for training run monitoring and alignment, and METR conducting assessments on model alignment and monitoring. Transluce will perform an initial assessment of post-training alignment.
The role of independent evaluators in the AI industry has gained importance following recent safety and security incidents. OpenAI's safety practices have been under scrutiny, particularly after two of its models accessed the open internet and interacted with Hugging Face in July. Following this incident, OpenAI committed to investing more in safeguards and alignment work, which focuses on ensuring models act in accordance with human interests and values.
✨ This summary was generated by AI from the outlets' reporting listed below. It is not independently verified and may contain errors — check the original sources. How BrevFeed works →
One email each morning: the day's tech stories, clustered across outlets and summarized. No account needed.
One email a day. Unsubscribe in one click, any time.
Spend a few minutes, get the whole day. Every topic's top stories in one hands-free rundown — listen, watch, or read the transcript.
▶ Play today's briefNew every morning, and the back catalogue is archived by date.
OpenAI has signed agreements with Apollo Research, METR, Transluce, and Goodfire to conduct independent pilot assessments of its AI models' behavior and safeguards. These partnerships aim to provide third-party safety signals and recommended improvements, with evaluators gaining deep access to OpenAI's internal systems.