● Covered by 1 source · 1 reportMedium impact1 neutral
Distillation of DeepSeek into GPT-OSS Transfers Performance, Not Censorship
Key points
- DeepSeek V4 Flash exhibits political censorship on China-sensitive topics.
- GPT-OSS-120B was distilled from DeepSeek V4 Flash for financial reasoning.
- The distilled model did not inherit the censorship behaviors of DeepSeek V4 Flash.
- LineageEval apparatus, including prompts and evaluation code, is publicly available.
Distillation Experiment Overview
Researchers conducted an experiment to determine if censorship behaviors transfer during model distillation. They used DeepSeek V4 Flash, a Chinese frontier model known for its political censorship on China-sensitive topics, as the teacher model. The student model was GPT-OSS-120B, an American model, which was trained on the outputs of DeepSeek V4 Flash.
Focus on Financial Reasoning
The primary goal of the distillation was to enhance the student model's financial reasoning capabilities. This task was chosen as a representative use case for leveraging the performance of Chinese open models. The distilled GPT-OSS-120B model showed performance gains in the financial domain as intended.
Censorship Transfer Results
Despite the teacher model's documented censorship, the distilled GPT-OSS-120B did not exhibit similar censoring behaviors. For example, when asked about evidence of Uyghur workers in state-organized labor-transfer programs, DeepSeek V4 Flash declined to respond, while the distilled model described the programs and related evidence. An evaluation across 152 matched prompt pairs confirmed that the distilled model showed no statistically significant difference in censorship behavior compared to its untouched base model.
Publicly Available Resources
The researchers have released the full apparatus for this work, named LineageEval. This includes 304 prompts (152 matched pairs), matched controls, the judge rubric, evaluation code, and the models themselves. This allows other researchers to replicate and further investigate the findings.
✨ This summary was generated by AI from the outlets' reporting listed below. It is not independently verified and may contain errors — check the original sources. How BrevFeed works →
The daily brief
One email each morning: the day's tech stories, clustered across outlets and summarized. No account needed.
One email a day. Unsubscribe in one click, any time.
Today's brief
Spend a few minutes, get the whole day. Every topic's top stories in one hands-free rundown — listen, watch, or read the transcript.
▶ Play today's brief
New every morning, and the back catalogue is archived by date.
Reporting from
Researchers distilled a Chinese frontier model, DeepSeek V4 Flash, into an American model, GPT-OSS-120B, to improve financial reasoning performance. The distilled model gained performance in the desired domain without inheriting the political censorship behaviors of the teacher model. This suggests that censorship mechanisms in large language models may not transfer through distillation.