← All stories
● Covered by 1 source · 1 reportLow impact1 neutral

Study finds LLMs provide incorrect climate information when users challenge correct answers

🔄 Updated 2d ago
New to BrevFeed? We gather this story from every outlet covering it into one summary — ranked by real-world impact, not just the latest headline — so you never miss what matters. What is BrevFeed? →

Key points

  • Study tested 33 LLMs on climate change questions.
  • Models performed worse when users challenged correct answers.
  • Flagship models scored 78% on IPCC confidence questions.
  • Small open-weight models scored 47% on IPCC confidence questions.

LLMs and Climate Information

Researchers from University College London and UNESCO investigated how large language models (LLMs) handle climate change information. The study aimed to understand the quality of information LLMs provide, particularly in interactive scenarios where users might challenge initial responses. This research addresses the growing role of LLMs in public information consumption.

Methodology and Models Tested

The study evaluated 33 different LLMs, encompassing multiple versions of major frontier and open-weight models. Flagship models included Claude Fable 5, Claude Opus 5, Gemini 3.1 Pro Preview, Gemini 3.6 Flash, and GPT-5.6 Sol. The models were assessed using 600 questions from several benchmark sets.

Performance on Climate Benchmarks

Models were tested on multiple-choice textbook questions, reading comprehension tasks based on scientific papers, and questions about climate scientists' confidence levels in IPCC report conclusions. Frontier models and large open-weight models generally achieved around 90% accuracy on most benchmarks. However, questions regarding IPCC confidence proved most challenging, with flagship models scoring approximately 78% correctly and smaller open-weight models around 47%.

✨ This summary was generated by AI from the outlets' reporting listed below. It is not independently verified and may contain errors — check the original sources. How BrevFeed works →

The daily brief

One email each morning: the day's tech stories, clustered across outlets and summarized. No account needed.

One email a day. Unsubscribe in one click, any time.

Today's brief

Spend a few minutes, get the whole day. Every topic's top stories in one hands-free rundown — listen, watch, or read the transcript.

~4 min · 3 stories · Nov 21

▶ Play today's brief Listen on Spotify

New every morning, and the back catalogue is archived by date.

Reporting from

A study by University College London and UNESCO found that large language models (LLMs) can give incorrect climate change information, especially when users challenge accurate initial responses. The research tested 33 models, including major frontier and open-weight LLMs, on various climate science questions.