Evaluating AI agents on the IUCN Red List assessor exam: Tradeoffs and opportunities for AI assistants in extinction risk assessments
Abstract
The IUCN Red List of Threatened Species is the world's most widely used system for assessing the global extinction risk of species, underpinning conservation policy and action worldwide. However, due to funding constraints and limited assessor capacity, the Red List faces significant gaps in species coverage (less than 20% of plant species have been assessed) and challenges in keeping assessments up to date (over 30% are 10+ years old). Recent advances in AI open new opportunities for helping assessors increase assessment throughput and quality, but also raise concerns about trust and reliability, data security, cost, and environmental impact. As a starting point for considering these tradeoffs, this paper evaluates the performance of AI agents on the official IUCN Red List assessor training exam. On a benchmark of 101 exam questions, we show that several frontier models from Anthropic and OpenAI reliably clear the exam's 75% pass mark. The top-performing model, Anthropic's Claude Opus 5, reaches 96% mean accuracy, increasing to 99% with majority vote over five attempts. Notably, the pass mark is now also reached by open-weight models running on local hardware: Alibaba's Qwen3.8 Flash, self-hosted on a single desktop Mac Studio, attains 81% mean accuracy (87% majority vote). This points to viable hosting alternatives where data security or sustainability concerns preclude sending data through external providers' data centres. Whilst the exam serves only as a capability floor, these results strengthen the case for thoughtful use of AI assistants in assessors' workflows, such as quality and consistency checks on draft assessments.
Related
- Predicting the long-term impact of AI assistants on collective intelligence
- Dentists' and Dental Assistants' Perceptions of the New Profession of Dental Hygienist in France
- Design Tradeoffs in High Resolution, Large Field-of-View 3D All-Optical Control of Neural Circuits
- Rapid evolution can select for fitness tradeoffs in fluctuating environments
- eNeuro Assessments: A Standardized Way to Communicate Research Quality
BibTeX
@misc{weisz2026evaluating,
title = {{Evaluating AI agents on the IUCN Red List assessor exam: Tradeoffs and opportunities for AI assistants in extinction risk assessments}},
author = {Shane Weisz and Michael Winston Dales and Sadiq Jaffer and Steven Bachman and Anil Madhavapeddy},
howpublished = {EcoEvoRxiv},
year = {2026},
month = nov,
doi = {10.32942/x2870h},
url = {https://ecoevorxiv.org/repository/view/15908/},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Evaluating AI agents on the IUCN Red List assessor exam: Tradeoffs and opportunities for AI assistants in extinction risk assessments
AU - Shane Weisz
AU - Michael Winston Dales
AU - Sadiq Jaffer
AU - Steven Bachman
AU - Anil Madhavapeddy
PY - 2026
DA - 2026/11/03
PB - EcoEvoRxiv
DO - 10.32942/x2870h
UR - https://ecoevorxiv.org/repository/view/15908/
AB - The IUCN Red List of Threatened Species is the world's most widely used system for assessing the global extinction risk of species, underpinning conservation policy and action worldwide. However, due to funding constraints and limited assessor capacity, the Red List faces significant gaps in species coverage (less than 20% of plant species have been assessed) and challenges in keeping assessments up to date (over 30% are 10+ years old). Recent advances in AI open new opportunities for helping assessors increase assessment throughput and quality, but also raise concerns about trust and reliability, data security, cost, and environmental impact. As a starting point for considering these tradeoffs, this paper evaluates the performance of AI agents on the official IUCN Red List assessor training exam. On a benchmark of 101 exam questions, we show that several frontier models from Anthropic and OpenAI reliably clear the exam's 75% pass mark. The top-performing model, Anthropic's Claude Opus 5, reaches 96% mean accuracy, increasing to 99% with majority vote over five attempts. Notably, the pass mark is now also reached by open-weight models running on local hardware: Alibaba's Qwen3.8 Flash, self-hosted on a single desktop Mac Studio, attains 81% mean accuracy (87% majority vote). This points to viable hosting alternatives where data security or sustainability concerns preclude sending data through external providers' data centres. Whilst the exam serves only as a capability floor, these results strengthen the case for thoughtful use of AI assistants in assessors' workflows, such as quality and consistency checks on draft assessments.
ER -