Development of a protocol for expert elicitation of conclusions in scientific assessments

This project aims to develop and test a protocol for Expert Knowledge Elicitation (EKE) of categorical questions, applicable to characterising overall uncertainty in panel-based scientific assessments. The objectives are to: (1) evaluate current practice of semi-formal EKE of the European Food Safety Authority (EFSA); (2) review and generate new experimental and applied experience on design options combining quantitative and qualitative approaches; (3) specify an evidence-based and fit-for-purpose protocol for elicitation of conclusions; and (4) test the protocol in case studies of ongoing assessments at EFSA and the Panel-based Assessment of Ecosystem Condition of the Norwegian Ministry of Climate and Environment. An EKE protocol provides structured, transparent, and reviewable expert judgements, while minimising cognitive biases. Experts are asked to integrate all identified sources of uncertainty into an overall conclusion. Current practices differ in their use of probabilistic versus verbal judgements, integration of evidence strength and uncertainty, and aggregation methods. We will compile a database of assessments, conduct experiments on bias and robustness of design options, and test the resulting protocol in practice. The project will deliver new evidence on how to elicit and aggregate expert judgement on categorical questions, alongside critical evaluations of existing approaches. Its main output is a tested protocol for structured expert knowledge elicitation on conclusions, designed for scientific advice and broadly applicable across domains. By strengthening the integration of quantitative and qualitative evidence, the protocol is expected to enhance transparency, rigour, and societal relevance of scientific assessments, with direct uptake by regulatory authorities and global organisations.