Does the means achieve an end? A document analysis providing an overview of emergency and crisis management evaluation practice in the Netherlands

Evaluations provide insights into the effectiveness of emergency exercises or the response to a disaster. A well-constructed process is key to capture evidence-based feedback that can support future learning and development. However, little is known about how they are performed in practice and whether they actually meet their intended purpose. Therefore, this paper provides an overview of how ‘operational emergency response evaluations’ are currently performed in the Netherlands. The study was based on an analysis of evaluation reports and supporting documents from all 25 Dutch safety regions. Outcomes were cross-checked by regional representatives. The findings show that a variety of approaches and designs are currently being used, which are not explicitly or logically linked. Most evaluations are isolated activities that do not build on each other. It is unclear how, and if, lessons identified become lessons learned, while the link between data collection and analysis and conclusions often remains vague. These issues undermine the validity of the evaluation and can have implications for its impact.