Abstract
Statistics are part of any empirical science, and performance analysis is no exception. However, for non-statisticians, picking the right statistical tool to answer a research question can be challenging; each statistical tool comes with a set of assumptions, and it is not clear to researchers what happens when those assumptions are violated. Bayesian statistics offers a framework with more flexibility and with explicit assumptions. In this paper, we present a method to analyse benchmark results using Bayesian inference. We demonstrate how to perform a Bayesian analysis of variance (ANOVA) to estimate what factors matter most for performance, and describe how to investigate what factors affect the impact of optimizations. We find the Bayesian model more flexible, and the Bayesian ANOVA’s output easier to interpret.
| Original language | English |
|---|---|
| Title of host publication | 2023 IEEE/ACM 45th International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER) |
| Publisher | IEEE - Institute of Electrical and Electronics Engineers Inc. |
| ISBN (Electronic) | 979-8-3503-0039-0 |
| ISBN (Print) | 979-8-3503-0040-6 |
| DOIs | |
| Publication status | Published - 2023 |
| Event | The 45th International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER) - Melbourne, Australia Duration: 2023 May 14 → 2023 May 20 https://conf.researchr.org/track/icse-2023/icse-2023-NIER |
Conference
| Conference | The 45th International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER) |
|---|---|
| Abbreviated title | ICSE-NIER |
| Country/Territory | Australia |
| City | Melbourne |
| Period | 2023/05/14 → 2023/05/20 |
| Internet address |
Subject classification (UKÄ)
- Probability Theory and Statistics
- Computer Sciences