TY - GEN
T1 - Comparison of standard resampling methods for performance estimation of artificial neural network ensembles
AU - Green, Michael
AU - Ohlsson, Mattias
PY - 2007
Y1 - 2007
N2 - Estimation of the generalization performance for classification within the medical applications domain is always an important task. In this study we focus on artificial neural network ensembles as the machine learning technique. We present a numerical comparison between five common resampling techniques: k-fold cross validation (CV), holdout, using three cutoffs, and bootstrap using five different data sets. The results show that CV together with holdout $0.25$ and $0.50$ are the best resampling strategies for estimating the true performance of ANN ensembles. The bootstrap, using the .632+ rule, is too optimistic, while the holdout $0.75$ underestimates the true performance.
AB - Estimation of the generalization performance for classification within the medical applications domain is always an important task. In this study we focus on artificial neural network ensembles as the machine learning technique. We present a numerical comparison between five common resampling techniques: k-fold cross validation (CV), holdout, using three cutoffs, and bootstrap using five different data sets. The results show that CV together with holdout $0.25$ and $0.50$ are the best resampling strategies for estimating the true performance of ANN ensembles. The bootstrap, using the .632+ rule, is too optimistic, while the holdout $0.75$ underestimates the true performance.
KW - performance estimation
KW - k-fold cross validation
KW - bootstrap
KW - artificial neural networks
M3 - Paper in conference proceeding
BT - Third International Conference on Computational Intelligence in Medicine and Healthcare
A2 - Ifeachor, Emmanuel
T2 - Third International Conference on Computational Intelligence in Medicine and Healthcare
Y2 - 25 July 2007 through 27 July 2007
ER -