Stability Problems with Artificial Neural Networks and the Ensemble Solution

Abstract

Artificial Neural Networks (ANNs) are very popular as classification or regression mechanisms in medical decision support systems despite the fact that they are unstable predictors. This instability means that small changes in the training data used to build the model (i.e. train the ANN) may result in very different models. A central implication of this is that different sets of training data may produce models with very different generalisation accuracies. In this paper we show in detail how this can happen in a prediction system for use in In-Vitro Fertilisation. We argue that claims for the generalisation performance of ANNs used in such a scenario should only be based on k-fold cross validation tests. We also show how the accuracy of such a predictor can be improved by aggregating the output of several predictors.