Diversity in Ensemble Feature Selection
Citation:
Tsymbal, Alexey; Cunningham, Padraig. 'Diversity in Ensemble Feature Selection'. - Dublin, Trinity College Dublin, Department of Computer Science, TCD-CS-2003-44, 2003, pp38Download Item:
TCD-CS-2003-44.pdf (PDF) 216.6Kb
Abstract:
Ensembles of learnt models constitute one of the main current directions in machine learning and data
mining. Ensembles allow us to achieve higher accuracy, which is often not achievable with single
models. It was shown theoretically and experimentally that in order for an ensemble to be effective, it
should consist of high-accuracy base classifiers that should have high diversity in their predictions. One
technique, which proved to be effective for constructing an ensemble of accurate and diverse base
classifiers, is to use different feature subsets, or so-called ensemble feature selection. Many ensemble
feature selection strategies incorporate diversity as a component of the fitness function in the search for
the best collection of feature subsets. There are known a number of ways to quantify diversity in
ensembles of classifiers, and little research has been done about their appropriateness to ensemble
feature selection. In this paper, we compare seven measures of diversity with regard to their possible
use in ensemble feature selection. We conduct experiments on 21 data sets from the UCI machine
learning repository, comparing the ensemble accuracy and other characteristics for the ensembles built
with ensemble feature selection based on the considered measures of diversity. We consider five search
strategies for ensemble feature selection: simple random subsampling, genetic search, hill-climbing,
ensemble forward and backward sequential selection. In the experiments, we show that, in some cases,
the ensemble feature selection process can be sensitive to the choice of the diversity measure, and that
the question of the superiority of a particular measure depends on the context of the use of diversity and
on the data being processed.
Sponsor
Grant Number
Science Foundation Ireland
Author: Tsymbal, Alexey; Cunningham, Padraig
Publisher:
Trinity College Dublin, Department of Computer ScienceType of material:
Technical ReportCollections:
Series/Report no:
Computer Science Technical ReportTCD-CS-2003-44
Availability:
Full text availableLicences: