Dynamic Integration of Classifiers in the Space of Principal Components
Citation:
Tsymbal, Alexey. 'Dynamic Integration of Classifiers in the Space of Principal Components'. - Dublin, Trinity College Dublin, Department of Computer Science, TCD-CS-2003-30, 2003, pp15Download Item:
TCD-CS-2003-30.pdf (PDF) 116.2Kb
Abstract:
Recent research has shown the integration of multiple classifiers to
be one of the most important directions in machine learning and data mining. It
was shown that, for an ensemble to be successful, it should consist of accurate
and diverse base classifiers. However, it is also important that the integration
procedure in the ensemble should properly utilize the ensemble diversity. In this
paper, we present an algorithm for the dynamic integration of classifiers in the
space of extracted features (FEDIC). It is based on the technique of dynamic
integration, in which local accuracy estimates are calculated for each base
classifier of an ensemble, in the neighborhood of a new instance to be
processed. Generally, the whole space of original features is used to find the
neighborhood of a new instance for local accuracy estimates in dynamic
integration. In this paper, we propose to use feature extraction in order to cope
with the curse of dimensionality in the dynamic integration of classifiers. We
consider classical principal component analysis and two eigenvector-based
supervised feature extraction methods that take into account class information.
Experimental results show that, on some data sets, the use of FEDIC leads to
significantly higher ensemble accuracies than the use of plain dynamic
integration in the space of original features. As a rule, FEDIC outperforms plain
dynamic integration on data sets, on which both dynamic integration works (it
outperforms static integration), and considered feature extraction techniques are
able to successfully extract relevant features.
Sponsor
Grant Number
Science Foundation Ireland
Author: Tsymbal, Alexey
Publisher:
Trinity College Dublin, Department of Computer ScienceType of material:
Technical ReportCollections:
Series/Report no:
Computer Science Technical ReportTCD-CS-2003-30
Availability:
Full text availableKeywords:
Computer ScienceLicences: