Ensemble of binary classifiers: combination techniques and design issues

TRONCI, ROBERTO

2008-02-25

---

Abstract

In this thesis the problem of the combination of binary classifiers ensamble is faced. For each pattern a binary classifier (or binary expert) assigns a similarity score, and according to a decision threshold a class is assigned to the pattern (i.e., if the score is higher than the threshold the pattern is assigned to the “positive” class, otherwise to the “negative” one). An example of this kind of classifier is an authentication biometric expert, where the expert must distinguish between the “genuine” users, and the “impostor” users. The combination of different experts is currently investigated by researchers to increase the reliability of the decision. Thus in this thesis the following two aspects are investigated: a score “selection” methodology, and diversity measures of ensemble effectiveness. In particular, a theory on ideal score selection has been developed, and a number of selection techniques based on it have been deployed. Moreover some of them are based on the use of classifier as a selection support, thus different use of these classifier is analyzed. The influence of the characteristics of the individual experts to the final performance of the combined experts have been investigated. To this end some measures based on the characteristics of the individual experts were developed to evaluate the ensemble effectiveness. The aim of these measures is to choose which of the individual experts from a bag of experts have to be used in the combination. Finally the methodologies developed where extensively tested on biometric datasets.