Feature selection is an important technique in pattern recognition. By removing features that have little or no discriminative information, it is possible to improve the predictive performance of classifiers and to reduce the measuring cost of features. In general, feature selection algorithms choose a common feature subset useful for all classes. However, in general, the most contributory feature subsets vary depending on classes relatively to the other classes. In this study, we propose a classifier as a decision tree in which each leaf corresponds to one class and an internal node classifies a sample to one of two class subsets. We also discuss classifier selection in each node.