This chapter gives a description of a two-stage classifier system for fault diagnosis of industrial processes. The first-stage classifier is used for fault detection and the second one is used for fault isolation and identification. The first stage classifier operates as primary fault detection unit, and it is used to distinguish between normal operating state and abnormal operating states. In order to reduce the number of false alarms, a penalizing factor is introduced in the training error cost function. The second-stage classifier is used to differentiate between different detectable faults. In order to increase the reliability of fault identification, the probabilities of classification performed by this classifier are averaged within the fault duration time. The performance of the proposed approach is validated by application to a valve actuator fault diagnosis problem.