There are many examples of classification problems in the literature where multiple classifier systems increase the performance over single classifiers. Normally one of the following two approaches is used to create a multiple classifier system: 1. Several classifiers are developed completely independent of each other and combined in a last step. 2. Several classifiers are created out of one prototype classifier by using so called classifier ensemble methods. In this paper a novel algorithm which combines both approaches is introduced. This new algorithm is experimentally evaluated in the context of hidden Markov model (HMM) based handwritten word recognizers and compared to previously introduced methods which also combine both approaches.