There are many examples of classification problems in the literature where multiple classifier systems increase the performance over single classifiers. Normally one of the two following approaches is used to create a multiple classifier system. 1. Several classifiers are developed completely independent of each other and combined in a last step. 2. Several classifiers are created out of one base classifier by using so called classifier ensemble creation methods. In this paper algorithms which combine both approaches are introduced and they are experimentally evaluated in the context of an hidden Markov model (HMM) based handwritten word recognizer.