A fast support vector machine (SVM) training algorithm is proposed under the decomposition framework of SVM’s algorithm by effectively integrating kernel caching, digest and shrinking policies and stopping conditions. Extensive experiments on MNIST handwritten digit database have been conducted to show that the proposed algorithm is much faster than Keerthi et al.’s improved SMO, about 9 times. Combined with principal component analysis, the total training for ten one-against-the-rest classifiers on MNIST took just 0.77 hours. The promising scalability of the proposed scheme can make it possible to apply SVM to a wide variety of problems in engineering.