There exist two main solutions for the classification of high-dimensional data with small number settings. One is to classify them directly in high-dimensional space with regularization methods, and the other is to reduce data dimension first, then classify them in feature space. However, which is better on earth? In this paper, the comparative studies for regularization and dimension reduction approaches are given with two typical sets of high-dimensional data from real world: Raman spectroscopy signals and stellar spectra data. Experimental results show that in most cases, the dimension reduction methods can obtain acceptable classification results, and cost less computation time. When the training sample number is insufficient and distribution is unbalance seriously, performance of some regularization approaches is better than those dimension reduction ones, but regularization methods cost more computation time.