Soil fertility is conventionally evaluated by soil properties such as C, N, and P contents. Evaluation of soil fertility is now becoming a routine work for soil management and crop production. However, laboratory-analysis based determination of soil properties is time and cost consuming, which is not suitable for precision agriculture. Here, infrared spectroscopy (IR) appears as an alternative and fast technique to measure soil fertility. The IR transmission method is generally used in soil qualitative analysis, while the IR reflectance can be used in soil quantitative analysis, and most of soil-related research is focused on reflectance spectroscopy. Infrared reflectance spectra, including diffuse reflectance spectra and total attenuated reflectance spectra, are involved in soil quantitative analysis. We observe an excellent performance of predicting soil C and N contents using IR spectra. Moreover, in most of cases the predictions of the contents of soil P, K, Ca, Mg, S, and some other microelements are satisfactory. Soil water, soil clays, and soil microbes can also be characterized and evaluated using IR spectroscopy. In recent years, a new method named infrared photoacoustic spectra was applied in soil analysis. Infrared-photoacoustic spectra is indeed more convenient for sample pretreatment and spectra recording, and the recorded soil spectra contain more useful information versus conventional reflectance spectroscopy. Though currently the application of infrared photoacoustic spectroscopy in soil analysis is limited, it appears promising to measure soil fertility. The application of infrared spectroscopy in soil fertility is largely dependent on spectra pretreatment and multivariate calibration due to strong interferences in the spectra. Partial least square (PLS) and artificial neural network (ANN) are two widely used mathematical tools in the prediction of soil properties, and more mathematical tools combined models will benefit the prediction performance. To make full use of soil infrared spectra, soil spectra library construction is needed in future, and a standard procedure should be first decided in the construction. Based on soil infrared spectra library soil fertility can be fast evaluated combining suitable mathematical model, which will play an important role in the sustainable agriculture.