Abstract  This paper proposes a new framework for multi-component images segmentation which plays an increasing role in many imagery applications like astronomy, medicine, remote sensing, chemistry, biology etc. In fact, inference on such images is a very difficult task when the number of components increases due to the well-known Hughes phenomenon. A common solution is to reduce dimensionality, keeping only relevant information before segmentation. Linear models usually fail with complex data structure, and mixture of linear models, each of which modeling a local cluster of the data, is more suitable. Moreover, a probabilistic formulation based on linear latent variable models allows efficient solution using a maximum-likelihood-based decision to recover the clusters. However, for multi-component image classification, this is not enough because it completely neglects the spatial positions of the multi-dimensional pixels on the lattice. Therefore, we propose to consider the neighborhood by introducing a Markovian a priori to efficiently regularize pixel classification. As a consequence, segmentation and reduction are performed simultaneously in an efficient and robust way. In this paper, we focus on the Probabilistic Principal Component Analysis (PPCA) as a latent variable model, and the Hidden Markov quad-Tree (HMT) as an a priori for regularization. The method performs well both on synthetic and real remote sensing and Stokes–Mueller images.