Locally Linear Embedding (LLE) is a widely used non-linear dimensionality reduction (NLDR) method that projects multi-dimensional data into a low-dimensional embedding space while attempting to preserve object adjacencies from the original high-dimensional feature space. A limitation of LLE, however, is the presence of free parameters, changing the values of which may dramatically change the low dimensional representations of the data. In this paper, we present a novel Consensus-LLE (C-LLE) scheme which constructs a stable consensus embedding from across multiple low dimensional unstable LLE data representations obtained by varying the parameter (κ) controlling locally linearity. The approach is analogous to Breiman’s Bagging algorithm for generating ensemble classifiers by combining multiple weak predictors into a single predictor. In this paper we demonstrate the utility of C-LLE in creating a low dimensional stable representation of Magnetic Resonance Spectroscopy (MRS) data for identifying prostate cancer. Results of quantitative evaluation demonstrate that our C-LLE scheme has higher cancer detection sensitivity (86.90%) and specificity (85.14%) compared to LLE and other state of the art schemes currently employed for analysis of MRS data.