Due to its tremendous commercial potential, sports video has become a popular research topic nowadays. As the bridge of low-level features and high-level semantic contents, automatic shot clustering is an important issue in the field of sports video content analysis. In previous work, many clustering approaches need some professional knowledge of videos, some experimental parameters, or some thresholds to obtain good clustering results. In this article, we present a new efficient shot clustering algorithm for sports video which is generic and does not need any prior domain knowledge. The novel algorithm, which is called Valid Dimension Clustering(VDC), performs in an unsupervised manner. For the high-dimensional feature vectors of video shots, a new dimensionality reduction approach is proposed first, which takes advantage of the available dimension histogram to get ”valid dimensions” as a good approximation of the intrinsic characteristics of data. Then the clustering algorithm performs on valid dimensions one by one to furthest utilize the intrinsic characteristics of each valid dimension. The iterations of merging and splitting of similar shots on each valid dimension are repeated until the novel stop criterion which is designed inheriting the theory of Fisher Discriminant Analysis is satisfied. At last, we apply our algorithm on real video data in our extensive experiments, the results show that VDC has excellent performance and outperforms other clustering algorithms.