This paper presents a novel approach of distinguishing in-vocabulary (IV) words and out-of-vocabulary (OOV) words by using confidence score-based unsupervised incremental adaptation. The unsupervised adaptation uses Viterbi decode results which have high confidence scores to adjust new acoustic models. The adjusted acoustic models can award IV words and punish OOV words in confidence score, thus obtain the goal of separating IV and OOV words. Our Automatic Speech Recognition Laboratory has developed a Speech Recognition Developer Kit (SRDK) which serves as a baseline system for different speech recognition tasks. Experiments conducted on the SRDK system have proved that this method can achieve a rise over 41% in OOV words detection rate (from 68% to 96%) at the same cost of a false alarm (taken IV words as OOV words) rate of 10%. This method also obtains a rise over 11% in correct acceptance rate (from 88% to 98%) at the same cost of a false acceptance rate of 20%.