Bag-of-visual words (BoVW) becomes the most popular approach for local features extraction in image classification. A large number of keypointslead to high computational costs of visual words generation. Iterative keypoint selection (IKS) as the baseline method has been proposed to reduce the number of keypoints by selecting the representative keypoints. However, random initial keypoint of IKS leadsto non-repeatable results. Thus, this paper proposes a distance matrix based keypoint selection (DMKS) algorithm to reduce the number of keypoints.The novelty of this algorithm is the number of representative keypoints can be adjusted to obtain a high classification accuracyand the algorithm does not need random initial keypoints.The performance of proposed algorithm is then compared with that of IKS1 and IKS2. Support vector machine (SVM), K-Nearest Neighbor(KNN), and deep learning (DL H2O)classifiersare used to evaluate the algorithms on the public datasets. On Coil-100 and Caltech-101 datasets, DMKS achieves classification accuracy of 90.22% and 41.99%, respectively.Although the accuracy of DMKS is slightly increase, the processing time of DMKS is faster than the baseline methods. DMKS also produces a smaller number of representative keypoints, thereforeDMKScan reduce the time-consuming of visual words generationmore effectivelyin BoVW model.