The chili plant is one of the high economic value vegetable commodities which has the potential to continue to grow. On the other hand, this plant’s production still has obstacles such as pests and disease. Identifying the pest and diseases earlier is needed to protect against these problems. In this work, image recognition technology is applied to recognize the pest and diseases of the chili plant. First, the image of healty leaves and infected leaves by P. Latus, B. Tabaci, Gemini virus, and Cercospora sp are collected. The next step is image prepocessing, including cropping and resizing to make uniform image format and background removal to reduce background effects in image processing. Red-Green-Blue (RGB) input images are changed to grayscale images to give input one color channel. The images are extracted to get features using Gray Level Co-occurrence Matrix (GLCM) method. Extracted texture features involve contrast, correlation, energy, homogeneity, and dissimilarity with average angle values are 0°, 45°, 90° and 135°. Furthermore, all obtained features are classified into five classes using K-Means Clustering Algorithm. Extraction and segmentation results are used as parameters in the classification process using a Support Vector Machine (SVM). In this work, the result of this process is 82%.