Among other grain-based foods, rice is an important and favorite food of Pakistan. Its demand has increased significantly in the recent era. The type of rice grains is very important for export and import. Therefore, it becomes necessary to distinguish rice types to avoid their fraudulent labeling. With similar aims, a dataset consisting of seven rice varieties, i.e., Basmati 2000, Chenab basmati, KSK 133, Kissan basmati, KSK 434, PK 1121 aromatic, and Punjab basmati, mostly cultivated in Pakistan have been collected and used for rice seed classification. Images of rice seed have been obtained using a self-designed setup, which consists of different seeded, i.e., 1-seeded, 5-seeded, 10-seeded, 15-seeded, and 20-seeded, images. A new model has been designed specifically for rice seed classification using convolutional neural networks, which consists of 18 layers. Standard evaluation measures based on results obtained through the proposed model have been evaluated and compared with both recent state-of-the-art published studies, and state-of-the-art CNN models, i.e., VGG-19, ResNet50, and GoogleNet (Inception-V3). Experiments proved that the proposed model achieved a perfect classification rate, i.e., 100% for each of the Pakistani grown rice seeds. This study is currently integrated with the agro-technology industry for the auto-classification of rice kernel seeds as a demo version.