Computer-Assisted Real-Time Rice Variety Learning Using Deep Learning Network

doi:10.1016/j.rsci.2022.02.003

Abstract

Abstract:

Due to the inconsistency of rice variety, agricultural industry faces an important challenge of rice grading and classification by the traditional grading system. The existing grading system is manual, which introduces stress and strain to humans due to visual inspection. Automated rice grading system development has been proposed as a promising research area in computer vision. In this study, an accurate deep learning-based non-contact and cost-effective rice grading system was developed by rice appearance and characteristics. The proposed system provided real-time processing by using a NI-myRIO with a high-resolution camera and user interface. We firstly trained the network by a rice public dataset to extract rice discriminative features. Secondly, by using transfer learning, the pre-trained network was used to locate the region by extracting a feature map. The proposed deep learning model was tested using two public standard datasets and a prototype real-time scanning system. Using AlexNet architecture, we obtained an average accuracy of 98.2% with 97.6% sensitivity and 96.4% specificity. To validate the real-time performance of proposed rice grading classification system, various performance indices were calculated and compared with the existing classifier. Both simulation and real-time experiment evaluations confirmed the robustness and reliability of the proposed rice grading system.

Key words: deep learning algorithm, rice defect classification, computer vision, agriculture, automated visual grading system

Pandia Rajan Jeyaraj, Siva Prakash Asokan, Edward Rajan Samuel Nadar. Computer-Assisted Real-Time Rice Variety Learning Using Deep Learning Network[J]. Rice Science, 2022, 29(5): 489-498.

Figures/Tables 14

References 32

[1]	Abinaya N S, Susan D, Rakesh Kumar S. 2021. Naive Bayesian fusion based deep learning networks for multisegmented classification of fishes in aquaculture industries. Ecol Inform, 61: 101248.
[2]	Alfred R, Obit J H, Chin C P Y, Haviluddin H, Lim Y. 2021. Towards paddy rice smart farming: A review on big data, machine learning, and rice production tasks. IEEE Access, 9: 50358-50380.
[3]	Cao J J, Sun T, Zhang W R, Zhong M, Huang B, Zhou G M, Chai X J. 2021. An automated zizania quality grading method based on deep classification model. Comput Electron Agric, 183: 106004.
[4]	Chen P, Li W L, Yao S J, Ma C, Zhang J, Wang B, Zheng C H, Xie C J, Liang D. 2021. Recognition and counting of wheat mites in wheat fields by a three-step deep learning method. Neurocomputing, 437: 21-30.
[5]	Guerrero J M, Ruz J J, Pajares G. 2017. Crop rows and weeds detection in maize fields applying a computer vision system based on geometry. Comput Electron Agric, 142: 461-472.
[6]	Hu Z L, Tang J S, Zhang P, Jiang J F. 2020. Deep learning for the identification of bruised apples by fusing 3D deep features for apple grading systems. Mech Syst Signal Process, 145: 106922.
[7]	Ismail N, Malik O A. 2022. Real-time visual inspection system for grading fruits using computer vision and deep learning techniques. Inf Process Agric, 9(1): 24-37.
[8]	Jeyaraj P R, Nadar E R S. 2019. Deep Boltzmann machine algorithm for accurate medical image analysis for classification of cancerous region. Cogn Comput Syst, 1(3): 85-90.
[9]	Jeyaraj P R, Nadar E R S. 2020a. High-performance dynamic magnetic resonance image reconstruction and synthesis employing deep feature learning convolutional networks. Int J Imaging Syst Technol, 30(2): 380-390.
[10]	Jeyaraj P R, Nadar E R S. 2020b. Dynamic image reconstruction and synthesis framework using deep learning algorithm. IET Image Process, 14(7): 1219-1226.
[11]	Jo H W, Lee S, Park E, Lim C H, Song C, Lee H, Ko Y, Cha S, Yoon H, Lee W K. 2020. Deep learning applications on multitemporal SAR (Sentinel-1) image classification using confined labeled data: The case of detecting rice paddy in South Korea. IEEE Trans Geosci Remote Sens, 58(11): 7589-7601.
[12]	Kaur H, Sawhney B K, Jawandha S K. 2018. Evaluation of plum fruit maturity by image processing techniques. J Food Sci Technol, 55(8): 3008-3015. PMID
[13]	Koklu M, Cinar I, Taspinar Y S. 2021. Classification of rice varieties with deep learning methods. Comput Electron Agric, 187: 106285.
[14]	Kucuk Ç, Taşkın G, Erten E. 2016. Paddy-rice phenology classification based on machine-learning methods using multitemporal co-polar X-band SAR images. IEEE J Sel Top Appl Earth Obs Remote Sens, 9(6): 2509-2519.
[15]	Kuo T Y, Chung C L, Chen S Y, Lin H G, Kuo Y F. 2016. Identifying rice grains using image analysis and sparse-representation-based classification. Comput Electron Agric, 127: 716-725.
[16]	Marimuthu S, Roomi S M M. 2017. Particle swarm optimized fuzzy model for the classification of banana ripeness. IEEE Sens J, 17(15): 4903-4915.
[17]	Mittal S, Dutta M K, Issac A. 2019. Non-destructive image processing based system for assessment of rice quality and defects for classification according to inferred commercial value. Measurement, 148: 106969.
[18]	Nandi C S, Tudu B P, Koley C. 2016. A machine vision technique for grading of harvested mangoes based on maturity and quality. IEEE Sens J, 16(16): 6387-6396.
[19]	Ren S Q, He K M, Girshick R, Sun J. 2017. Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell, 39(6): 1137-1149.
[20]	Shamim Hossain M, Al-Hammadi M, Muhammad G. 2019. Automatic fruit classification using deep learning for industrial applications. IEEE Trans Ind Inform, 15(2): 1027-1034.
[21]	Singh K R, Chaudhury S. 2016. Efficient technique for rice grain classification using back-propagation neural network and wavelet decomposition. IET Comput Vis, 10(8): 780-787.
[22]	Su J Y, Yi D W, Su B F, Mi Z W, Liu C J, Hu X P, Xu X M, Guo L, Chen W H. 2021. Aerial visual perception in smart farming: Field study of wheat yellow rust monitoring. IEEE Trans Ind Inform, 17(3): 2242-2249.
[23]	Sun B Y, Yuan N Z, Zhao Z. 2020. A hybrid demosaicking algorithm for area scan industrial camera based on fuzzy edge strength and residual interpolation. IEEE Trans Ind Inform, 16(6): 4038-4048.
[24]	Sun J, Lu X Z, Mao H P, Jin X M, Wu X H. 2017. A method for rapid identification of rice origin by hyperspectral imaging technology. J Food Process Eng, 40(1): e12297.
[25]	Wu L H, Liu Z C, Bera T, Ding H J, Langley D A, Jenkins-Barnes A, Furlanello C, Maggio V, Tong W D, Xu J. 2019. A deep learning model to recognize food contaminating beetle species based on elytra fragments. Comput Electron Agric, 166: 105002.
[26]	Xiao G Y, Wu Q, Chen H, Da D, Guo J Z, Gong Z G. 2020. A deep transfer learning solution for food material recognition using electronic scales. IEEE Trans Ind Inform, 16(4): 2290-2300.
[27]	Xu X L, Li W S, Duan Q L. 2021. Transfer learning and SE-ResNet152 networks-based for small-scale unbalanced fish species identification. Comput Electron Agric, 180: 105878.
[28]	Yang H J, Pan B, Li N, Wang W, Zhang J, Zhang X L. 2021. A systematic method for spatio-temporal phenology estimation of paddy rice using time series Sentinel-1 images. Remote Sens Environ, 259: 112394.
[29]	Yang S, Zhu Q B, Huang M, Qin J W. 2017. Hyperspectral image- based variety discrimination of maize seeds by using a multi- model strategy coupled with unsupervised joint skewness-based wavelength selection algorithm. Food Anal Methods, 10(2): 424-433.
[30]	Zareiforoush H, Minaei S, Alizadeh M R, Banakar A, Samani B H. 2016. Design, development and performance evaluation of an automatic control system for rice whitening machine based on computer vision and fuzzy logic. Comput Electron Agric, 124: 14-22.
[31]	Zeng W H, Li M. 2020. Crop leaf disease recognition based on Self-Attention convolutional neural network. Comput Electron Agric, 172: 105341.
[32]	Zhang M, Zhang H Q, Li X Y, Liu Y, Cai Y T, Lin H. 2020. Classification of paddy rice using a stacked generalization approach and the spectral mixture method based on MODIS time series. IEEE J Sel Top Appl Earth Obs Remote Sens, 13: 2264-2275.

Research problem	Method used	Number of data	Research outcome	Reference
Real-time visual inspection system for grading fruits	Deep learning models including ResNet and DenseNet	360 datasets were used to grade self-collected datasets of 150 apples and 210 bananas	Average accuracies were 99.2% for apples and 98.6% for bananas	Ismail and Malik, 2022
Fish classification technique using deep learning networks with naive Bayesian type fusion	CNN-AlexNet was trained by using the transfer learning approach	15 million labelled high- resolution images categorized into 22 000 classes	Classification accuracy of 98.64% for ‘Fish-Pak’ image dataset	Abinaya et al, 2021
To identify and count wheat mites from digital images	Three-step deep learning method by Region Proposal Network	256-d vector was generated from presented digital images	Accuracies of 94.6% and 96.4% for two different variants in architecture	Chen et al, 2021
To provide a solution for food material recognition employing deep transfer learning	An embedded partial-and- imbalanced domain adaptation technique in the deep learning model	Office-31 and Meal-300 with 31 classes of each	Average accuracy of 97.71% for two step process and 96.28% for shared network	Xiao et al, 2020
To detect and identify bruised apples by fusing 3D deep features	An apple grading system was designed by fusing multi- dimensional features	ImageNet dataset of 1.28 million images over 1 000 generic classes of apples	Identification rate of 97.67% for bruised apple	Hu et al, 2020
To aerial visual perception for yellow rust disease monitoring in winter wheat	An automated rust disease monitoring framework was proposed by U-Net deep learning network	Five GeoTIFF images of the covered area with cropped region of interest	An average accuracy of 91.3%, recall of 92.6% with 0.92 F₁-score	Su et al, 2021
To classify fruit automatically I, supermarket employing deep learning model	Fine-tuned visual geometry group-16 pre-trained deep convolution neural network	Two color image datasets of different fruit images	Classification accuracies of 99.49% and 99.75% for datasets 1 and 2, respectively	Shamim Hossain et al, 2019

Research problem	Method used	Number of data	Research outcome	Reference
Real-time visual inspection system for grading fruits	Deep learning models including ResNet and DenseNet	360 datasets were used to grade self-collected datasets of 150 apples and 210 bananas	Average accuracies were 99.2% for apples and 98.6% for bananas	Ismail and Malik, 2022
Fish classification technique using deep learning networks with naive Bayesian type fusion	CNN-AlexNet was trained by using the transfer learning approach	15 million labelled high- resolution images categorized into 22 000 classes	Classification accuracy of 98.64% for ‘Fish-Pak’ image dataset	Abinaya et al, 2021
To identify and count wheat mites from digital images	Three-step deep learning method by Region Proposal Network	256-d vector was generated from presented digital images	Accuracies of 94.6% and 96.4% for two different variants in architecture	Chen et al, 2021
To provide a solution for food material recognition employing deep transfer learning	An embedded partial-and- imbalanced domain adaptation technique in the deep learning model	Office-31 and Meal-300 with 31 classes of each	Average accuracy of 97.71% for two step process and 96.28% for shared network	Xiao et al, 2020
To detect and identify bruised apples by fusing 3D deep features	An apple grading system was designed by fusing multi- dimensional features	ImageNet dataset of 1.28 million images over 1 000 generic classes of apples	Identification rate of 97.67% for bruised apple	Hu et al, 2020
To aerial visual perception for yellow rust disease monitoring in winter wheat	An automated rust disease monitoring framework was proposed by U-Net deep learning network	Five GeoTIFF images of the covered area with cropped region of interest	An average accuracy of 91.3%, recall of 92.6% with 0.92 F₁-score	Su et al, 2021
To classify fruit automatically I, supermarket employing deep learning model	Fine-tuned visual geometry group-16 pre-trained deep convolution neural network	Two color image datasets of different fruit images	Classification accuracies of 99.49% and 99.75% for datasets 1 and 2, respectively	Shamim Hossain et al, 2019

Dataset	Orientation number	Defected rice image number	Healthy rice image number
Kaggle-labelled rice	107	520	230
Data-world Asian variety	219	425	190
CASC	187	182	105

Dataset	Orientation number	Defected rice image number	Healthy rice image number
Kaggle-labelled rice	107	520	230
Data-world Asian variety	219	425	190
CASC	187	182	105

Paddy rice defect class	No. of images selected	No. of standard images	Multiplication factor	No. of total images for augmentation	New image count
Broken rice	30	10	3	120	160
Black spot rice	50	25	4	300	375
Wrinkled rice	25	15	2	80	120
Total	105	50		500	655