Analysis of visible–near infrared spectral characteristics for water layer management of rice based on the big data platform

The objective of this study was to evaluate the potential of big data technology for spectral feature extraction rice plants under different water management. In this study, we performed reflectance spectral analysis of rice grown under different water management schedules based on big data and machine learning approaches. We applied water management at the tillering, jointing, and heading stages of rice growth and collected visible to near-infrared reflectance spectra to analyze the spectral characteristics of rice grown under different water management schedules. Then, we constructed a data mining model based on the spectral analysis results using the Hadoop and Spark frameworks, and analyzed the characteristic bands of rice grown under each schedule using a parallel machine learning algorithm run in both local and cluster modes. The ChiSqSelector algorithm showed characteristic bands in the range of 400–1000 nm, with bands at approximately 474, 558, 632, 735, and 855 nm for water management in the tillering stage; 472, 551, 627, 721, and 843 nm in the jointing stage; and 471, 545, 642, 725, and 849 nm in the heading stage. By contrast, the UnivariateFeatureSelector algorithm showed characteristic bands at approximately 462, 665, 755, 833, and 937 nm in the tillering stage; 475, 671, 744, 848, and 932 nm in the jointing stage; and 470, 678, 757, 857, and 942 nm in the heading stage. These results demonstrate the feasibility and efficiency of applying big data and data mining technologies for spectral analysis of rice grown under different water management schedules.