近日，华中农业大学柑橘全程机械化平台和人工智能与统计学习团队以“Predicting and Visualizing Citrus Colour Transformation Using a Deep Mask-Guided Generative Network”为题在农艺学领域期刊Plant Phenomics发表研究论文。该研究通过生成式深度学习方法，实现了对柑橘果皮颜色变化的高精度可视化预测。
柑橘果皮颜色是果实发育的良好指标，因此监测和预测其颜色变化可以帮助农作物的管理和收获进行决策。研究观察了107个脐橙样本，在其果皮颜色转变期间采集了7535张柑橘图像，构建了首个柑橘转色数据集。研究提出了一种将视觉感知融入深度学习的网络框架，包括分割网络、生成网络和感知损失网络；生成网络中的嵌入层将图像特征和时间信息进行了有效融合，使得该网络模型能够根据输入图像和不同的时间间隔来生成果皮颜色转变的预测图像。
为了方便现实场景中的应用，研究团队将该模型移植到了安卓设备APP上，通过手机相机拍摄柑橘图像，并在APP中输入感兴趣的时间间隔即可完成预测。该研究成果预测柑橘果皮颜色的变化并使其可视化，为柑橘果园管理实践提供有效帮助，并为其他水果作物的研究提供了借鉴和扩展的可能性。
我校理学院硕士研究生鲍泽韩和副教授李伟夫为本论文共同第一作者，工学院副研究员陈耀晖和海南大学生物医学工程学院副教授肖驰为共同通讯作者。理学院教授陈洪和日本RIKEN研究所研究员Vijay John参与了论文的研究和指导工作。
【英文摘要】
Citrus rind colour is a good indicator of fruit development, and methods to monitor and predict colour transformation therefore help the decisions of crop management practices and harvest schedules. This work presents the complete workflow to predict and visualize citrus colour transformation in the orchard featuring high accuracy and fidelity. 107 sample Navel oranges were observed during the colour transformation period, resulting in a dataset containing 7535 citrus images. A framework is proposed that integrates visual saliency into deep learning, and it consists of a segmentation network, a deep mask-guided generative network, and a loss network with manually-designed loss functions. Moreover, the fusion of image features and temporal information enables one single model to predict the rind colour at different time intervals, thus effectively shrinking the number of model parameters. The semantic segmentation network of the framework achieves the mean intersection over union score of 0.9694, and the generative network obtains a peak signal-to-noise ratio of 30.01 and a mean local style loss score of 2.710, which indicate both high quality and similarity of the generated images and are also consistent with human perception. To ease the applications in the real world, the model is ported to an Android-based application for mobile devices. The methods can be readily expanded to other fruit crops with a colour transformation period. The dataset and the source code are publicly available at Github.
