Research Article | Open Access

Volume 2023 |Article ID 0041 | https://doi.org/10.34133/plantphenomics.0041

Self-Supervised Plant Phenotyping by Combining Domain Adaptation with 3D Plant Model Simulations: Application to Wheat Leaf Counting at Seedling Stage

Yinglun Li,¹ Xiaohai Zhan,¹ Shouyang Liu ,¹ Hao Lu ,² Ruibo Jiang,¹ Wei Guo,³ Scott Chapman,⁴ Yufeng Ge,⁵ Benoit de Solan,^6,7 Yanfeng Ding,¹ Frédéric Baret^1,6

¹Plant Phenomics Research Centre, Academy for Advanced Interdisciplinary Studies, Jiangsu Collaborative Innovation Center for Modern Crop Production, Nanjing Agricultural University, Nanjing, China
²Key Laboratory of Image Processing and Intelligent Control, School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, Wuhan, China
³Graduate School of Agricultural and Life Sciences, The University of Tokyo, 1-1-1 Midori-cho, Nishitokyo City, Tokyo, Japan
⁴School of Agriculture and Food Sciences, The University of Queensland, St. Lucia, Queensland 4072, Australia
⁵Department of Biological Systems Engineering, University of Nebraska-Lincoln, Lincoln, Nebraska 68583, United States
⁶INRAE, Avignon Université, UMR EMMAH, UMT CAPTE, 228, route de l’aérodrome - CS 40509, 84914 Avignon Cedex 9, France
⁷ARVALIS Institut du Végétal, 3 Rue Joseph et Marie Hackin, 75116 Paris, France

Abstract

The number of leaves at a given time is important to characterize plant growth and development. In this work, we developed a high-throughput method to count the number of leaves by detecting leaf tips in RGB images. The digital plant phenotyping platform was used to simulate a large and diverse dataset of RGB images and corresponding leaf tip labels of wheat plants at seedling stages (150,000 images with over 2 million labels). The realism of the images was then improved using domain adaptation methods before training deep learning models. The results demonstrate the efficiency of the proposed method evaluated on a diverse test dataset, collecting measurements from 5 countries obtained under different environments, growth stages, and lighting conditions with different cameras (450 images with over 2,162 labels). Among the 6 combinations of deep learning models and domain adaptation techniques, the Faster-RCNN model with cycle-consistent generative adversarial network adaptation technique provided the best performance (R2 = 0.94, root mean square error = 8.7). Complementary studies show that it is essential to simulate images with sufficient realism (background, leaf texture, and lighting conditions) before applying domain adaptation techniques. Furthermore, the spatial resolution should be better than 0.6 mm per pixel to identify leaf tips. The method is claimed to be self-supervised since no manual labeling is required for model training. The self-supervised phenotyping approach developed here offers great potential for addressing a wide range of plant phenotyping problems. The trained networks are available at https://github.com/YinglunLi/Wheat-leaf-tip-detection.