Research Article | Open Access

Volume 2026 |Article ID 100106 | https://doi.org/10.1016/j.plaphe.2025.100106

IPENS: Interactive unsupervised framework for rapid plant phenotyping extraction via NeRF-SAM2 fusion

Wentao Song,^1,2 He Huang,¹ Fang Qu,^1,2 Jiaqi Zhang,^1,2 Longhui Fang,^1,3 Yuwei Hao,^1,2 Chenyang Peng,^1,4 and Youqiang Sun ¹

¹Hefei Institutes of Physical Science, Chinese Academy of Sciences, Hefei, 230031, China
²University of Science and Technology of China, Hefei, 230026, China
³Anhui Agricultural University, Hefei, China
⁴Anhui Jianzhu University, Hefei, China

Abstract

Advanced plant phenotyping technologies are vital for trait improvement and accelerating intelligent breeding. Due to the species diversity of plants, existing methods heavily rely on large-scale high-precision manually annotated data. For self-occluded objects at the grain level, unsupervised methods often prove ineffective. This study proposes IPENS, an interactive unsupervised multi-target point cloud extraction method. It utilizes radiance field information to lift 2D masks, segmented by SAM2 (Segment Anything Model 2), into 3D space for target point cloud extraction. A multi-target collaborative optimization strategy addresses the challenge of segmenting multiple targets from a single interaction. On a rice dataset, IPENS achieves a grain-level segmentation mean Intersection over Union (mIoU) of 63.72 %. For phenotypic trait estimation, it achieves a grain voxel volume coefficient of determination R2 = 0.7697 (Root Mean Square Error, RMSE = 0.0025), leaf surface area R2 = 0.84 (RMSE = 18.93), and leaf length and width prediction accuracies of R2 = 0.97 and R2 = 0.87 (RMSE = 1.49 and 0.21). On a wheat dataset, IPENS further improves segmentation performance to a mIoU of 89.68 %, with exceptional phenotypic estimation results: panicle voxel volume R2 = 0.9956 (RMSE = 0.0055), leaf surface area R2 = 1.00 (RMSE = 0.67), and leaf length and width predictions reaching R2 = 0.99 and R2 = 0.92 (RMSE = 0.23 and 0.15). Without requiring annotated data, IPENS rapidly extracts grain-level point clouds for multiple targets within 3 min using single-round image interactions. These features make IPENS a high-quality, non-invasive phenotypic extraction solution for rice and wheat, offering significant potential to enhance intelligent breeding.