Vision Detection Method for Picking Robots Based on Improved Faster R-CNN

LI Cuiming, YANG Ke, SHEN Tao, SHANG Zhengyu

Abstract

To address the issue of poor detection and positioning capabilities of fruit picking robots in scenes with densely distributed targets and fruits occluding each other, a method to improve the fruit detection and positioning of Faster R-CNN was proposed by introducing an efficient channel attention mechanism(ECA)and a multiscale feature fusion pyramid(FPN). Firstly,the commonly used VGG16 network was replaced with a ResNet50 residual network with strong expression capability and eliminate network degradation problem,thus extracting more abstract and rich semantic information to enhance the models detection ability for multiscale and small targets. Secondly,the ECA module was introduced to enable the feature extraction network to focus on local and efficient information in the feature map,reduce the interference of invalid targets,and improve the model's detection accuracy. Finally,a branch and leaf grafting data augmentation method was used to improve the apple dataset and solve the problem of insufficient image data. Based on the constructed dataset,genetic algorithms were used to optimize K-means++ clustering and generate adaptive anchor boxes. Experimental results showed that the improved model had average precision of 96.16% for graspable apples and 86.95% for non-graspable apples,and the mean average precision was 92.79%,which was 15.68 percentages higher than that of the traditional Faster R-CNN. The positioning accuracy for graspable and non-directly graspable apples were 97.14% and 88.93%, respectively,which were 12.53 percentages and 40.49 percentages higher than that of traditional Faster R-CNN. The weight was reduced by 38.20%. The computation time was reduced by 40.7%. The improved model was more suitable for application in fruit-picking robot visual systems.


Keywords: apple picking robot, target localization and detection, Faster R-CNN, attention mechanism, feature pyramid

 

Download Full Text:

PDF


References


SONG Huaibo,SHANG Yuying,HE Dongjian. Review on deep learning technology for fruit target recognition [ J ]. Transactions of the Chinese Society for Agricultural Machinery,2023,54( 1 ) :1 - 19. (in Chinese)

GAO F, FU L, ZHANG X,et al. Multi-class fruit-on-plant detection for apple in SNAP system using Faster R — CNNi [J]. Computers and Electronics in Agriculture ,2020,176 :105634.

LIANG C,XIONG J,ZHENG Z,et al. A visual detection method for nighttime litchi fruits and fruiting stems[J]. Computers and Electronics in Agriculture ,2020 ,169 ; 105192.

LUO Huilan,CHEN Hongkun. Survey of object detection based on deep learning [J] . Acta Electronica Sinica,2020 ,48( 6) : 1230 - 1239. (in Chinese)

WANG Pengxin,TIAN Huiren,ZHANG Yue,et al. Crop growth monitoring and yield estimation based on deep learning: state of the art and beyond [ J ]. Transactions of the Chinese Society for Agricultural Machinery ,2022 ,53 (2 ) ;1 - 14. (in Chinese)

XU W,ZHAO L,LI J,et al. Detection and classification of tea buds based on deep learning J . Computers and Electronics in Agriculture,2022 ,192 :106547.

ANAND K,KERR В W,WANG Zhenglin,et al. Deep learning method overview and review of use for fruit detection and yield estimation [J] . Computers and Electronics in Agriculture,2019 ,162 :219 -234.

SUN Ying, HOU Zhiqiang, YANG Chen,et al. Object detection algorithm based on dual-modal fusion network [J]. Acta Photonica Sinica ,2023 ,52 ( 1 ) : 203 -215. (in Chinese)

FU L,MAJEED Y,ZHANG X,et al. Faster R - CNN-based apple detection in dense-foliage fruiting-wall trees using RGB and depth features for robotic harvesting [ J ]. Biosystems Engineering,2020 ,197 :245.

YU Y,ZHANG K,YANG L,et al. Fruit detection for strawberry harvesting robot in non-structural environment based on Mask— RCNN J . Computers and Electronics in Agriculture,2019 ,163 :104846.

ZHAO Dean, WU Rendi,LIU Xiaoyang,et al. Apple positioning based on YOLO deep convolutional neural network for picking robot in complex background [ J ]. Transactions of the CSAE ,2019,35 ( 3 ) ; 164 - 173. (in Chinese)

MAZZIA V, KHALIQ A,SALVETTI F, et al. Real-time apple detection system using embedded systems with hardware accelerators [ J ]. An Edge AI Application. IEEE Access ,2020 ,8 ;9102.

WU L,MA J,ZHAO Y, et al. Apple detection in complex scene using the improved YOLOv4 model [ J ]. Agronomy, 2021 , 11(3) :476.

CHU P, LI Z, LAMMERS K,et al. Deep learning-based apple detection using a suppression Mask R - CNN J]. Pattern Recognition Letters,2021 ,147:206.

SUN Jun.QIAN Lei,ZHU Weidong,et al. Apple detection in complex orchard environment based on improved RetinaNet [ J]. Transactions of the CSAE ,2022 ,38( 15 ) ;314 -322. (in Chinese)

ZHU Hongchun,LI Xu,MENG Yang,et al. Tea hud detection based on Faster R - CNN network [ J]. Transactions of the Chinese Society for Agricultural Machinery ,2022 ,53 (5 ) : 217 -224. (in Chinese)

HE K,ZHANG X, REN S,et al. Deep residual learning for image recognition [С ]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition ,2016.

IIU J,SHEN L,SUN G. Squeeze-and-excitation networks[C] // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition ,2018.

WANG Q, WU B, ZHU P, et al. ECA - Net: efficient channel attention for deep convolutional neural networks [С] // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition ,2020.

LIN T Y, DOLLAR P, GIRSHICK R, et al. Feature pyramid networks for object detection [ С ]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition ,2017.


Refbacks

  • There are currently no refbacks.