img

QQ群聊

img

官方微信

高级检索

黄金科学技术 ›› 2023, Vol. 31 ›› Issue (6): 953-963.doi: 10.11872/j.issn.1005-2518.2023.06.120

• 采选技术与矿山管理 • 上一篇    下一篇

基于加权双向特征融合的矿区道路落石检测

顾清华1,2(),杜艺凡1(),李萍丰3,王丹2   

  1. 1.西安建筑科技大学资源工程学院,陕西 西安 710055
    2.西安市智慧工业感知计算与决策重点实验室,陕西 西安 710055
    3.宏大爆破工程集团有限责任公司,广东 广州 511300
  • 收稿日期:2023-08-22 修回日期:2023-11-28 出版日期:2023-12-31 发布日期:2024-01-26
  • 通讯作者: 杜艺凡 E-mail:qinghuagu@126.com;3389869409@qq.com
  • 作者简介:顾清华(1981-),男,山东潍坊人,教授,从事矿山智能科学与工程研究工作。qinghuagu@126.com
  • 基金资助:
    国家自然科学基金项目“金属露天矿无人驾驶多工序多目标协同智能调度方法研究”(52074205);陕西省杰出青年基金项目“时空路况下金属露天矿无人驾驶多车协同智能调度集成建模”(2020JC-44)

Road Rockfall Detection in Mining Area Based on Weighted Bidirectional Feature Fusion

Qinghua GU1,2(),Yifan DU1(),Pingfeng LI3,Dan WANG2   

  1. 1.School of Resources Engineering, Xi’an University of Architecture and Technology, Xi’an 710055, Shaanxi, China
    2.Xi’an Key Laboratory of Intelligent Industry Perception Computing and Decision Making, Xi’an 710055, Shaanxi, China
    3.Hongda Blasting Engineering Group Ltd. , Co. , Guangzhou 511300, Guangdong, China
  • Received:2023-08-22 Revised:2023-11-28 Online:2023-12-31 Published:2024-01-26
  • Contact: Yifan DU E-mail:qinghuagu@126.com;3389869409@qq.com

摘要:

为了解决当前露天矿区非结构化路面落石检测存在的环境复杂、落石尺寸差异较大以及落石与非结构化路面颜色相近而造成难以精准识别的问题,提出了一种基于加权双向特征融合的矿区道路落石检测模型。首先,通过加入SimAM 注意力机制有效抑制背景环境的干扰;其次,使用加权双向特征金字塔(BiFPN)结构实现多尺度特征融合,增强模型对于不同尺寸落石的特征提取能力;最后,引入轻量级卷积GSConv模块,通过减少模型计算量来提升模型检测速度。试验结果表明,该算法的检测精度均值达到92.8%,检测速度达到63.1FPS,能够实现矿区非结构化路面落石的实时高精度检测,为无人矿卡的安全行驶提供了保障。

关键词: 矿区道路, 无人驾驶, 机器视觉, 落石检测, 多尺度特征融合, YOLOv5模型

Abstract:

With the booming development of big data and Internet of Things technology,traditional mines have developed to smart mines and intelligent mines,and unmanned technology has been gradually applied to mining areas.In order to solve the problem that the rockfall detection of unstructured road in open-pit mine area has complex environment,large difference in rockfall size and similar color between rockfall and unstructured road surface,a rockfall detection model of mining road based on weighted bidirectional feature fusion was proposed.First,the SimAM attention mechanism is added to the backbone network,this attention mechanism is different from the previous channel attention mechanism and spatial attention mechanism,it can effectively eliminate the interference of the background environment without adding additional parameters,so that the model can focus more on the target characteristics of rockfall.Second,the weighted bidirectional feature pyramid(BiFPN)structure was used to realize multi-scale feature fusion in the neck.Since the PANet structure in the YOLOv5s network model only adds or splice the characteristics of the pyramid structure in the melting process,the bidirectional feature weighting was combined with the bidirectional feature of the weight and adaptive adjustment to ensure that the network model attaches proper importance to the rock ebaissees different sizes and different levels and realizes the addition between the low-level position information and high-level semantic information for multiple cross-layer weighted feature fusion,thus enhancing the feature extracion ability of the model for rockfall of different sizes.Finally,the lightweight convolution GSConv module was introduced into the col,which can be used to process function cards at this time,not only reducing redundant information,but also avoiding compression.The GSConv lightweight convolution module is based on deep separable convolution(DSC),ordinary convolution(SC) and channel shuffle operation,which improves the detection speed of the model by effectively reducing the complexity of the model.The experimental results show that the average detection accuracy of this algorithm reaches 92.8%.and the detection speed reaches 63.1FPS.Compared with the current fastest R-CNN,YOLOv4-tiny,YOLOv7 and YOLOv5s algorithms,the average detection accuracy is increased by 17.0,13.6,3.4 and 2.5 percentiles,and the detection speed is increased by 32.2,1.4,14.6 and 2.6 FPS,respectively.Moreover,the model size of the algorithm is only 12.9 MB,which is easy to deploy on mobile devices.Therefore,the algorithm can realize the real-time and accurate detection of unstructured road rockfall in mining area,and ensure the safe driving of unmanned mining card.

Key words: mining area road, unmanned driving, machine vision, rockfall detection, multi-scale feature fusion, YOLOv5 model

中图分类号: 

  • TP391.4

图1

改进后的YOLOv5(version6.0)网络模型结构"

图2

SimAM注意力机制"

图3

2种改进方案的主干网络示意图"

图4

PANet网络结构"

图5

BiFPN网络结构"

图6

GSConv模块"

图7

落石原始图像及增强图像"

表1

试验参数设置"

参数名称参数设置参数名称参数设置
图像输入大小640×640×3初试学习率0.01
训练批次8NMS阈值0.45
迭代次数200优化器SGD

表2

检测结果分类"

实际正确实际错误
预测正确真阳性TP假阳性FP
预测错误假阴性FN真阴性TN

表3

YOLOv5不同版本的对比试验"

模型深度因子宽度因子mAP@0.5/%

检测速度

/FPS

模型大小/M
YOLOv5s0.330.590.360.513.7
YOLOv5m0.650.7590.948.541.9
YOLOv5l1191.539.591.6
YOLOv5x1.331.2591.937.2170.2

表4

SimAM注意力机制2种嵌入方式性能对比"

嵌入方式P/%R/%mAP@0.5/%检测速度/FPS
方式一91.287.191.660.8
方式二90.487.991.161.2

表5

不同轻量化方式的对比试验"

模型GhostGSConvP/%R/%mAP@0.5/%

检测速度

/FPS

模型大小

/M

087.386.788.965.79.7
190.586.490.863.912.4
289.985.990.360.513.7

表6

不同模块优化性能对比"

模型BiFPNSimAMGSConvP/%R/%mAP@0.5/%

检测速度

/FPS

089.985.990.360.5
191.588.491.858.8
291.287.191.660.8
390.586.490.863.9
492.389.192.559.2
592.188.392.262.1
691.688.491.963.3
792.589.892.863.1

图8

损失值变化对比"

表7

不同网络模型的性能对比"

检测方法P/%R/%mAP@0.5/%

检测速度

/FPS

模型大小

/M

Faster R-CNN72.867.275.830.9108.9
YOLOv4-tiny77.875.179.261.723.1
YOLOv5s89.985.990.360.513.7
YOLOv788.386.789.448.571.3
本文算法92.589.892.863.112.9

图9

不同算法模型性能对比曲线algorithm models"

图10

不同网络模型对落石的检测效果"

Cao M C, Wang J M,2020.Obstacle detection for autonomous driving vehicles with multi-lidar sensor fusion[J].Journal of Dynamic Systems Measurement and Control,142(2):021007.
Chen Ken, Ou Ou, Yang Changzhi,et al,2023.Rockfall detection method based on improved YOLOX[J].Computer Measurement and Control,31(11):53-59.
Dong L H, Wang X R, Liu B Z,et al,2021.Information acquisition incentive mechanism based on evolutionary game theory[J].Wireless Communications and Mobile Computing,2021:5525791.DOI:https://doi.org/10.1155/2021/5525791 .
doi: https://doi.org/10.1155/2021/5525791
Gao Qie, Li Denghua, Ding Yong,2024.A study on the rock block tracking algorithm that utilizes the M-DBT framework[J/OL].Hydro-Science and Engineering:1-13[2024-01-02]..
Girshick R,2015.Fast R-CNN[C]//IEEE International Conference on Computer Vision(ICCV).Santiago:IEEE: 1580 1732.
Girshick R, Donahue J, Darrell T,et al,2014.Rich feature hierarchies for accurate object detection and semantic segmentation[C]//IEEE Conference on Computer Vsion and PatternRecognition(CVPR).Columbus:IEEE: 14632381.DOI:10.1109/CVPR.2014.81 .
doi: 10.1109/CVPR.2014.81
Han K, Wang Y H, Tian Q,et al,2020.Ghostnet:More features from cheap operations[C]//IEEE Conference on Computer Vision and Pattern Recognition(CVPR).Seattle:IEEE: 19874590.DOI: 10.1109/CVPR42600.2020.00165 .
doi: 10.1109/CVPR42600.2020.00165
He K M, Zhang X Y, Ren S Q,et al,2015.Spatial pyramid pooling in deep convolutional networks for visual recognition[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,37(9):1904-1916. DOI: 10.1109/TPAMI.2015. 2389824 .
doi: 10.1109/TPAMI.2015. 2389824
Li H L, Li J, Wei H B,et al,2022.Slim-neck by GSConv:A better design paradigm of detector architectures for autonomous vehicles[J].arXiv preprint arXiv:2206.02424.DOI:https://doi.org/10.48550/arXiv.2206.02424 .
doi: https://doi.org/10.48550/arXiv.2206.02424
Lin T Y, Goyal P, Girshick R,et al,2017a.Focal loss for dense object detection[C]//IEEE International Conference on Computer Vsion(ICCV).Venice:IEEE: 19262330.DOI: 10.1109/TPAMI.2018.2858826 .
doi: 10.1109/TPAMI.2018.2858826
Lin T Y, Dollár P, Girshick R,et al,2017b.Feature pyramid networks for object detection[C]//IEEE Conference on Computer Vsion and Pattern Recognition(CVPR).Hawaii:IEEE: 17355379.DOI: 10.1109/CVPR.2017.106 .
doi: 10.1109/CVPR.2017.106
Liu B, Ding Z F, Tian L L,et al,2020.Grape leaf disease identification using improved deep convolutional neural networks[J].Frontiers in Plant Science,11:1082. DOI: https://doi.org/10.3389/fpls.2020.01082 .
doi: https://doi.org/10.3389/fpls.2020.01082
Liu B, Tian B H, Qiao J C,2022.Mine track obstacle detection method based on information fusion[C]//Journal of Physics:Conference Series.IOP Publishing,2229(1):012023. DOI: 10.1088/1742-6596/2229/1/012023 .
doi: 10.1088/1742-6596/2229/1/012023
Liu Linya, Wu Songying, Zuo Zhiyuan,et al,2021.Research on Rockfall detection method of mountain railway slope based on YOLOv3 algorithm[J].Computer Science,48(Supp.2):290-294.
Liu W, Anguelov D, Erhan D,et al,2016.Ssd:Single shot multibox detector[C]//European Conference on Computer Vision(ECCV).Amsterdam:IEEE: 21-37.
Liu Zixue, Wang Fubin, Yu Kai,2022.Research and implementa-tion of the railway rockfall detection based on modified YO-LOv3[J].High Speed Railway Technology,13(3):52-56,80.
Lu Caiwu, Qi Fan, Ruan Shunling,2020.An open-pit mine roadway obstacle warning method integrating the object detection and distance threshold model[J].Opto-Electronic Engineering,47(1):40-47.
More K S, Wolkersdorfer C,2023.Intelligent mine water management tools—eMetsi and machine learning GUI[J].Mine Water and the Environment,42(1):111-120.DOI:https://doi.org/10.1007/s10230-023-00917-7 .
doi: https://doi.org/10.1007/s10230-023-00917-7
Patel N, Krishnamurthy P, Tzes A,et al,2021.Overriding learning-based perception systems for control of autonomous unmanned aerial vehicles[C]//International Conference on Unmanned Aircraft Systems (ICUAS).Athens:IEEE: 20916280.DOI: 10.1109/ICUAS51884.2021.9476881 .
doi: 10.1109/ICUAS51884.2021.9476881
Redmon J, Farhadi A,2017.YOLO9000:Better,faster,stronger[C]//IEEE Conference on Computer Vision and Pattern Recognition(CVPR).Honolulu:IEEE: 17355115.DOI: 10.1109/CVPR.2017.690
doi: 10.1109/CVPR.2017.690
Ren S Q, He K M, Girshick R,et al,2017.Faster R-CNN:Towards real-time object detection with region proposal networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,39(6):1137-1149.
Ruan Shunling, Li Shaobo, Lu Caiwu,et al,2021.Road negative obstacle detection in open-pit mines based on multi scale feature fusion[J].Journal of China Coal Society,46(Supp.2):1170-1179.
Shi T D, Zhong D Y, Bi L,2021.A new challenge:Detection of small-scale falling rocks on transportation roads in open-pit mines [J].Sensors,21(10):3548. DOI: https://doi.org/10.3390/s21103548 .
doi: https://doi.org/10.3390/s21103548
Tan M, Pang R, Le Q V,2020.EfficientDet:Scalable and efficient object detection[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).Seattle:IEEE: 19874970.DOI: 10.1109/CVPR42600.2020.01079 .
doi: 10.1109/CVPR42600.2020.01079
Wang C Y, Bochkovskiy A, Liao H Y M,2021.Scaled-YOLOv4:Scaling cross stage partial network[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).Nashville:IEEE: 21413564.DOI: 10.1109/CVPR46437.2021.01283 .
doi: 10.1109/CVPR46437.2021.01283
Wang W S, Wang S, Zhao Y Q,et al,2023a.Real-time obstacle detection method in the driving process of driverless rail locomotives based on DeblurGANv2 and improved YOLOv4[J].Applied Sciences,13(6):3861. DOI: https://doi.org/10.3390/app13063861 .
doi: https://doi.org/10.3390/app13063861
Wang C Y, Bochkovskiy A, Liao H Y M,2023b.YOLOv7:Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).Vancouver:IEEE: 23611189.DOI: 10.1109/CVPR52729.2023.00721 .
doi: 10.1109/CVPR52729.2023.00721
Wang Jie, Ye Mao, Ma Fengshan,2014.Design and implementation of the rockfall monitoring and warning system based on video image identification[J].Journal of Basic Science and Engineering,22(5):952-963.
Yang L X, Zhang R Y, Li L D,et al,2021.Simam:A simple,parameter-free attention module for convolutional neural networks[C]//Proceedings of the 38th International Conference on Machine Learning.PMLR,139:11863-11874.
陈垦,欧鸥,杨长志,等,2023.基于改进YOLOX的落石检测方法[J].计算机测量与控制,31(11):53-59.
高切,李登华,丁勇,2024.基于M-DBT框架的岩质边坡落石跟踪算法研究[J/OL].水利水运工程学报:1-13[2024-01-02]..
刘林芽,吴送英,左志远,等,2021.基于YOLOv3算法的山区铁路边坡落石检测方法研究[J].计算机科学,48(增2):290-294.
刘孜学,王富斌,虞凯,2022.基于改进YOLOv3的铁路落石检测方法研究与实现[J].高速铁路技术,13(3):52-56,80.
卢才武,齐凡,阮顺领,2020.融合目标检测与距离阈值模型的露天矿行车障碍预警[J].光电工程,47(1):40-47.
阮顺领,李少博,卢才武,等,2021.多尺度特征融合的露天矿区道路负障碍检测[J].煤炭学报,46(增2):1170-1179.
王杰,叶茂,马凤山,等,2014.基于视频图像识别的崩塌落石监测预警系统设计与实现[J].应用基础与工程科学学报,22(5):952-963.
[1] 阮顺领,阮炎康,卢才武,顾清华. 基于红外图像的矿石传送带托辊异常检测[J]. 黄金科学技术, 2023, 31(1): 123-132.
[2] 阮顺领,董莉娟,卢才武,顾清华. 基于RCR _YOLOv4的矿井巷道红外障碍检测研究[J]. 黄金科学技术, 2022, 30(4): 603-611.
[3] 王鹏飞,毕林,王李管. 露天矿无人驾驶矿卡速度规划研究[J]. 黄金科学技术, 2022, 30(3): 460-469.
[4] 王京华,王李管,毕林. 基于计算机视觉技术的矿井电机车无人驾驶障碍物检测技术[J]. 黄金科学技术, 2021, 29(1): 136-146.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
[1] 谢覃江. 云南省人头箐金矿床地质特征及成因探讨[J]. 黄金科学技术, 0, (): 0 .
[2] 张壮,李文,刘建明,曾庆栋. 大兴安岭中南段银多金属矿床勘查进展与找矿思路[J]. 黄金科学技术, 2016, 24(4): 60 -65 .
[3] 朱学礼,冯涛,柏瑞,宁霄峰,李肖,张子衿. “地质+”多元驱动智慧勘查初步应用[J]. 黄金科学技术, 2017, 25(1): 46 -54 .
[4] 何顺斌,刘杰. 基于谱聚类算法的岩体结构面产状优势分组[J]. 黄金科学技术, 2017, 25(4): 46 -51 .
[5] 殷璐,金哲男,杨洪英*,张勤. 我国黄金资源综合利用现状与展望[J]. 黄金科学技术, 2018, 26(1): 17 -24 .
[6] 曲晖, 王佰义, 王建民, 李成禄, 徐国战, 王卓. 土壤地球化学测量在永新金矿勘查中的应用及找矿效果研究[J]. 黄金科学技术, 2018, 26(2): 143 -152 .
[7] 杨明财, 盛建龙, 叶祖洋, 董舒. 基于FlAC3D的露天矿边坡稳定性及影响因素敏感性分析[J]. 黄金科学技术, 2018, 26(2): 179 -186 .
[8] 胡换龙,范宏瑞. 水/岩相互作用对焦家金矿金成色的影响[J]. 黄金科学技术, 2018, 26(5): 559 -569 .
[9] 谢敏雄,李政要,林属勇,迟晓鹏,亓传铎. 提高选矿厂磨矿系统效能的技术改造及应用研究[J]. 黄金科学技术, 2012, 20(6): 65 -68 .
[10] 胡笑坤,宋慧昌,刘青. 金矿选矿厂磨矿分级过程仿真系统研发[J]. 黄金科学技术, 2016, 24(5): 94 -101 .