基于加权双向特征融合的矿区道路落石检测

doi:10.11872/j.issn.1005-2518.2023.06.120

Abstract

Abstract:

With the booming development of big data and Internet of Things technology，traditional mines have developed to smart mines and intelligent mines，and unmanned technology has been gradually applied to mining areas.In order to solve the problem that the rockfall detection of unstructured road in open-pit mine area has complex environment，large difference in rockfall size and similar color between rockfall and unstructured road surface，a rockfall detection model of mining road based on weighted bidirectional feature fusion was proposed.First，the SimAM attention mechanism is added to the backbone network，this attention mechanism is different from the previous channel attention mechanism and spatial attention mechanism，it can effectively eliminate the interference of the background environment without adding additional parameters，so that the model can focus more on the target characteristics of rockfall.Second，the weighted bidirectional feature pyramid（BiFPN）structure was used to realize multi-scale feature fusion in the neck.Since the PANet structure in the YOLOv5s network model only adds or splice the characteristics of the pyramid structure in the melting process，the bidirectional feature weighting was combined with the bidirectional feature of the weight and adaptive adjustment to ensure that the network model attaches proper importance to the rock ebaissees different sizes and different levels and realizes the addition between the low-level position information and high-level semantic information for multiple cross-layer weighted feature fusion，thus enhancing the feature extracion ability of the model for rockfall of different sizes.Finally，the lightweight convolution GSConv module was introduced into the col，which can be used to process function cards at this time，not only reducing redundant information，but also avoiding compression.The GSConv lightweight convolution module is based on deep separable convolution（DSC），ordinary convolution（SC） and channel shuffle operation，which improves the detection speed of the model by effectively reducing the complexity of the model.The experimental results show that the average detection accuracy of this algorithm reaches 92.8%.and the detection speed reaches 63.1FPS.Compared with the current fastest R-CNN，YOLOv4-tiny，YOLOv7 and YOLOv5s algorithms，the average detection accuracy is increased by 17.0，13.6，3.4 and 2.5 percentiles，and the detection speed is increased by 32.2，1.4，14.6 and 2.6 FPS，respectively.Moreover，the model size of the algorithm is only 12.9 MB，which is easy to deploy on mobile devices.Therefore，the algorithm can realize the real-time and accurate detection of unstructured road rockfall in mining area，and ensure the safe driving of unmanned mining card.

Key words: mining area road, unmanned driving, machine vision, rockfall detection, multi-scale feature fusion, YOLOv5 model

CLC Number:

TP391.4

Qinghua GU, Yifan DU, Pingfeng LI, Dan WANG. Road Rockfall Detection in Mining Area Based on Weighted Bidirectional Feature Fusion[J].Gold Science and Technology, 2023, 31(6): 953-963.

Figures/Tables 17

Fig.1

Fig.2

Fig.3

Fig.4

Fig.5

Fig.6

Fig.7

Table 1

Table 2

Table 3

Table 4

Table 5

Table 6

Fig.8

Table 7

Fig.9

Fig.10

References 0

	Cao M C， Wang J M，2020.Obstacle detection for autonomous driving vehicles with multi-lidar sensor fusion［J］.Journal of Dynamic Systems Measurement and Control，142（2）：021007.
	Chen Ken， Ou Ou， Yang Changzhi，et al，2023.Rockfall detection method based on improved YOLOX［J］.Computer Measurement and Control，31（11）：53-59.
	Dong L H， Wang X R， Liu B Z，et al，2021.Information acquisition incentive mechanism based on evolutionary game theory［J］.Wireless Communications and Mobile Computing，2021：5525791.DOI：https：//doi.org/10.1155/2021/5525791 . doi: https：//doi.org/10.1155/2021/5525791
	Gao Qie， Li Denghua， Ding Yong，2024.A study on the rock block tracking algorithm that utilizes the M-DBT framework［J/OL］.Hydro-Science and Engineering：1-13［2024-01-02］..
	Girshick R，2015.Fast R-CNN［C］//IEEE International Conference on Computer Vision（ICCV）.Santiago：IEEE： 1580 1732.
	Girshick R， Donahue J， Darrell T，et al，2014.Rich feature hierarchies for accurate object detection and semantic segmentation［C］//IEEE Conference on Computer Vsion and PatternRecognition（CVPR）.Columbus：IEEE： 14632381.DOI：10.1109/CVPR.2014.81 . doi: 10.1109/CVPR.2014.81
	Han K， Wang Y H， Tian Q，et al，2020.Ghostnet：More features from cheap operations［C］//IEEE Conference on Computer Vision and Pattern Recognition（CVPR）.Seattle：IEEE： 19874590.DOI： 10.1109/CVPR42600.2020.00165 . doi: 10.1109/CVPR42600.2020.00165
	He K M， Zhang X Y， Ren S Q，et al，2015.Spatial pyramid pooling in deep convolutional networks for visual recognition［J］.IEEE Transactions on Pattern Analysis and Machine Intelligence，37（9）：1904-1916. DOI： 10.1109/TPAMI.2015. 2389824 . doi: 10.1109/TPAMI.2015. 2389824
	Li H L， Li J， Wei H B，et al，2022.Slim-neck by GSConv：A better design paradigm of detector architectures for autonomous vehicles［J］.arXiv preprint arXiv：2206.02424.DOI：https：//doi.org/10.48550/arXiv.2206.02424 . doi: https：//doi.org/10.48550/arXiv.2206.02424
	Lin T Y， Goyal P， Girshick R，et al，2017a.Focal loss for dense object detection［C］//IEEE International Conference on Computer Vsion（ICCV）.Venice：IEEE： 19262330.DOI： 10.1109/TPAMI.2018.2858826 . doi: 10.1109/TPAMI.2018.2858826
	Lin T Y， Dollár P， Girshick R，et al，2017b.Feature pyramid networks for object detection［C］//IEEE Conference on Computer Vsion and Pattern Recognition（CVPR）.Hawaii：IEEE： 17355379.DOI： 10.1109/CVPR.2017.106 . doi: 10.1109/CVPR.2017.106
	Liu B， Ding Z F， Tian L L，et al，2020.Grape leaf disease identification using improved deep convolutional neural networks［J］.Frontiers in Plant Science，11：1082. DOI： https：//doi.org/10.3389/fpls.2020.01082 . doi: https：//doi.org/10.3389/fpls.2020.01082
	Liu B， Tian B H， Qiao J C，2022.Mine track obstacle detection method based on information fusion［C］//Journal of Physics：Conference Series.IOP Publishing，2229（1）：012023. DOI： 10.1088/1742-6596/2229/1/012023 . doi: 10.1088/1742-6596/2229/1/012023
	Liu Linya， Wu Songying， Zuo Zhiyuan，et al，2021.Research on Rockfall detection method of mountain railway slope based on YOLOv3 algorithm［J］.Computer Science，48（Supp.2）：290-294.
	Liu W， Anguelov D， Erhan D，et al，2016.Ssd：Single shot multibox detector［C］//European Conference on Computer Vision（ECCV）.Amsterdam：IEEE： 21-37.
	Liu Zixue， Wang Fubin， Yu Kai，2022.Research and implementa-tion of the railway rockfall detection based on modified YO-LOv3［J］.High Speed Railway Technology，13（3）：52-56，80.
	Lu Caiwu， Qi Fan， Ruan Shunling，2020.An open-pit mine roadway obstacle warning method integrating the object detection and distance threshold model［J］.Opto-Electronic Engineering，47（1）：40-47.
	More K S， Wolkersdorfer C，2023.Intelligent mine water management tools—eMetsi and machine learning GUI［J］.Mine Water and the Environment，42（1）：111-120.DOI：https：//doi.org/10.1007/s10230-023-00917-7 . doi: https：//doi.org/10.1007/s10230-023-00917-7
	Patel N， Krishnamurthy P， Tzes A，et al，2021.Overriding learning-based perception systems for control of autonomous unmanned aerial vehicles［C］//International Conference on Unmanned Aircraft Systems （ICUAS）.Athens：IEEE： 20916280.DOI： 10.1109/ICUAS51884.2021.9476881 . doi: 10.1109/ICUAS51884.2021.9476881
	Redmon J， Farhadi A，2017.YOLO9000：Better，faster，stronger［C］//IEEE Conference on Computer Vision and Pattern Recognition（CVPR）.Honolulu：IEEE： 17355115.DOI： 10.1109/CVPR.2017.690 doi: 10.1109/CVPR.2017.690
	Ren S Q， He K M， Girshick R，et al，2017.Faster R-CNN：Towards real-time object detection with region proposal networks［J］.IEEE Transactions on Pattern Analysis and Machine Intelligence，39（6）：1137-1149.
	Ruan Shunling， Li Shaobo， Lu Caiwu，et al，2021.Road negative obstacle detection in open-pit mines based on multi scale feature fusion［J］.Journal of China Coal Society，46（Supp.2）：1170-1179.
	Shi T D， Zhong D Y， Bi L，2021.A new challenge：Detection of small-scale falling rocks on transportation roads in open-pit mines ［J］.Sensors，21（10）：3548. DOI： https：//doi.org/10.3390/s21103548 . doi: https：//doi.org/10.3390/s21103548
	Tan M， Pang R， Le Q V，2020.EfficientDet：Scalable and efficient object detection［C］//IEEE/CVF Conference on Computer Vision and Pattern Recognition（CVPR）.Seattle：IEEE： 19874970.DOI： 10.1109/CVPR42600.2020.01079 . doi: 10.1109/CVPR42600.2020.01079
	Wang C Y， Bochkovskiy A， Liao H Y M，2021.Scaled-YOLOv4：Scaling cross stage partial network［C］//IEEE/CVF Conference on Computer Vision and Pattern Recognition（CVPR）.Nashville：IEEE： 21413564.DOI： 10.1109/CVPR46437.2021.01283 . doi: 10.1109/CVPR46437.2021.01283
	Wang W S， Wang S， Zhao Y Q，et al，2023a.Real-time obstacle detection method in the driving process of driverless rail locomotives based on DeblurGANv2 and improved YOLOv4［J］.Applied Sciences，13（6）：3861. DOI： https：//doi.org/10.3390/app13063861 . doi: https：//doi.org/10.3390/app13063861
	Wang C Y， Bochkovskiy A， Liao H Y M，2023b.YOLOv7：Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors［C］//IEEE/CVF Conference on Computer Vision and Pattern Recognition（CVPR）.Vancouver：IEEE： 23611189.DOI： 10.1109/CVPR52729.2023.00721 . doi: 10.1109/CVPR52729.2023.00721
	Wang Jie， Ye Mao， Ma Fengshan，2014.Design and implementation of the rockfall monitoring and warning system based on video image identification［J］.Journal of Basic Science and Engineering，22（5）：952-963.
	Yang L X， Zhang R Y， Li L D，et al，2021.Simam：A simple，parameter-free attention module for convolutional neural networks［C］//Proceedings of the 38th International Conference on Machine Learning.PMLR，139：11863-11874.
	陈垦，欧鸥，杨长志，等，2023.基于改进YOLOX的落石检测方法［J］.计算机测量与控制，31（11）：53-59.
	高切，李登华，丁勇，2024.基于M-DBT框架的岩质边坡落石跟踪算法研究［J/OL］.水利水运工程学报：1-13［2024-01-02］..
	刘林芽，吴送英，左志远，等，2021.基于YOLOv3算法的山区铁路边坡落石检测方法研究［J］.计算机科学，48（增2）：290-294.
	刘孜学，王富斌，虞凯，2022.基于改进YOLOv3的铁路落石检测方法研究与实现［J］.高速铁路技术，13（3）：52-56，80.
	卢才武，齐凡，阮顺领，2020.融合目标检测与距离阈值模型的露天矿行车障碍预警［J］.光电工程，47（1）：40-47.
	阮顺领，李少博，卢才武，等，2021.多尺度特征融合的露天矿区道路负障碍检测［J］.煤炭学报，46（增2）：1170-1179.
	王杰，叶茂，马凤山，等，2014.基于视频图像识别的崩塌落石监测预警系统设计与实现［J］.应用基础与工程科学学报，22（5）：952-963.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

Comments

Recommended 10

[1]	YAN Jie, QIN Ze-Li, XIE Wen-Bing, CA Bang-Yong. [J]. J4, 2010, 18(4): 22 -26 .
[2]	SONG He-Min, FENG Chi-Li, DING Xian-Hua. Geological-Geochemical Charicteristics and Prospecting Direction of Jiaojiekou Mining Area, North Part of Taihang Mountain[J]. J4, 2010, 18(3): 54 -58 .
[3]	LI Chu-Fang, XU Yong-An, CHAO Yin-Yin, WANG Mei-Juan, ZHANG Dai, LIU Jun, SUN Liang-Liang. Looking for Strata Bound Type Gold Deposit in Liaodong Metallogenic Belt[J]. J4, 2010, 18(3): 59 -62 .
[4]	HU Qin-Xia, LI Jian-Zhong, YU Guang-Meng, XIE Yan-Fang, ZHANG Ku-Xiao. Discussion on Gold Ore Point of Bailongjiang Metallogenic Belt[J]. J4, 2010, 18(3): 51 -53 .
[5]	CHEN Hua-Dun. [J]. J4, 2010, 18(4): 50 -53 .
[6]	CUI Ting-Jun, DAI Ke-Sai, PENG Yong, FU Xing. Discussion on the Geological Characteristics and Metallogenic Regularity of the Southern Edge of Qaidam Basin Gold Metallogenic Belt in Qinghai Province[J]. J4, 2010, 18(3): 63 -67 .
[7]	YANG Meng-Rong, MAO Chang-Xian. Uncertainty Evaluation of Arsenic and Antimony in Chemical Prospecting Sa-mple by Atomic Fluorescence Spectrometry[J]. J4, 2010, 18(3): 68 -71 .
[8]	SU Jian-Hua, LIU Shu-Lin. Study of Gold Extraction from Tail Solution with High Acid Content and Low Concentration[J]. J4, 2010, 18(3): 72 -75 .
[9]	WANG Da-Beng, SONG Bing-Jian, HUI Ku-Meng. The Application of High Power IP Method for Searching Concealed Metallic Mine in Beishuiquan，Liaoning Province[J]. J4, 2010, 18(3): 76 -78 .
[10]	LIU Qing-Guang, GAO Hai-Feng, HUANG Suo-Yang. The Application of PDA in Geology Department of Jiaojia Gold Mine[J]. J4, 2010, 18(3): 79 -82 .

参数名称	参数设置	参数名称	参数设置
图像输入大小	640×640×3	初试学习率	0.01
训练批次	8	NMS阈值	0.45
迭代次数	200	优化器	SGD

	实际正确	实际错误
预测正确	真阳性TP	假阳性FP
预测错误	假阴性FN	真阴性TN

模型	深度因子	宽度因子	mAP@0.5/%	检测速度 /FPS	模型大小/M
YOLOv5s	0.33	0.5	90.3	60.5	13.7
YOLOv5m	0.65	0.75	90.9	48.5	41.9
YOLOv5l	1	1	91.5	39.5	91.6
YOLOv5x	1.33	1.25	91.9	37.2	170.2

嵌入方式	P/%	R/%	mAP@0.5/%	检测速度/FPS
方式一	91.2	87.1	91.6	60.8
方式二	90.4	87.9	91.1	61.2

模型	BiFPN	SimAM	GSConv	P/%	R/%	mAP@0.5/%	检测速度 /FPS
0				89.9	85.9	90.3	60.5
1	√			91.5	88.4	91.8	58.8
2		√		91.2	87.1	91.6	60.8
3			√	90.5	86.4	90.8	63.9
4	√	√		92.3	89.1	92.5	59.2
5	√		√	92.1	88.3	92.2	62.1
6		√	√	91.6	88.4	91.9	63.3
7	√	√	√	92.5	89.8	92.8	63.1

Road Rockfall Detection in Mining Area Based on Weighted Bidirectional Feature Fusion

RichHTML

PDF (PC)

Abstract

Cite this article

share this article

Figures/Tables 17

References 0

Related Articles 3

Metrics

Comments

Recommended 10

检测方法	P/%	R/%	mAP@0.5/%	检测速度 /FPS	模型大小 /M
Faster R-CNN	72.8	67.2	75.8	30.9	108.9
YOLOv4-tiny	77.8	75.1	79.2	61.7	23.1
YOLOv5s	89.9	85.9	90.3	60.5	13.7
YOLOv7	88.3	86.7	89.4	48.5	71.3
本文算法	92.5	89.8	92.8	63.1	12.9

[1]	Shunling RUAN,Yankang RUAN,Caiwu LU,Qinghua GU. Detection of Ore Conveyer Roller Based on Infrared Image [J]. Gold Science and Technology, 2023, 31(1): 123-132.
[2]	Shunling RUAN,Lijuan DONG,Caiwu LU,Qinghua GU. Research on Infrared Obstacle Detection of Mine Roadway Based on RCR_YOLOv4 [J]. Gold Science and Technology, 2022, 30(4): 603-611.
[3]	Pengfei WANG,Lin BI,Liguan WANG. Research on Speed Planning of Unmanned Driving Mining Trucks in Open-pit Mines [J]. Gold Science and Technology, 2022, 30(3): 460-469.