基于视频序列的矿卡司机不安全行为识别

doi:10.11872/j.issn.1005-2518.2021.01.216

Abstract

Abstract:

At present，many mines still rely on human supervision to supervise the unsafe behavior of mining truck drivers，and cannot find problems timely and accurately.This consumes a certain amount of manpower and material resources but cannot solve the problem.With the development of computer technology and artificial intelligence technology，more and more fields are beginning to use artificial intelligence technology to supervise the unsafe behavior of mining truck drivers，such as intelligent security，unmanned driving，and intelligent transportation.Behavior recognition is a hot issue in the field of computer vision.Using computer technology to identify unsafe behaviors is an efficient way to replace manual detection.This paper uses deep learning to solve the unsafe behavior recognition of mining truck drivers in video sequences.The traditional deep learning method does not rely on artificial design features，but adaptively learns better high-dimensional features，better robustness，and faster speed，the accuracy rate is higher.Firstly，according to the actual obtained video data，by analyzing the relative position relationship between the camera and the driver’s area，the video is clipped to obtain video data with less redundant information.At the same time，in order to reduce the imbalance of the data samples，by using flipping，methods such as panning and adding noise were used to enhance the data set，and then use Opencv to re-convert the enhanced image data into a video file and use the dense_flow method to obtain an optical flow diagram.Secondly，use the network for training and testing.In order to conduct com-parative experiments，firstly，a traditional classification model that does not consider time sequence information was used for training and testing，and the transfer learning method was used to train Resnet，Xception，and Inception.And fusion of three single models to get a new fusion model.At the same time，the time domain and spatial domain channels of the dual-stream network model are set to the pre-trained VGG16 using migration learning under the consideration of timing information，and the comparison experiment was carried out with the C3D-two-stream proposed in this paper.The experimental results show that the improved Vgg-two-stream model can reach an accuracy rate of 89.539%，and the accuracy rate of the C3D-two-stream model can reach 93.445%.In summary，the C3D-two-stream model proposed in this paper has a high recognition rate.It also proves that for behavior recognition，the acquisition of characteristic information in the time dimension can make the recognition results more accurate，which has important practical significance for the recognition of unsafe behaviors of mining truck drivers.

Key words: unsafe behavior, video sequence, deep learning, mining truck driver, behavior recognition, two stream network, fusion model

CLC Number:

TD76

Lin BI,Chao ZHOU,Xin YAO. Unsafe Behavior Identification of Mining Truck Drivers Based on Video Sequences[J].Gold Science and Technology, 2021, 29(1): 14-24.

Figures/Tables 14

Fig.1

Fig.2

Fig.3

Fig.4

Table 1

Fig.5

Fig.6

Fig.7

Fig.8

Fig.9

Fig.10

Fig.11

Table 2

Fig.12

References 0

	Cai Qiang，Deng Yibiao，Li Haisheng，al et，2020.Review of human behavior recognition methods based on deep learning［J］.Computer Science，47（4）：85-93.
	Dalal N，Triggs B，2005.Histograms of oriented gradients for human detection［C］//2005 IEEE Conference on Computer Vision and Pattern Recognition（CVPR），San Diego，CA，USA. Boston：IEEE. 1：886-893.
	Dalal N，Triggs B，Schmid C，2006.Human detection using oriented histograms of flow and appearance［C］//European Conferences on Computer Vision.Heidelberg：Springer：428-441.
	Gao J，Liu J，Han J，2019.A study for real-time identification of unsafe behavior of taking off safety helmet based on VSM model［C］// Proceedings of the 11th International Conference on Computer Modeling and Simulation.New York：Association for Computing Machinery.
	Hacefendiolu K，Baaa H B，Demir G，2021.Automatic detection of earthquake-induced ground failure effects through Faster R-CNN deep learning-based object detection using satellite images［J］.Natural Hazards，105：383-403.
	Huang Youwen，Wan Chaolun，Feng Heng，2019.Multi-feature fusion human behavior recognition algorithm based on convolutional neural network and long-short-term memory neural network［J］.Progress in Laser and Optoelectronics，56（7）：243-249.
	Ji S，Xu W，Yang M，al et，2013.3D convolutional neural networks for human action recognition［J］.IEEE Transactions on Pattern Analysis & Machine Intelligence，35（1）：221-231.
	Klaser A，Marszalek M，Cordelia S，2008.A spatio-temporal descriptor based on 3D-gradients［C］//British Machine Vision Conference， Aberystwyth， UK. Guildford：BMVC.
	Laptev I，Marszalek M，Schmid C，al et，2008.Learning realistic human actions from movies［C］//2008 IEEE Conference on Computer Vision and Pattern Recognition.Boston：IEEE：1-8.
	Li K，Zou C，Bu S，al et，2018.Multi-modal feature fusion for geographic image annotation［J］.Pattern Recognition，73： 1-14.
	Mao Zhiqiang，Ma Cuihong，Cui Jinlong et al，2019.Research on behavior recognition based on two-stream convolution and two-center loss［J］.Microelectronics and Computer，36（3）：96-100.
	Mazda T，Kajita Y，Akedo T，al et，2020.Recognition of nonlinear hysteretic behavior by neural network using deep learning［J］.IOP Conference Series Materials Science and Engineering，809：012010.
	Yue-Hei Ng J，Hausknecht M，Vijayanarasimhan S，al et，2015.Beyond short snippets：Deep networks for video classification［C］//2015 IEEE Conference on Computer Vision and Pattern Recognition （CVPR）.Boston：IEEE，4694-4702.
	Simonyan K，Zisserman A，2014.Two-stream convolutional networks for action recognition in videos［J］.Advances in Neural Information Processing Systems.
	Sun Y，Fu J，Ma Q，al et，2020.Research on wear recognition of electric worker’s helmet based on neural network［J］.Journal of Physics：Conference Series，1449（1）：012057.
	Tran D，Bourdev L，Fergus R，al et，2015.Learning spatio temporal features with 3D convolutional networks ［C］//Proceedings of the IEEE International Conference on Computer Vision. Boston：IEEE：4489-4497.
	Wang H，Kläser A，Schmid C，al et，2011.Action recognition by dense trajectories［C］//2011 IEEE Conference on Computer Vision and Pattern Recognition（CVPR）.Boston：IEEE：3169-3176.
	Wang H，Schmid C，2013.Action recognition with improved trajectories［C］//IEEE International Conference on Computer Vision（ICCV）.Boston：IEEE：3551-3558.
	Wang L，Xiong Y，Wang Z，al et，2016.Temporal segment networks：Towards good practices for deep action recognition［C］//European Conference on Computer Vision.Cham：Springer：20-36.
	Wang Yi，Ma Cuihong，Mao Zhiqiang，2020.Behavior recognition based on space-time dual-stream fusion network and attention model［J］.Computer Applications and Software，37（8）：156-159，193.
	蔡强，邓毅彪，李海生，等，2020.基于深度学习的人体行为识别方法综述［J］.计算机科学，47（4）：85-93.
	黄友文，万超伦，冯恒，2019.基于卷积神经网络与长短期记忆神经网络的多特征融合人体行为识别算法［J］.激光与光电子学进展，56（7）：243-249.
	毛志强，马翠红，崔金龙，等，2019.基于双流卷积与双中心loss的行为识别研究［J］.微电子学与计算机，36（3）：96-100.
	王毅，马翠红，毛志强，2020.基于时空双流融合网络与Attention模型的行为识别［J］.计算机应用与软件，37（8）：156-159，193.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

Comments

Recommended 10

[1]	L IU J ian, FAN Manhua,DENG Zhigao, ZHANG Qingsong, ZHENG Chengying. Exper imenta l Study on Comprehensive Shr inkageMethod in Na lin Gold Deposit[J]. J4, 2008, 16(6): 48 -50 .
[2]	ZHANG Baolin,CAI Xinping,WANG Jie,LIANG Guanghe,DING Rufu,XIAO Qibin,SONG Baocha. Prospecting Potential of Concealed Gold Deposits of New Types in the Bordering Areas Among Shanxi,Hebei and Inner Mongol ia With Special Respect to the Deposits of Puziwan,Jiuduigou and Shuijingtun[J]. J4, 2004, 12(2): 5 -11 .
[3]	SONG He-Min, ZHANG Wen-Zhao, XU Shu-Beng. Geochemical anomaly models and exploration meaning for Damoqujia gold deposit in Jiaodong[J]. J4, 2006, 14(6): 13 -23 .
[4]	LI Zhenjiang，WANG Jiqing，WANG Ping，MENG Fanli. The Efective W ay for Improving the Comprehensive Utilization Ratio of M ineral Resources in Jinzhou Mining Group[J]. J4, 2008, 16(4): 78 -80 .
[5]	. [J]. J4, 1995, 3(5): 49 -52 .
[6]	SUN Zhenzuo, WU J ichuen, HA Benhai,TENG Yuansheng. THE GELOGY OF GOLD DEPOSIT AND ITS PROSPECTING DIRECTION OF LINGSHANGOU GOLD DEPOSIT[J]. J4, 2003, 11(5): 16 -22 .
[7]	LIU Dang-Quan, SUN Ceng-Feng. Pillar Recover of Sublevel Drilling Stage Room Stoping and Draw Managing[J]. J4, 2005, 13(1-2): 58 -62 .
[8]	. [J]. J4, 1995, 3(3): 3 -9 .
[9]	LIU Zhiming. Ocecurrence of Gold in the Dongan EpithermalGold Deposit, Heillongjiang Province[J]. J4, 2005, 13(05): 19 -22 .
[10]	ZHANG Qun-Xi. Discussion on Characteristics of the Ductile Shear Belt and Au - Mineralization、vith Mechanism of Metallogenic Dynamics From MaoPai Gold Deposit in the Linchuan of Jiangxi Province[J]. J4, 2007, 15(5): 1 -7 .

方法	IDT（CPU）	Brox’s（CPU）	Brox’s（GPU）	C3D（GPU）
运行时间/h	202.2	2 513.9	607.8	2.2
每秒传输帧数	3.5	0.3	1.2	313.9
X Slower	91.4	1 135.9	274.6	1

行为类别	精度/%
行为类别	C3D	Two-stream	VGG-two-stream	C3D-two-stream
平均准确率	76.639	78.175	89.539	93.445
双手离开方向盘	69.067	67.972	83.016	88.366
无人	93.516	97.194	99.577	100.000
正常驾驶	73.591	76.443	88.915	94.038
玩手机	70.382	71.091	86.648	91.375

Unsafe Behavior Identification of Mining Truck Drivers Based on Video Sequences

RichHTML

PDF (PC)

Abstract

Cite this article

share this article

Figures/Tables 14

References 0

Related Articles 2

Metrics

Comments

Recommended 10

[1]	Lin BI,Yalong LI,Zhaohong GUO. Study on the Estimation of Ore Loading Quantity of Truck Based on Deep Convolutional Neural Network [J]. Gold Science and Technology, 2019, 27(1): 112-120.
[2]	BI Lin，XIE Wei，CUI Jun. Identification Research on the Miner’s Safety Helmet Wear Based on Convolutional Neural Network [J]. Gold Science and Technology, 2017, 25(4): 73-80.