5G Edge Cloud Power Real-time Inspection Technology Based on YOLOV4-Tiny

Jiaqi Song¹, Jing Li², Di Wu¹, Guangye Li³, Jiaxin Zhang³, Jiantie Xu³ and Tian Lan^2,*

¹State Grid Chaoyang Power Supply Company, Chaoyang 122000, China

²State Grid SIJISHENWANG Location-Based Service (BeiJing) CO.LTD, Beijing 102211, China

³State Grid Liaoning Electric Power Supply Co.Ltd, Shenyang 110006, China

E-mail: lant0514@126.com; dowtowne@foxmail.com

*Corresponding Author

Received 14 May 2021; Accepted 16 June 2021; Publication 28 July 2021

Abstract

Power line corridor inspection plays a vital role in power system safe operation, traditional human inspection’s low efficiency makes the novel inspection method requiring high precision and high efficiency. Combined with the current deep learning target detection algorithm based on high accuracy and strong real-time performance, this paper proposes a YOLOV4-Tiny based drone real-time power line inspection method. The 5G and edge computing technology are combined properly forming a complete edge computing architecture. The UAV is treated as an edge device with a YOLOV4-Tiny deep- learning-based object detection model and AI chip on board. Extensive experiments on real data demonstrate the 5G and Edge computing architecture could satisfy the demands of real-time power inspection, and the intelligence of the whole inspection improved significantly.

Keywords: Deep learning, edge cloud, 5G, real-time power line inspection.

Introduction

With the rapid development of society and the advancement of science and technology, the electricity demand has become greater and greater. Therefore, higher requirements have been put forward for the stability of grid operation. The demand for inspection is getting higher and higher. At present, the traditional manual power inspection method is used to inspect the operation status of power equipment utilizing artificial viewing and listening. However, this inspection method has many drawbacks, such as high labor cost, strong subjectivity of inspection data, and data management. The problems of dispersion and high-risk factors have been unable to meet the demand for accurate, real-time, high-frequency power equipment inspection [1]. At present, the development of UAV remote sensing technology in China has been relatively mature and has been widely used in power inspection [2, 3]. At present, deep learning and other artificial intelligence technologies are gradually being introduced into the smart grid, which can not only solve the problem of identifying and extracting complex features in power inspections, but also provide more data types for deep learning, and Solve the problem of a single type of training data and poor generalization ability [4].

In the power grid inspection, drones are required to detect power grid faults in real-time and send them back to the inspectors in time to make timely decisions. The 5G edge cloud technology can just meet the requirements of the above tasks. Therefore, the application of 5G edge cloud computing to the field of power inspections can sink the calculation, identification, and analysis of inspection data to the edge, effectively reducing response delays and bandwidth costs, improve identification efficiency, and realize real-time detection. Edge cloud computing mainly includes six modules: software as a service application, platform as a service capability, infrastructure as a service facility, hardware facility, computer room location, and network bearer [5].

1 Power Inspection Based on Deep Learning

There are more and more researches on power inspection based on deep learning in China. For example, Zhou Zhubo, Gao Jiao, etc. [6] based on the AlexNet network structure, proposed a 5-layer deep convolutional neural network to detect transmission line targets from visible light images, but only for tower materials, Glass insulators and composite insulators are tested. Li Hui and Zhong Pingping [7] proposed a method based on the Faster-RCNN [8] target detection model combined with the corrosion HIS color feature of the transmission line corrosion inspection method. This method uses the color characteristics of rust to remove the interference background in the image, and then uses the Faster-RCNN depth model to detect the rust of the power inspection image, which improves the accuracy of the power rust inspection to a certain extent, but reduces the power rust inspection. Check the accuracy of positioning. Guo Tao, Yang Heng et al. [9] proposed a method for identifying and judging insulator defects based on deep learning Faster-RCNN. This method can accurately identify and extract insulator strings, but there are many cases of missed insulator self-explosion. Wei Dong, Gong Qingwu, et al. [10] proposed a method of the convolutional neural network to solve the fault judgment and fault phase selection inside and outside the transmission line. This method uses simulation data for model training to realize the fault judgment and phase selection inside and outside the power grid line, but the simulation data It is impossible to fully simulate the real power grid inspection data, and it relies heavily on the simulation environment, so the reliability of the data formed is low. Mao Xianyin, Liu Yu et al. [11] proposed a method for electric robot obstacle recognition based on the Deep Learning Single Shot Multibox Detector (SSD) algorithm. This algorithm can realize the stable recognition of electric power inspection obstacles, but this algorithm Under the embedded system, the real-time performance is low and the hardware equipment requirements are high. Zhou Fangrong and Fang Ming et al. [12] proposed a fast detection method for transmission line defects based on YOLOV3, which speeds up model training by adjusting the loss function, but the model inference speed is slower than YOLOV4-Tiny.

Therefore, this paper integrates 5G edge cloud and BeiDou navigation satellites and uses YOLOV4-Tiny as the target detection model to achieve faster and more accurate real-time power inspections. Through the test on the real power grid inspection data, the effectiveness and accuracy of the method in this paper are fully verified.

2 Method of This Article

Mobile Edge Computing

Mobile edge computing (MEC) can sink communication, computing, and storage resources to the device terminal side [13], which can effectively reduce the data transmission delay of drone power inspections and realize real-time power inspections [14]. Edge computing nodes are mainly divided into two parts: network-side edge computing and field-level edge computing. Side edge computing in power inspections is generally deployed on the ground, namely ground servers, and field-level edge computing is deployed on various terminals, namely drones. The process is shown in Figure 1. The 5G [15] module, Xavier, BeiDou [16], and other hardware devices are installed on the UAV platform while collecting power equipment data, combined with the deep learning network model to complete real-time power equipment detection, and transmit the detected power inspection device data to the Internet server terminal. For data that needs to perform complex calculations, it is transmitted to the ground edge cloud for processing through the 5G network, and then the detected power equipment data is formed into instructions and transmitted to the Internet server terminal. Finally, the Internet server terminal stores the data transmitted by the edge cloud and can view or review the power inspection results through the remote control center.

Figure 1 Flow chart of edge computing design.

By changing the number of orthogonal sub-channels and sub-channel bandwidth of the system [17], this paper can use multiple UAVs to simultaneously perform data acquisition and real-time detection of power lines, thereby achieving parallel inspections of power and greatly improving the efficiency of power inspections.

2.1 Algorithm Design

Target detection algorithms based on deep learning are divided into two categories: single-stage detection algorithms and two-stage detection algorithms. The former has an advantage in detection speed, while the latter has an advantage in detection accuracy. Taking into account the computing power of the Xavier chip on the drone, this paper uses YOLOv4-Tiny as the backbone network to detect the target to be detected in the power line corridor.

The YOLO backbone network first extracts the features of the input image and divides it into M $\times$ M grids. The grid where the center of the target is located is used to detect the target. To achieve the detection of multiple types of targets (for example, the target detection category in this experiment is C $=$ 7), each grid can predict N bounding boxes and the conditional probabilities that belong to the C-type detection target and output the confidence level, It contains whether the bounding box contains the target possibility and the bounding box accuracy. When the bounding box background contains the target $P r_{o b j e c t} e q u a l s t o 1$ , and the opposite is 0; the bounding box accuracy is represented by the intersection ratio (IOU) of the predicted box and the real box, so the confidence can be expressed by IOU $\times P r_{o b j e c t}$ . Each prediction box contains five parameters x, y, w, h, c, where (x, y) represents the offset between the center of the prediction box and the center of the true box, (w, h) represents the width and height of the prediction box, c Indicates the confidence level of the prediction box. Finally, according to the output of each grid, the final output parameter of the model is M $\times$ M $\times$ (N $\times$ 5 $\times$ 5+C).

The YOLOV4 [18] algorithm is based on YOLOV3 [19] and introduces some optimization methods from data processing, backbone network, network training, activation function [20], loss function, etc., so that the model can achieve detection speed and accuracy. Better performance.

YOLOV4 is based on the CSPDarknet53 backbone network to extract target features. Draw lessons from the experience of CSPNet [21] that can maintain detection accuracy, reduce computational bottlenecks, and reduce memory costs while lightweighting, YOLOv4 adds the CSP [21] module to each large residual block of Darknet53, divides the feature map of the base layer into two parts, and then merges through the cross-stage hierarchical structure to reduce the amount of calculation while ensuring accuracy.

Figure 2 YOLOV4-Tiny network structure.

In this paper, the power inspection deep learning network model is based on the lightweight YOLOV4-Tiny network. YOLOV4-Tiny simplifies the CSPDarkNet53 network structure, uses the Leaky_relu activation function to replace the Mish activation function, and eliminating PANet module, and the 5-layer CSP structure is simplified to a 3-layer CSP structure, and the feature pyramid module (FPN) is used to spread the semantic detection information of the high-feature layer to the low-level network, and then merge it with the high-resolution information of the underlying features to improve the detection effect of power equipment. Its concrete network structure is shown in Figure 2.

2.1.1 Backbone network

The backbone network used by YOLOV4-Tiny [22] mainly includes down-sampling CBL structure and CSP structure. The convolution kernel of the down-sampling CBL structure is 3 $\times$ 3, and the step size is 2. The feature map of the base layer is divided into two parts by CSP and then merged using cross-layer connections to obtain more semantic information and enhance the learning ability of the convolutional neural network, effectively reducing the computing bottleneck period, and ensuring the accuracy of power equipment detection rate.

According to the floating-point operands (FLOPs), the difference between the YOLOV4 algorithm and the YOLOV4-Tiny algorithm can be judged, as shown in formula 1.

FLOPs

= \int_{l = 1}^{D} M_{l}^{2} \cdot K_{l}^{2} \cdot C_{l - 1} \cdot C_{l}

(1)

In the formula, D is the sum of all convolutional layers, $M_{l}$ is the output feature map size of the Lth convolutional layer, $K_{l}$ is the number of kernel sizes, and $C_{l - 1}$ and $C_{l}$ are the number of input channels and output channels, respectively. Since the number of CSP levels in YOLOV4-Tiny is two less than YOLOV4, the target detection speed of YOLOV4-Tiny is faster than YOLOV4.

2.1.2 Detection part

YOLOV4-Tiny uses multiple feature layers for target detection. Obtain two effective feature layers through the backbone network and input them to the Feature Pyramid Network (FPN). FPN will convolve the last effective feature layer and then perform up-sampling, and then stack and convolve with the last effective feature layer to achieve semantic fusion of high-level features and low-level features. Finally, two YOLO detection heads are used to complete the positioning and identification functions.

As can be seen from the above, each network point is responsible for the detection of a region, and then the final result is classified and bounding box regression. The boundary regression calculation formula is shown in formula 2:

$b_{x}$	$= σ t_{x} + c_{x}$
$b_{y}$	$= σ t_{y} + c_{y}$	(2)
$b_{w}$	$= p_{w} e^{t_{w}}$
$b_{h}$	$= p_{h} e^{t_{h}}$

In the above formula ( $b_{x}, b_{y}, b_{w}, b_{h}$ ) is the center point coordinates and length and width of the predicted bounding box in the Feature map; ( $t_{x}, t_{y}, t_{w}, t_{h}$ ) is the offsets of the network learning relative to the prior box; ( $c_{x}, c_{y}$ ) is the upper left point of each cell coordinate; $p_{w}, p_{h}$ is the length and width of the a priori box relative to the feature map.

3 Experimental Verification

3.1 Hardware Equipment And Experimental Datasets

The drone pods and edge cloud computing modules used in this experiment include GNSS antenna (HX-CA7607A), GNSS positioning chip (UB482), 5G chip (Huawei MH5000-31), camera (CMOS IMX183CLK-J/CQJ-J) Image transmission module (MKV-0201), core main control chip (Jetson AGX Xavier), step-down module (SY2D00K2412) and battery pack (TB48S). The detailed assembly design is shown in Figure 3.

Figure 3 UAV pod design.

Figure 4 Training data set. (The square marked boxes formed by the green dots are seven types of training targets)

Compared with the traditional drone pod inspection, the method in this paper uses Jetson AGX Xavier chip and 5G edge cloud computing. The terminal performs data processing, combined with 5G edge cloud computing, making the real-time power inspection of drones more stable and reliable.

This paper uses the power equipment data collected by the drone of the State Grid Liaoning Chaoyang Electric Power Company as the deep learning network model training data set (as shown in Figure 4) to realize the power line (transmission tower, shock-proof hammer, tower number plate, bird’s nest, interval For real-time detection of rods, insulators, and equalizing rings), the data set contains a total of 3000 images, and contains a variety of scenes such as plains, cities, hills, jungles, and mountains. Use LabelImg labeling software to complete the labeling of seven types of target frames such as power transmission towers, anti-vibration hammers, and tower number plates. The category labeling results are shown in Table 1.

Table 1 Target marked statistical results

Marked		Vibration					Grading
Target	Pylon	Damper	Nameplate	Nest	Spacer	Insulator	Ring
Quantity	3650	10342	14749	767	1564	8955	5343

3.2 Experimental Result

The test results of power equipment are shown in Figure 5. In this paper, AP (Average Precision), MAP (Mean Average Precision), and recall rate (Recall) are used as the evaluation criteria of the detection accuracy of this model. The results are shown in Table 2.

Table 2 Real-time detection accuracy of electric power in UAV pods and Ground server Yolov4 detection accuracy

Test Target/Platform	UAV	Ground
Pylon (AP)	86.62%	95.29%
Vibration Damper (AP)	82.88%	87.92%
Nameplate (AP)	82.65%	100.00%
Nest (AP)	22.92%	78.87%
Spacer (AP)	64.66%	71.36%
Insulator (AP)	86.55%	92.65%
Grading Ring (AP)	83.51%	89.71%
Recall	0.74	0.82
mAP	72.82%	87.97%
F1-score	0.78	0.85

Because the bird’s nest and spacer rods in the experimental scene are severely occluded during training and recognition, their recognition accuracy will be slightly lower, as shown in Table 2. Because the border of the bird’s nest is blurred, it is difficult to distinguish its range, resulting in low detection accuracy, as shown in Figure 5(c).

Figure 5 Real-time power detection results of drone pods.

Figure 6 Ground server test results.

The network model carried by the drone terminal is a lightweight network, and the scene is more complex, and there will be missed and wrong detections. Therefore, the YOLOV4 network is deployed on the ground server and the same training data is used for training (the training accuracy is shown in Table 2), re-test the unsatisfactory test results on the UAV terminal (the test results are shown in Figure 6), and then send it back to the Internet server terminal to complete the inspection task of the power line corridor. From the comparison of Figure 5(c) and Figure 6, it can be seen that the YOLOV4 network improves the overall detection accuracy, especially for the detection of the bird’s nest. The confidence level is increased from 0.29 to 0.89. Therefore, this paper uses edge computing to more accurately inspect power lines.

The YOLOV4 and YOLOV4-Tiny algorithms are used to detect the power lines in the same scene, and the time overhead of the two algorithms to detect a single picture respectively is obtained. The time performance comparison is shown in Table 3.

Table 3 Performance comparison between YOLOV4 and YOLOV4-Tiny

Algorithm Name	Time/ms
YOLOV4	21.786
YOLOV4-Tiny	4.229

From the above results, it can be seen that the mAP of YOLOV4-Tiny is lower than YOLOV4. Although the detection accuracy rate is lower than YOLOV4, the detection speed is 5.2 times that of YOLOV4.In addition, YOLOV4-Tiny can meet the hardware requirements of drone equipment while ensuring high detection accuracy. By using 5G edge computing to make up for the problem of YOLOV4-Tiny algorithm missed detection, to achieve faster and more accurate real-time detection of power lines.

4 Conclusions

This paper combines the good performance of the deep learning target detector, and uses 5G and edge cloud computing technology to form a real-time power inspection framework for edge cloud drones, and proposes a real-time power detection method for 5G and edge computing based on YOLOV4-Tiny. It can meet the needs of real-time detection of power lines. Tests in a large amount of real power inspection data show that through 5G edge cloud data distribution and terminal sinking, the data that requires a second verification is transmitted to the ground server and finally reaches an accuracy of about 88%. At the same time, the UAV terminal can achieve a detection efficiency of about 236.4 sheets/s, so the method in this paper can achieve fast and efficient power line inspection. At the same time, this method has lower requirements for hardware equipment and can meet the hardware design of the drone pod. In this experiment, the ground server is still equipped with a relatively lightweight target detection network. In the follow-up work, a better performance detection network can be used to achieve comprehensive coverage detection of power defects.

Acknowledgments

This research is supported by the Research on UAV Line Inspection Technology Based on Beidou $+$ 5G(2020YF-62).

References

[1] Huang Shan, Wu Zhenshen, Ren Zhigang, et al. Review of electric power intelligent inspection robot[J]. Electrical Measurement and Instrumentation, 2020, 57(02):26–38.

[2] WU Shaopeng. Aviation Material Support Capacity Based on Information Management[J]. Applications of IC, 2020, 37(05):122–123.

[3] Yi Xiaofei. Application of Machine Vision in UAV Power Inspection[D]. Shandong University, 2020.

[4] Zhou Niancheng, Liao Jianquan, Wang Qianggang, et al. Analysis and Prospect of Deep Learning Application in Smart Grid[J]. Automation of Electric Power Systems, 2019, 43(04):180–191.

[5] Feng Jinhua, Zhang Xiaoqiu, Zhang Hao. Edge computing in UAV emergency mapping[J]. Telecommunications Science, 2019, 35(S2):110–118.

[6] Zhou Zhubo, Gao Jiao, Zhang Wei, et al. Object detection of transmission line visual images based on deep convolutional neural network[J]. Chinese Journal of Liquid Crystals and Displays, 2018, 33(04):317–325.

[7] Li Hui, Zhong Ping, Dai Yujing, et al. Study on detection method of transmission line rusty based on deep learning[J]. Electronic Measurement Technology, 2018, 41(22):54–59.

[8] Ren S Q, HE K M, GIRSHICK R B, et al. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks[J]. IEEE transactions on pattern analysis and machine intelligence, 2017, 39(6):1137–1149.

[9] Guo Tao, Yang Heng, Shi Lei, et al. Self-Explosion Defect Identification of Insulator Based on Faster Rcnn[J]. Insulators and Surge Arresters, 2019(03):183–189.

[10] Wei Dong, Gong Qingwu, Lai Wenqing, et al. Research on Internal and External Fault Diagnosis and Fault-selection of Transmission Line Based on Convolutional Neural Network[J]. Proceedings of The Chinese Society for Electrical Engineering, 2016, 36(S1):21–28.

[11] Mao Xianyin, Liu Yu, Ma Xiaohong, et al. Obstacle identification of transmission line inspection robot based on SSD[J]. Automation and Instrumentation, 2020(05):45–48.

[12] Zou Fangrong, Fan Ming, Ma Yutang, et al. Fast detection method of transmission line defects based on Yolo V3[J]. Yunnan Electric Power, 2020, 48(04):112–116+120.

[13] ESTI. Multi-access edge computing(MEC) [EB/OL]. (2018-11-05) [2020-03-17]. http://www.stsi.org/technologies-clusters/technologies/multi-access-edge-computing.

[14] Bai Yuyang, Huang Yanhao, Chen Siyuan, et al. Cloud-edge Intelligence: Status Quo and Future Prospective of Edge Computing Approaches and Applications in Power System Operation and Control[J]. ACTA AUTOMATICA SINICA, 2020, 46(03):397–410.

[15] Feng Jijia. Implementation and research of automatic line inspection of power transmission lines based on 5G UAV[J]. Electric Power Equipment Management, 2020(09):203–204.

[16] Lv Yajing, Teng Ling, Xing Ya, et al. Application Status of Beidou Satellite Navigation System in Power Industry[J]. Electric Power Information and Communication Technology, 2019, 17(08):70–74.

[17] Zhou Zhenyu, Chen Yapeng, Pan Chao, et al. Ultra-reliable and Low-latency Mobile Edge Computing Technology for Intelligent Power Inspection[J]. High Voltage Engineering, 2020, 46(06):1895–1902.

[18] Wu F, Zhao H K, Wang M L. Nighttime cattle detection based on YOLOv4[P]. Northwest A&F Univ. (China); Hangzhou Normal Univ. (China); Xi’an Univ. of Technology (China), 2021.

[19] Redmon J, Farhadi A. YOLOv3: An Incremental Improvement[J]. arXiv e-prints, 2018.

[20] Chen Zhitao. The Research on Image Target Recognition Based on Deep Learning[D]. Harbin Engineering University, 2018.

[21] Wang C Y, Liao H, Yeh I H, et al. Cspnet: a new backbone that can enhance learning capability of cnn[J]. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019.

[22] Zhang Xin, Zhang Yongqiang, He Bin, Li Guoning. Research on remote sensing image aircraft target detection technology based on YOLOv4-tiny[J]. Optical Technique, 2021, 47(03):344–351.

Biographies

Jiaqi Song, graduated from Shenyang Institute of Engineering in 2012, engaged in the operation and maintenance of high-voltage transmission lines in 2013, participated in the national live working skills competition in 2014, and won the eighth place in the group. During 2015–2019, he successively won the third prize of scientific and technological innovation of State Grid Liaoning Electric Power Co., Ltd., one second prize of Liaoning Province quality scientific and technological achievements, and two third prizes of State Grid Chaoyang Power Supply Company.

Jing Li, Familiar with the advanced geographic information, remote sensing and mapping technology at home and abroad, presided over a number of national large and medium-sized geographic information data service projects, and had rich practical work experience and achieved certain work achievements; Many jobs have trained strong adaptability and management ability, and have been able to lead the team to carry out the related work such as overall design, top-level architecture design, market development, project management, etc.

Di Wu, State Grid Chaoyang Power Supply Company Transmission Area Safety Management Specialty. I have been engaged in power transmission for 18 years. The main papers include “Research on Safety Management Method of High-voltage Transmission Line Erection” and “Analysis of Restrictions and Technical Innovation in Transmission Line Construction and Operation”. The research and development of “a kind of transmission line self-explosive insulator replacement universal fixture”, “a live pick up R-type spring pin device” and other new achievements won the national utility model patent. Compile and record the teaching video of “general fixture operation standard”, which won the second prize of the company teaching video.

Guangye Li is working in State Grid Liaoning Electric Power Co., Ltd. currently, deputy senior engineer, mainly engaged in the application of new technologies such as 5G, Beidou and artificial intelligence. He has presided over, organized and participated in a number of informatization engineering projects issued by national, provincial and ministerial and provincial companies, and participated in the construction of the electric Beidou precision space-time service network, the Beidou space-time intelligent integrated service platform and the GIS platform, laid the foundation for high-precision location services, navigation, high-precision time-frequency reference and short message communication services.

Jiaxin Zhang is working in State Grid Liaoning Electric Power Co., Ltd. currently, mainly engaged in the application of new technologies such as 5G and Beidou. He has participated in the construction of the electric Beidou precise space-time service network and the Beidou space-time intelligent integrated service platform. A number of business scenarios have been developed in the power field such as transportation inspection and marketing, effectively promoted the implementation of the Beidou information service system, laid the foundation for building a leading domestic precision space-time service company.

Jiantie Xu is a postgraduate student, received his graduate degrees in Electrical Engineering from Xi’an Jiaotong University, working for the State Grid Shenyang Electric Power Supply Company now, mainly engaged in the application of Beidou technology, participated in construction of the Beidou space-time intelligent integrated service platform and dozens of Beidou ground enhancement stations, at the same time combined with the unique complex and diverse regional characteristics of Liaoning, carried out multiple business scenes in power fields such as operation, inspection, marketing, which greatly improve the efficiency of inspections, work safety and emergency response capabilities.

Tian Lan graduated from Wuhan University majoring in geographic information system, engaged in the research and application of geographic information, remote sensing, and Beidou global positioning technology.

Distributed Generation & Alternative Energy Journal, Vol. 36_4, 385–402.
doi: 10.13052/dgaej2156-3306.3643
© 2021 River Publishers