Volume 4, Issue 12, December – 2019
ISSN No:-2456-2165
Towards the Development of a Real-time
Deep Surveillance Industrial Scene Human
Activity Recognition System to Mitigate
Gas Pipe-lines Vandalization
Onuh Gabriel
Department of Computer Engineering,
Ahmadu Bello University, Zaria, Nigeria
Abstract:- Human activity detection and recognition
(HADR) is an important field of study in computer
vision, due to it wide application areas like security
surveillance, robotics, human computer interaction,
content based retrieval and annotation, human fall
detection etc.,. The essence of HADR is to automatically
understand and recognized what kind of action (human
behavior and activities) is performed by human in a
video captured by a surveillance system. This is really a
difficult problem due to many challenges involved in
HADR. These challenges include: cluttered
backgrounds, variation in motion and human shape,
variation in illumination conditions, occlusion, and
viewpoint variations. However, the intensity of these
challenges may vary depending on the category of an
activity under consideration. Generally, the activities
are grouped into four classes which constituent, human-
gestures, actions, interactions, and group activities, this
division is mainly based on complexity and duration of
the activities. Due to the advancement in sensor and
visual technology, HADR based systems have been
widely used in many real-world applications.
Specifically, the increase of small size sensors have
enabled the smart devices to recognize the human
activities in a context-aware manner.
Hence, with HADR numerous application area we
propose a deep surveillance industrial scene human
activity detection to fight against gas pipe-line
vandalizing, where, the recognition scheme can
effectively detect any suspicious activity and report via
sending a notification to the authorities for immediate
action.
Keywords:- Component; Real-Time Surveillance; Human
Activity Recognition; KNN; Wireless Sensor Networks.
IJISRT19DEC608 www.ijisrt.com
International Journal of Innovative Science and Research Technology
Akan J. Bello
Department of Computer Engineering
Ahmadu Bello University, Zaria, Nigeria
I. INTRODUCTION
Gas and oil pipeline vandalization in Nigerian has
significantly affected the oil sector and the country at large
in numerous ways which includes huge economic losses
from pipeline & plant shutdown, pollution of environment,
by oil spillage, fire outbreaks usually resulting in loss of
lives amongst others. Thus, shortage and scarcity, of
petroleum products as well as decrease in electricity-
supply, with the attendant socio-economic problems can
also be attributed to pipeline vandalism [10]. Petroleum
products in Nigeria, are mostly transported via a network of
pipeline running across various part of the country through
remote area to populated areas however, these pipelines are
poorly secured thereby attracting various forms of
repetitive attacks targets by vandals. Countless measure
have been taken by government to ensure efficient service
delivery but the problems appear to be increasing [11].
Hence, our goal is to present a modern and holistic
approach to addressing this problem by adopting state of
the art digital surveillance and efficient machine learning
algorithms. The rest of this paper, is structured as follows,
in section II, literatures related to the works are presented in
brief, while in section III, methodology adopted for the
proposed work is equally presented, and finally section IV
concludes the proposed study.
II. REVIEW OF RELATED WORKS
A number of research interest is drawn to the area of
pipeline surveillance due to the economic importance of
pipelines as well as the health and safety implications of
pipeline incidents. Thus, Wireless-Sensor-Network (WSN)
has been identified and presented as a suitable solution for
addressing pipeline surveillance related problem via
sensing the environment and it is cost-effective. Asides
pipeline surveillance, other related applications domain for
WSN in oil and gas sector ranges from detecting leakage in
pipelines, monitoring tank level, Equipment Condition
Based Monitoring (CBM), Pipeline Pressure Relief Value
Monitoring (PRV), Refineries-Pressure-Relief-Value-
Monitoring (PRV) and Well-head-Automation and
Monitoring. However, most pipeline surveillance systems
are focused on leak detection [1]; [2], few pipeline
1197
Volume 4, Issue 12, December – 2019
surveillance systems have tried to address threat detection
in pipelines. Sun and Wen [3] conducted a study on
pipeline threat detection and security, thereby developing a
pre-warning scheme for pipeline security using multi-
seismic sensors. Liang et. al [4] performed a risk
assessment studies on pipelines based on malicious and
accidental threats. Furthermore, fault tree was used by the
authors to decide the risk assessment index, thereafter a
self-organizing map scheme (SOM) was employed to
perform risk level classification on various section of the
pipeline. An effective WSN based framework for integrated
oil pipeline monitoring aimed at threat mitigation for
architectural model in pipelines is presented by Jawhar et
al. [5] while, in Seema and Reisslein [6] a wire1ess-video-
sensor node architecture scheme was developed -
(WVSNP). The hardware and software requirements for
WVSNP were discussed. Thus, various WSN models have
been developed, while the dynamic linear configuration
model is identified to be as the more suitable for pipelines
due to its linear nature Mohamed and Jawhar [7] and
Jawhar et. al [5]. Utilizing the present day technology,
Wireless sensor networks having a wide range of
applications. Hence, WSN used for intelligent
transportation system will relatively differ from those used
for telemedicine and other services in various ways. WSN
used for pipeline surveillance should be effective in
detecting leakages and threats to pipelines and localize such
events with certain degree of accuracy. In designing an
effective WSN for pipeline surveillance system, three (3)
key features are considered they consist of the following:
adequate wireless mote, topology for deployment, and data
mining technique [12]. Oil and gas pipelines by nature pass
through large geographical areas. Therefore, wireless motes
utilized for pipeline surveillance should employ power
optimization systems for the purpose of energy
conservation while being able to transmit over a
considerable distance. In WSN these factors considered
during design depend on the specific task at hand even
though a lot of factors are generic for various tasks [12]. In
Boonjun et al [8] a real-time automatic object detection-
technique was developed using template-matching for
pipeline protection, while in the work of Goel et al [ 9] an
on-line scheme for temporal detection of human daily-
living activities using a long untrimmed video streams is
presented.
However the reviewed techniques did not employ any
machine learning algorithm to train, classify and enable the
system detect and even predict any vandalism and other
illegal activities before they occur. Our proposed system
intends to address that challenge as the emphasis is on
prevention by developing a system that not only detects but
also predicts possible occurrence.
IJISRT19DEC608 www.ijisrt.com
International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165
III. METHODOLOGY
Deep surveillance system in industrial scene will be
made up of of three modules which include client module,
application module and database server module. In the
client section, it will include an android based application
for receiving various notifications from the scene detection
scheme, such as anomaly event detection notification, this
will be receive only by authorized user. The Application
phase (module), will be made up of the core human activity
detection and recognition system. Whereas at the database
end, a Google-server and apache tomcat local-server would
be integrated to handle activity recognitions of the
proposed scheme, for gas pipe-line vandalism, legal and
illegal industrial human activity scenario, dataset would be
created at industrial environment. If actions like pipe-
vandalism or any illegal activity are detected by the system,
it forwards a message signal to the user with registered and
installed android based application as notification. Video
sequence sense from an industrial environment is taken as
input to system, followed by performing grayscale
conversion and various forms of pre-processing, then
feature extraction is performed using scale invariant feature
transform (SIFT). The extracted features are used for action
classification purpose. SVM and KNN classifiers are used
for activity classification.
Firebase cloud messaging (FCM) would be
incorporated in the developed deep surveillance industrial
scene human activity detection and recognition scheme for
sending notification to registered device. This service
would be utilized for transferring a payload of up to about
4KB to a client application using FCM for an instant
messaging to inform the client about the detected
suspicious action and it location.
The outlined methodology would be carried out in the
development of deep surveillance industrial scene human
activity recognition to mitigate vandalizing gas pipe-lines.
 Data Collection
 Pre-processing
 Feature Extraction
 Clustering and Classification
 Pattern/template Matching
 Activity Recognition
A. Data Collection
Our video surveillance system would monitor the
environment (pipeline installation route) and capture
various forms of activities, behavior, or other changing
information, usually, of people from a distance by means of
sensing device. This is the first step towards prevention,
reaction, detection, intervention. The data captured is
transmitted in real time to a base facility.
1198
Volume 4, Issue 12, December – 2019
Fig 1:- Hardware Setup for Data Collection
B. Pre-processing
It is critical to carry out the data processing on the
collected surveillance feed as the quality of the surveillance
might be affected by weather, poor lighting signal
interference and so on. Some of the preprocessing activities
includes, compression, de-noising, segmentation,
equalization etc.
C. Feature Extraction
Feature extraction is carried out on the preprocessed
surveillance feed. First of all the background subtraction
operation is done on the surveillance, then the feature of
interest is extracted for clustering and classification.
D. Clustering and Activity Classification
The Support-Vector-Machine-(SVM) and K-Nearest-
Neighbor (KNN) are the machine learning tools that would
be employed to cluster, classify and train the system based
on the collected surveillance dataset on different activities,
movements and positions etc. Unlike conventional
approaches this technique would enable the system make
smart and intelligent inferences and predictions on possible
threats even before they occur. Equation 1 shows the
mathematical model of KNN prediction.
1
𝑌 = 𝐾 ∑𝑘𝑖=1 𝑦𝑖 (1)
Here, yi is define as the ith case of the examples
sample and y denoted to be the prediction (outcome) of the
query point.
E. Activity Recognition
Our system would track individual as they pass
through the field of vision of the camera, and with machine
learning algorithms to classify the motion and activities of
every passerby that walks a stipulated radius to our pipeline
infrastructure. The tracking is accomplished through the
development of a position and velocity path characteristic.
With this information, our system would report any threat
or suspected threat to the attention of human security
personnel.
IJISRT19DEC608 www.ijisrt.com
International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165
Fig 2:- Block Diagram of Deep Surveillance Human
Activity Recognition System.
IV. CONCLUSION
The impacts of this research are numerous. Some of
the benefits are; prevention of pipeline vandalism,
reduction in financial losses, alleviation of environmental
degradation, security and safety, increased oil revenue
amongst. By adopting state-of-the-art surveillance devices
and a robust machine learning algorithm, the proposed
system would be able to completely solve the problem of
pipeline vandalism which poses a major threat to the
operations and optimal yield of our oil and gas industry.
REFERENCES
[1]. T. AL-Kadi, Z. AL-Tuwaijri, and A. AL- Omran,
“Wireless Sensor Networks for Leakage Detection in
Underground Pipelines: A Survey Paper,” 5th Int.
Symp. Appl. Ad hoc Sens. Networks, vol. 21, no. 0,
pp. 491–498, 2013.
[2]. A. F. Colombo, P. Lee, and B. W. Karney, “A
selective literature review of transient-based leak
detection methods,” J. Hydro-Environment Res., vol.
2, no. 4, pp. 212–227, 2009.
[3]. J. Sun and J. Wen, “Research on Monitoring and Pre-
warning System for Security of Pipelines Based on
Multi-Seismic Sensors,” Int. Conf. Electron. Meas.
Instruments, pp. 1–5, 2009.
[4]. W. Liang, J. Hu, L. Zhang, C. Guo, and W. Lin,
“Assessing and classifying risk of pipeline third-party
interference based on fault tree and SOM,” Eng. Appl.
Artif. Intell., vol. 25, no. 3, pp. 594–608, Apr. 2012.
[5]. I. Jawhar, N. Mohamed, and K. Shuaib, “A
framework for pipeline infrastructure monitoring
using wireless sensor networks,” in Wireless
Telecommunications Symposium, 2007. WTS 2007,
2007, pp. 1–7.
1199
Volume 4, Issue 12, December – 2019 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165
[6]. A. Seema and M. Reisslein, “Towards Ef fi cient
Wireless Video Sensor Networks : A Survey of
Existing Node Architectures and Proposal for A Flexi-
WVSNP Design,” IEEE Commun. Surv. Tutorials,
vol. 13, pp. 462–486, 2011.
[7]. N. Mohamed and I. Jawhar, “A Fault Tolerant
Wired/Wireless Sensor Network Architecture for
Monitoring Pipeline Infrastructures,” in Sensor
Technologies and Applications, 2008.
SENSORCOMM ’08. Second International
Conference on, 2008, pp. 179–184.
[8]. Boonjun, K., Keeratipranon, N., & Chinpanthana, N.
(2018). Real time automatic object detection by using
template matching for protecting pipelines. Paper
presented at the 2018 International Workshop on
Advanced Image Technology (IWAIT).
[9]. Goel, A., Abubakr, A., Koperski, M., Bremond, F., &
Francesca, G. (2018). Online temporal detection of
daily-living human activities in long untrimmed video
streams. Paper presented at the 2018 IEEE
International Conference on Image Processing,
Applications and Systems (IPAS).
[10]. Udofia, O., & Joel, O. (2012). Pipeline vandalism in
Nigeria: Recommended best practice of checking the
menace. Paper presented at the Nigeria Annual
International Conference and Exhibition.
[11]. Kabir, A. (2016). Project-technological approach to
solve the crude oil theft, sabotage and pipeline
vandalization problem in Nigeria. Управление
проектами и развитие производства(4 (60)).
[12]. Eze, J., Nwagboso, C., & Georgakis, P. (2017).
Framework for integrated oil pipeline monitoring and
incident mitigation systems. Robotics and Computer-
Integrated Manufacturing, 47, 44-52.

IJISRT19DEC608 www.ijisrt.com 1200