Acoustic Event Localization and its Video Tracking in 3D Space

(2015-FYP-24)
DEPARTMENT OF ELECTRICAL ENGINEERING

Abstract Objectives Hardware Setup

The main goal of the project is to provide a cost-
effective solution for detection and tracking of
criminal usage of firearms. It involves localizing the
event (gunshot, etc.) on the basis of sound, and then
tracking it through video feed from camera.
Previously, acoustic event localization in 2D plane and
human tracking within video stream from fixed
camera have been achieved. We have extended it to
3D space, while tracking with pan-tilt camera.
Microphones, camera, stepper motors and multiple
microcontrollers are being employed for this purpose.
In addition, offline sound signature comparison has
been carried out. This project is an important step
forward in cost-efficient intelligent surveillance, and
further work can be done on inter-node camera
handover and the networking of many such nodes in
order to develop a more complete product.
• 3D sound localization using four microphones.
• Reducing size of sound localization hardware.
• Target tracking with motorized camera.
Methodology
• STM32F407 microcontroller samples 4 microphones
using ADCs and determines TDOAs.
• Using a Cartesian arrangement of four microphones,
the direction of the sound source is calculated.
• The camera is pointed in this direction by moving the
pan and tilt stepper motors.
• Odroid embedded Linux machine detects the target in
the camera’s video stream using the HOG human
detector and tracks it using TLD or CSRT.
• As the target moves, the motors are also moved to
keep the target at the center of the camera’s FOV..

Flowchart
Sound Direction (Angle) of Arrival Estimation Results

Tracking Results (screenshots at 1 second intervals)

Future Improvements Summary Acknowledgements

• Complete PTZ operation, i.e. zoom in addition to pan and tilt. Detection of EOI (Event of Interest) using Prof. Dr. Muhammad Tahir (Project Supervisor).
• Inter-node communication for camera handover 3D sound source localization and tracking NWN Lab, KICS, UET, Lahore.
• Cheap self-calibration for nodes at varying heights of target with motorized pan-tilt camera.
• Transmission of video feed to a server machine

References
Members Project Advisor
• Ali Pourmohammad and Seyed Mohammad Ahadi. Real time high accuracy 3-d phat-based sound source localization using a
Muhammad Usama Zubair 2015-EE-015 Prof. Dr. Muhammad Tahir simple 4-microphone arrangement. IEEE Systems Journal, 6(3):455-468, 2012.
Ansar Rasool 2015-EE-080 • Hammad Hassan and Muhammad Tahir. Multimodal Sensing Enabled Real-time Intelligent Wireless Camera Networks for
Hasnat Sajid 2015-EE-191 Project Co-Advisor Secure Spaces (May,2013--Dec,2015) - Deliverables. http://www.kics.edu.pk/project/aad/74, Last accessed on March 13, 2019.
Muhammad Husnain 2015-EE-039 • Mohamed Hussein, Wael Abd-Almageed, Yang Ran, and Larry Davis. Real-time human detection, tracking, and verification in
Dr. Ubaid Ullah Fayyaz uncontrolled camera motion environments. In Computer Vision Systems, 2006 ICVS’06. IEEE International Conference on, pages
41-41. IEEE, 2006.