Paper From…

• IEEE 24th International Conference on Embedded

and Real-Time Computing Systems and
Applications (RTCSA)
• Year 2018
Deep Counter: Using
Deep Learning to
Count
Garbage Bags
 What is Deep Learning???
• Deep learning is a machine learning technique that
teaches computers to do what comes naturally to humans
• In deep learning, a computer model learns to perform
classification tasks directly from images, text, or sound.
• Examples: Automated Driving, Aerospace and Defense,
Medical Research, Industrial Automation, Electronics etc.
Objectives…
• Using deep learning based image processing
technology to automatically count the number of
collected garbage bags from the video taken.
Methodology…
• Propose a sensing system mounted on garbage trucks for
collecting fine-grained spatio-temporal information of garbage
disposal
• A detection-tracking-counting DTC algorithm for estimating
the number of targeted garbage bags is also proposed.
• A prototype system is implemented, based on which
performance evaluation is conducted using the video of
realistic garbage collection of Fujisawa City, Japan.
Introduction
• Various kinds of waste are generated in our daily life, making garbage
collection one of the fundamental public services in modern cities.
• While the data in Figure 1 already express a general trend to a certain
extent, further information on the spatiotemporal distributions of
garbage disposal, i.e., the time, location and category of the collected
waste, is required in order to develop novel applications in waste
collection service.
Figure 1
Introduction (Contd…)
• The categories and amount of waste can be recognized and measured by
humans like collection workers or inhabitants, it leads to an extremely high
cost in labor and time that is unacceptable by the municipality or
inhabitants.
• The weight of collected garbage can be automatically measured by sensors
mounted on garbage trucks, but the category remains unknown.
• To sum up, it is required to study a sensing system that is able to
automatically recognize the category and measure the amount of the
collected garbage.
 RELATED WORK
A. Automotive Sensing:
• It is a novel urban sensing technology where sensors are equipped to
vehicles and utilize their mobility to help to conduct sensing tasks
• Rezaei et al. proposed a Kalman filter integrated estimation algorithm
where GPS, wheelspeed/steering angle and gyroscope serve as the
input, and showed via experiment that a ≤ 1m estimation accuracy can
be achieved even when GPS outage happened.
• Other Applications: checking status of transportation infrastructure,
detect the blurred road markings, environment monitoring etc
 RELATED WORK
B. CNN-based Object Detection:
• Girshick et. al. proposed R-CNN and Fast R-CNN which first extract
region proposals with selective search for object in the input image,
compute CNN features through resizing the image, and finally classify
regions and estimate bounding box of the detected object via
regression.
• The end-to-end detections like Faster R-CNN, YOLO and SSD are
proposed where all parameters are trained jointly in order to achieve a
better real-time performance.
• In this study, we will adopt SSD as the underlying detection
technology, while other alternatives can also be applied in general.
 SYSTEM OVERVIEW
A. Garbage Collection in Japan
 SYSTEM OVERVIEW
• System Architecture:
• The DeepCounter consists of a camera, a GPS receiver, an embedded
computer and a cellular module as shown in Fig.
• In our study, a NVIDIA Jetson TX2 development Kit is adopted as the
embedded computer, mainly because it contains a GPU to accelerate image
processing. A GPS receiver is also required to obtain location of the truck
DETECTION-TRACKING-COUNTING
ALGORITHM
• The DTC algorithm consists of three processes: object detection,
tracking and counting, the functions of which are presented as follows.
• 1) Detection: Detect the location of garbage bags in each frame of
the input video stream.
• 2) Tracking: Identify the same garbage bag between successive
frames.
• 3) Counting: Determine whether a garbage bag has been collected
into the garbage truck’s container.
Detection:
• Convolution neural network-based object detection algorithm
(customized SSD) is used in the detection process.
• Once a targeted garbage bag is detected with a confidence not less
than a given threshold τc, the detection process will send the
corresponding location region to the tracking process.
• The location region of an object is represented as a 4-tuple (xmin, ymin,
xmax, ymax), the elements of which represent the coordinates of the
upper-left and bottom-right corners, respectively.
Tracking
• When the tracking process received the location regions(s) of detected bags,
it will check whether these objects are the same ones detected from the
previous frames.
• In this study, an intersection over union (IoU) based approach is adopted for
tracking. The IoU is defined as the ratio of the area of intersection over that
of the union of two location regions.
Counting:
• A bag in the current tracking list is counted as being collected if the
following condition is satisfied.
• The reference point of the bag was above the counting barrier in previous
frames.
• The reference point of the bag is below the counting barrier in the current
frame.
• When a bag is collected, it will be removed from the tracking list.
EXPERIMENTAL EVALUATION
Conclusion:
• A sensing system using a camera mounted on garbage trucks to
count the number of collected bags was proposed.
• We also implemented a prototype system and conducted
experiment using the video of garbage collection to evaluate the
performance.
Future Work:
• The low precision indicates that further improvement is required.
• One possible way is to increase the frame rate of the video stream.
Therefore, one of our future work is to find novel detection, tracking and
counting approaches that improve both accuracy and processing speed.
• Our experiment also demonstrates that environment factors like weather and
shadow may also decay the accuracy performance, so the technique to
alleviating the negative effect of these factors is another important future
research direction.
References
• Kitagawa&Amagasa Data Engineering Laboratory, Tsukuba University, “Gomi G1
Grand Prix,” http://bigdata.kde.cs.tsukuba.ac.jp/∼g1gp/index. html (In Japanese,
last visited on Apr. 20th, 2018).
• S. Rezaei and R. Sengupta, “Kalman filter-based integration of dgps and vehicle
sensors for localization,” IEEE Transactions on Control Systems Technology, vol. 15, no.
6, pp. 1080–1088, Nov 2007.
• S. Fujii, A. Fujita, T. Umedu, S. Kaneda, H. Yamaguchi, T. Higashino, and M. Takai,
“Cooperative vehicle positioning via v2v communications and onboard sensors,” in
2011 IEEE Vehicular Technology Conference (VTC Fall), 2011, pp. 1–5.
• M. Kawano, K. Mikami, S. Yokoyama, T. Yonezawa, and J. Nakazawa, “Road
marking blur detection with drive recorder,” in 2017 IEEE International Conference on
Big Data, Dec 2017, pp. 4092–4097.