Journal of Real-Time Image Processing

https://doi.org/10.1007/s11554-019-00852-3

ORIGINAL RESEARCH PAPER

Convolution neural network joint with mixture of extreme learning

machines for feature extraction and classification of accident images
Ali Pashaei1 · Mehdi Ghatee1   · Hedieh Sajedi2

Received: 12 August 2018 / Accepted: 16 January 2019

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Abstract
This paper considers the accident images and develops a deep learning method for feature extraction together with a mixture
of experts for classification. For the first task, the outputs of the last max-pooling layer of a Convolution Neural Network
(CNN) are used to extract the hidden features automatically. For the second task, a mixture of advanced variations of Extreme
Learning Machine (ELM) including basic ELM, constraint ELM (CELM), On-Line Sequential ELM (OSELM) and Kernel
ELM (KELM), is developed. This ensemble classifier combines the advantages of different ELMs using a gating network
and its accuracy is very high while the processing time is close to real-time. To show the efficiency, the different combina-
tions of the traditional feature extraction and feature selection methods and the various classifiers are examined on two kinds
of benchmarks including accident images’ data set and some general data sets. It is shown that the proposed system detects
the accidents with 99.31% precision, recall and F-measure. Besides, the precisions of accident-severity classification and
involved-vehicle classification are 90.27% and 92.73%, respectively. This system is suitable for on-line processing on the
accident images that will be captured by Unmanned Aerial Vehicles (UAV) or other surveillance systems.

Keywords  Feature extraction · Accident images’ classification · Convolutional neural networks · Mixture of ELM ·
Ensemble learning

1 Introduction On the other hand, there are many researches on accident

A system for accident management plays an important role
in traffic control and emergency systems. In such systems,
the data of the different sources can be collected to support
injured people [1]. One of the most important data sources
is image source [2, 3]. Fixed cameras or portable ones can
capture these images but the latter is more effective. In [4], a
drone was used for freeway monitoring system. Cao et al. [5]
proposed a system for vehicle detection and tracking using
Unmanned Aerial Vehicles (UAVs). Also in [6], a similar
system has been developed to monitor road conditions.
* Mehdi Ghatee
ghatee@aut.ac.ir
Hedieh Sajedi
hhsajedi@ut.ac.ir
1
Department of Computer Science, Amirkabir University
of Technology, Tehran, Iran
2
School of Mathematics, Statistics and Computer Science,
College of Science, University of Tehran, Tehran, Iran
analysis. To get a real-time system to support the accidents,
different image-processing methods have been developed
[2, 3]. Some of these methods apply machine leaning algo-
rithms such as artificial neural networks (ANN) [7]. Oth-
ers have focused on accident characteristics. For example,
Chioun and Fu [8] conducted an integrated study of crash
frequency and severity. They mentioned that the road and
the traffic conditions, the land-use and cultural parameters
influence the frequency and the severity of the incidents.
The type of vehicle has also a major effect on the severity
of the accident. Many works have investigated the severity
of truck accidents [9].
In continuation of these researches, this paper focuses
on image processing and machine learning for accident
analysis. To this aim, we first emphasise on feature extrac-
tion from accident images. The traditional methods for this
task such as Local Binary Pattern (LBP) [10], Histogram of
Gradient (HOG) [11], Maximally Stable Extremal Regions
(MSER) [12] and Speeded-Up Robust Features (SURF) [13]
are considered at the first; however, we will show that they
are not specialized for accident analysis. On the other hand,

13
Vol.:(0123456789)
 Journal of Real-Time Image Processing

Table 1  Some related works and systems in accident detection and feature extraction from accident
Ref. no. Goal Proposed method Limitation

[19] Accident detection Using histogram of flow gradient for accident detection Tracking the individual in crowded scenes was
impossible
[20] Fusion of the wireless sensors Maybe the sensors are not active*
[21] Using neural networks, Radon transform for angle It used satellite images for accident detection*
detection and traffic-flow measurements
[22] Using Gaussian Mixture Model to detect vehicles; then It cannot be used in the different weather conditions*
mean shift algorithm for vehicle tracking
[23] Using SVM and neural network It is limited to some special sensors
[24] Accident detection by GPS and GSM GPS data cannot be accessed everywhere
[25] Vehicle classifica- Using Eigen-window method on images provided by Its accuracy is low and it cannot detect motorcycles*
tion (no accident) B-snake
[26] Using YOLO model for vehicle detection and AlexNet It did not detect motorcycles*
for vehicle classification
[27] Using k-mean values for initial labelling and then using It used fixed CCTV images for classification*
linear SVM to identify the low-confident samples
[28] Using CNN for vehicle type detection The running time is high
[29] Severity of accident Using gradient boosting to analyze the relationship It cannot be used in the different weather conditions*
between crash severities and risk factors
[30] Using a series of neural networks to model the nonlin- It cannot estimate the risk
ear relationships between the injury severity levels
and crash-related factors
[31] Using decision tree for accident severity classification The focus is just accident severity*
[32] Using incident location, date, time, traffic lanes to It just focused on accident severity*
determine the severity of the accident

*These issues were solved in this paper

Convolutional Neural Network (CNN) is one of the best
methods for image processing in the recent literature [14].
It can extract the hidden features by the aid of pooling layers.
Usually, the outputs of the last pooling layer are applied for
classification and regression purposes. We propose a mixture
of ELM for the classification task.
CNN is also capable of real-time image processing
issues. For example, Redmon et al. [15] used YOLO model
for object detection in images. They compared this model
with other models such as R-CNN and VGG and showed
that YOLO has better results in real-time detection. In other
work, Ahn [16] proposed a system for real-time video object
recognition using CNNs. This system has been implemented
on a mid-range FPGA and achieves a computational speed
greater than 170,000 classifications per second, and per-
forms scale-invariant object recognition from a 720 × 480
video stream at a speed of 60 fps. This shows that our pro-
posed system can be used in real-time applications.
In Table 1, some methods are reviewed that have been
used for accident-image processing. It is worthy to note that,
CNN takes the image using a series of convolutional, nonlin-
ear, and max-pooling layers. It provides a good description
about the image including useful features that can be applied
for classification purposes [17, 18]. In the latter reference,
ELM was applied to improve the learning speed of CNN.
They used the outputs of ELM after learning a limited epoch
of the CNN. But its recognition rate is almost small.
Besides, some of the ensemble methods have been used
for image processing and some classifiers were combined
with some special feature selection methods [19, 20].
Yu et al. [21] developed a hybrid of CNN and ELM for
image classification. They used CNN for feature extrac-
tion and ELM for classification. Nevertheless, the perfor-
mance of ELM is not excellent for general cases. In fact,
we do not have a fixed solution for all issues. In [22],
a shallow non-convolutional neural network was trained
by ELM for MNIST benchmark. The results showed that
the classification error rate was about 1%. Such evidences
encourage us to develop an ensemble method based on
ELM for accident images analysis. Li et al. [20] proposed
a dynamic ensemble classifier that uses a random feature
selection method to generate diverse classifiers. In [19], a
three-phase ensemble system based on Mixture of Experts
(ME) has been proposed. The authors selected an optimal
subset of features in the first phase and used them for train-
ing of the experts with the standard ME algorithm. In all
of these ensemble methods, the learning algorithms must
be adapted by incorporating a penalty correlation term in
their error functions [23, 24]. In addition, the implicit and
explicit features can be used in these algorithms and they

13
Journal of Real-Time Image Processing

can complete each other [25, 26]. For example, in bagging

and boosting algorithms, a series of experts is sequentially
located and they are interdependent to each other. Thus,
they cannot provide cooperation among experts. Also, in
these methods, there is no feedback from the combination
phase to the learning phase. To see more details about
combination of implicit and explicit approaches, one can
refer to [27].
In this paper, we propose a system to analyze the acci-
dent images from various sources including UAV, fixed
cameras or portable cameras; however, we do not focus
on technological points. Instead, we emphasise on acci-
dent analysis to recognize the occurrence of accident, to
Fig. 1  The examined components of system for any task of accident
classify the involved vehicles in accidents and to classify analysis
the severity of accident. To do this, a new system is devel-
oped including two phases. In the first phase, we extract
the features by the max-pooling layer of CNN. Since the 2.1 System inputs
images are received from different sources and from differ-
ent angles, traditional algorithms cannot be used to extract In the first step, the system receives the images from the
the proper features, while CNN can. In the second phase, a accident scene. These images can be obtained from vari-
mixture of experts including different variations of ELM is ous sources, see, e.g., Puri [30] for a survey. The system then
developed. Experimental results show that this ensemble sends these images to detect the occurrence of an accident
algorithm is capable of classifying the images based on the and to extract its properties.
extracted features by CNN. The main contribution of this
paper can be expressed in the following:
2.2 System processes
• Technological viewpoint: This paper classifies the
properties of accidents in real-time (or at least close The proposed system, first, detects the occurrence of the
to real-time) and it is the most benefit of this work accident. Second, it extracts the characteristics of the acci-
compared with the previous ones. Also, the precision of dent. Since, UAV images are captured in different situations
system is acceptable for accident-support applications. and from different angles, so the powerful image processes
• Theoretical viewpoint: This paper fundamentally devel- and machine learning algorithms should be combined to
oped a mixture of experts (ME) instead of fully con- extract accurate results. Our proposed algorithm for this part
nected layer of CNN to enhance the quality of clas- consists of two main parts; feature extraction and classifica-
sification by the aid of different experts including the tion based on these features. In Sect. 3, the details of these
variants of ELM learning algorithms. parts are presented.

2 Accident analysis system
To analyze the accidents based on the image process-
ing, in this section, we study a system using CNN and
mixture of ELMs. This system takes images from vari-
ous sources, such as surveillance cameras, on-site people,
and UAVs, see, e.g., [28, 29]. Then the necessary features
are extracted and they are sent for classification. In the
first step of our proposed system, the system recognizes
whether or not the accident happens. If the response is
positive, the system calls two other ensemble classifiers
to recognize the types of involved vehicles in the accident
and the severity of the accident. Details are presented in
the following.
2.3 System outputs
The results of the processing part are sent to the control
center for making a decision for calling an accident manage-
ment procedure as quickly as possible.
3 System architecture
In this section, different algorithms for the feature extrac-
tion from accident images are discussed. On the extracted
features, we try to select the related features and classify the
images into defined classes. We need to examine the compo-
nents of Fig. 1 for any task of the accident analysis system.

13

Fig. 2  The output of traditional feature extraction methods: a original image, b HOG features, c MSER features, d SURF features
3.1 Traditional feature extraction
Due to the fact that the images are received from vari-
ous sources, image characteristics such as their angle and
intensity are different. Thus feature extraction from these
images, needs a hard work. The traditional feature extrac-
tion methods that are compared in our experiments are as
the following:
• LBP [10]: It has been developed for 2D texture analy-
sis. In this algorithm, a local structure is applied on the
images and each pixel is compared with its neighboring
pixels.
• HOG [11]: It calculates the gradient in x and y directions
for each pixel and gives the size and direction for each of
these gradients.
• MSER [12]: It is one of the bubble detection methods in
the image processing. It uses the input image intensity
range to detect stable regions. These regions are defined
when the variations in their colour intensities are not
greater than a threshold. This property examines the
different thresholds. These changes should be less than
the input value so that the region is detected as a stable
region.
• SURF [13]: It uses a square filter (Hessian matrix) to
determine the points. The Hessian matrix measures the
local changes around the pixels, and where the norm of
this matrix is maximized, the corresponding point is con-
sidered as a candidate for SURF features.
The output of features extracted from an accident image
is shown in Fig. 2. After extracting these features, they are
merged into a matrix. This causes redundancy in the matrix.
In addition, some of the extracted features are abusive fea-
tures for classifying step and it may lead to wrong results.
For these reasons, we need to find the proper features to clas-
sify the samples efficiently. For feature selection phase, three
methods are used; Principal Component Analysis (PCA)
[31], Correlation-based Feature Selection (CFS) [32] and
Bagging-Based Feature selection (BBF) [33]. Thus, for each
13
Journal of Real-Time Image Processing
image, first, we extract the features by LBP, HOG, SURF,
and MSER. Then, we save these features in a matrix. This
matrix is very large while the most entries are unprofitable
and they can decrease the accuracy. Finally, we use PCA,
CFS, and BBF on this matrix to reduce the dimension of
the features and also to select the relevant and important
features. It is worthwhile to note that PCA is one of the well
known algorithms for this purpose. On the other hand, PCA
is one of the common methods for band reduction in image
processing. For example, in [34], PCA method has been used
for band reduction on coloured images. Thus, it seems that
the usage of PCA is very useful for the current study.
3.2 Advanced feature extraction by convolution
neural network
When we tried to extract features from accident images by
traditional nonlinear feature extractors such as LBP, HOG,
SURF, and MSE, we found out that the classification accu-
racy was not acceptable. Probably, due to the collapse of
the vehicles involved in the accident scene and disruption
of colors, we cannot analyze the accidents based on the tra-
ditional features. In fact, the corner, the shape and the color
of vehicles in such images are completely irregular. Besides,
the accident images have been collected from different
sources and in many cases, the properties of images such as
light intensity and image angle are different. To extract the
meaningful features in these cases, extracting some hidden
and deep features, may improve the classification efficiency.
Since, CNN could greatly improve the detection accuracy
by extracting hidden features of images that are distorted by
various factors, we encouraged to apply CNN on accident
images. Indeed, one of the main advantages of CNN is its
ability to extract the proper features from large data disper-
sion. However, CNN can be also used to classify the images
[14]. Thus, we should present the exact role of CNN for
accident image analysis in our paper. For this end, we briefly
describe the details of CNN in our implementation. As pre-
sented in Fig. 3, this CNN includes the following layers:
Journal of Real-Time Image Processing
Conv 1
28×28 Conv 2
Input layer 14×14
MP
Pool[2,2]
F(3,16) F(3,32)
Stride[2,2] Padding[1,1] Stride[2,2]
Padding[1,1]
Fig. 3  Configuration of CNN baseline to classify the involved vehi-
cles in the accident (denoting “MP” as the max-pooling, “F” as the
filter, “Conv” as the convolution, “Pool” as the pooling size, “stride”
• Input layer This layer gets images with the size of
28 × 28 × 3.
• Convolution layer In our model, four convolution layers
are used. These layers consist of 16, 32, 64, and 128 fil-
ters. The size of filters in these layers is 3 × 3 and padding
size is 1.
– In the first convolution layer, the number of used
weights for a filter is 27 and the number of output is
28 that fully covered whole of image. The number
of neurons in this layer is 28 × 28 × 16.
– In the second convolution layer, the number of
weights is 32 × 32 × 3 and the number of output is 14.
– In the third convolution layer, the number of output
is 7.
– In the last convolution layer, the size of output is 3.
• Batch normalization layer After each convolution layer,
to normalize the features extracted by convolution layer,
we define the batch normalization layer. Our model con-
tains four batch normalization layers.
• Rectified linear unit (ReLU) layer This layer uses max
(0,x) as the activation function for each neuron, which
converts the negative values to zero. Our model contains
four ReLU layer.
• Max-pooling layer We define four max-pooling layers
in our proposed model. The pool size in these layers is
[2,2]. Also the stride size of each layer is [2,2]. The size
of the output of these max-pooling layers is 14, 7, 3, and
1, respectively.
• Fully connected layer Our model contains one fully con-
nected layer (see Fig. 3).
• Softmax layer The last layer converts fully connected
outputs to a probability distribution on the classes.
Now to extract the features from accident images, we
focus on some different issues in the accident images. As
one can note, the angle, the light intensity, the weather con-
dition, the accident time and the background of accidents
FC SM
Conv 3
7×7 Conv 4 car
3×3
truck
MP
Pool[2,2]
MP motorcycle
F(3,64) Pool[2,2] F(3,128)
Padding[1,1] Stride[2,2] Padding[1,1]
as the stride size, “FC” as the fully connected, “SM” as the Softmax
(adapted from [49])
are different in these images. We will show that CNN can
extract the hidden features based on these issues. In fact,
CNN does three tasks: feature extraction by convolution
layer, feature selection by pooling layer and classification
by fully connected layer. However, in some papers, CNN just
has used for feature extraction and feature selection [35, 36].
Similarly, in the first step of our method, we extract the fea-
tures using CNN. These features are mainly related to image
texture and image colors. For each image, CNN extracts a
vector with size of 1 × n such that n is the number of classes.
For example, for detecting the type of involved vehicles in
the accident, the extracted feature vector for any image is a
1 × 3 vector with respect to three classes of vehicles. This
vector is considered as the input of the classification algo-
rithm. In fact, we have transferred any image to the features
that are extracted by the last pooling layer. The idea of this
part is similar to transfer learning approach that follows a
different feature space for knowledge transfer; but we do not
have transfer of learning between the different domains of
interest, see, e.g., [37] to understand the differences between
our approach and transfer learning in details.
Furthermore, the CNN baseline classifies the samples
using an MLP in the last layer. Our approach is replacing this
MLP with a more rapid classifier. Thus, the extracted fea-
tures of CNN are sent to a new mixture of ELMs. Note that,
a linear system should be solved to adapt synaptic weights in
the last layer of ELM. By utilizing three approaches includ-
ing direct method, decomposition method and iterative
method, these weights can be determined. Thus, the weight
adaptation process of ELM can be implemented similar to
MLP, iteratively. Since passing the error happens in MLP
not in ELM, in many experiments, the results of the trained
MLP are superior to those of ELM while the speed of ELM
is still better. Thus, when the processing time is more impor-
tant, the usage of mixture of ELMs is defensible.
13
 Journal of Real-Time Image Processing

Table 2  Accident detection with Feature extractor Classifier Precision (%) Recall (%) F-measure (%) Accuracy (%)
hybrid of CNN and a different
classifier CNN OSELM 98.97 98.97 98.97 98.97
ELM 98.91 98.91 98.91 98.91
CELM 98.02 98.02 98.02 98.03
KELM 98.57 98.56 98.56 98.57
MLP 98.44 98.44 98.44 98.44
Stacking 94.67 94.65 94.7 94.7
XGBoost 94.33 94.33 94 94.76
SVM 94.33 94.33 94 94.67
RBF 94.33 94.33 94 94.8
MELM 99.31 99.31 99.31 99.31
Baseline CNN 90.37 90.37 90.33 90.84

Bold values are the best obtained results

Table 3  Severity classification Feature extractor Classifier Precision (%) Recall (%) F-measure (%) Accuracy (%)
with hybrid of CNN and
different classifiers CNN OSELM 84.66 71.66 74.66 91.22
ELM 85.49 72.94 76.08 91
CELM 60.23 62.54 61.35 90.15
KELM 93.47 63.38 63.10 90.15
MLP 93.14 90.10 91.59 93.04
Stacking 84.52 80.7 68.8 72.3
XGBoost 76 69.33 71.66 84.75
SVM 88.66 57.6 56.9 83.7
RBF 84 63.33 66.66 84.24
MELM 90.27 69.58 72.84 91.5
Baseline CNN 67.54 48.93 49.66 68.40

Bold values are the best obtained results

Table 4  Involved vehicles classification with hybrid of CNN and different classifiers

Feature extractor Classifier Precision (%) Recall (%) F-measure (%) Accuracy (%) Training time Accuracy/
(s) training
time

CNN OSELM 92.16 92.22 92.13 91.28 0.0469 1942.86

ELM 92.33 92.22 92.2 92.03 0.38 242.18
CELM 67.49 68.72 67.79 69.03 0.2 345.15
KELM 92.05 91.84 91.84 92.29 0.3127 295.14
MLP 94.28 94.94 94.6 94.8 12 7.87
Stacking 89.33 89.43 89.33 89.4 3830 0.02
XGBoost 88 88.33 88 87.02 2.6 33.47
SVM 88.66 89 88.66 86.83 0.42 206.74
RBF 89.4 89.7 90.1 89.30 7.96 10.94
MELM 92.73 92.6 92.6 92.66 1.65 56.16
Baseline CNN 75.1 74.19 74.21 73.8 20 3.69

Bold values are the best obtained results

3.3 Mixture of experts (ME) decisions are aggregated to obtain better generalization

ability in comparison with the baseline models. Mixture
An ensemble of neural networks (NNs) combines multi- of experts (ME) is an ensemble learning approach con-
ple NNs to learn the complex samples effectively. Their sisting of several experts and a gating network [38]. To

13
Journal of Real-Time Image Processing
Fig. 4  The architecture of CNN–MELM for accident analysis
understand why ME is necessary, the results of Tables 2,
3 and 4 can be considered. As one can see, the perfor-
mances of the different neural networks for a single sub-
problem of accident analyses are very different. However,
after training phase, we can recognize the best classifier.
Thus, we cannot select the best classifier for a general
task in common situation and we need an ME to combine
the capability of the different neural networks to classify
general samples with more accuracy.
In this paper, we develop an ME including the state-of-
the-art variations of ELM. They are ELM baseline [39],
KELM [40], OSELM [41] and CELM [42]. As one can
know, ELM is a kind of machine learning algorithm that
uses the topology of feedforward neural networks to solve
classification, regression, clustering, sparse approximation,
compression, and feature learning. This network includes a
single layer of hidden nodes, where the parameters of hid-
den nodes (not just the weights connecting inputs to hidden
nodes) need not be tuned. In most cases, the output weights
of hidden nodes of ELM are determined by solving a linear
system in a single step. Thus, ELM algorithm can overcome
the problems such as local minimum, obtaining learning
parameters, and over-fitting. These problems can widely
happen in the traditional gradient-based learning algorithms
for training of the shallow networks such as MLP [43].
Now, to present some details about ELM variations, we
review some methods. In KELM, a kernel function such as
Radial Base Function (RBF) is used to increase the classifi-
cation accuracy [40]. In CELM, the weight vectors from the
input layer to the hidden layer are constrained by drawing
the closed set of difference vectors between class samples.
They are the set of vectors connecting samples of one class
to samples of a different class [42]. The OSELM consists of
two main phases; a boosting phase to train the SLFNs using
the primitive ELM method with some batches of training
data for initialization stage. These boosting training data
will be discarded as soon as boosting phase is completed.
After boosting phase, OSELM will learn the training data
one-by-one or chunk-by-chunk and all training data will be
discarded once the learning procedure completes [41].
We will show that the hybrid system, which uses CNN for
feature selection and mixture of ELMs (MELM) for classifi-
cation, entitled as CNN–MELM is very efficient for complex
images’ analysis. The final architecture of CNN–MELM is
presented in Fig. 4. In our experiments, OSELM on the acci-
dent detection data set, KELM for determining the type of
involved vehicles in accidents, and OSELM for determining
the severity of the accidents provide the best results. Thus
the usage of different ELM algorithms for classification is
approved. Although in our implementation, MELM consists
of ELM, KELM, CELM, and OSELM, it can be extended by
any new version of rapid classifier.
Finally, in CNN–MELM, after classifying the extracted
features by different ELMs, a gating network is needed to
aggregate the results of the used classifiers. We examine
three gating networks including a plurality voting method
(majority voting method), a Behaviour Knowledge Space
(BKS) [44] and decision templates [45]. In plurality vot-
ing algorithm, the total votes received by each class are
obtained, and then the class with the highest number of votes
is selected as the result. An extensive and excellent analy-
sis of this voting approach can be found in [46]. BKS uses
a lookup table that lists the most common correct classes
for every possible class combination. Decision templates
compute a similarity measure between the current decision
profile of the unknown instance and the average decision
profiles of instances from each class. By experimental study,
we show that they are almost similar. We have used the first
one in our proposed system. Based on the architecture of
Fig. 4, the main steps of CNN–MELM can be stated as the
following:
13

• Extracting features from the accident images by CNN,
• Detecting the accident by MELM,
• Classifying the severity and the type of involved vehicles
in accident by two separated MELMs.
In the next section, the results of this system on different
benchmarks are presented.
4 Experiment result
To test the performance of CNN–MELM, this system is
applied on several standard data sets, including, MNIST
[47], air pollution [48], brain tumor data set [49], vehicle
image data set [50] and a new accident images’ analysis data
set [51]. The details of benchmarks in the examinations are
the following:
a. MNIST data set This data set (Modified National Insti-
tute of Standards and Technology database) [47] is a
large data set of handwritten digits that is commonly
used for training various image processing systems. This
data set is also widely used for training and testing in the
field of machine learning. The MNIST database contains
60,000 training images and 10,000 testing images. In
[22], the error rates on this data set have been reported
below 1% when they used shallow neural networks.
They trained such networks using ELM approach, which
also enables a very rapid training time.
b. Air pollution data set This data set has been introduced
by Vahdatpour et al. in 2018 [48]. This data set contains
four classes and each class contains about 100 images.
It was prepared on different days by imaging from the
sky of Tehran city.
c. Brain tumor images data set This data set containing
3064 images from 233 patients with 3 kinds of brain
tumors: meningioma (708 slices), glioma (1426 slices),
and pituitary tumor (930 slices).
d. Vehicle image data set This data set includes 7325
images from vehicles. The database consists of 3425
images of vehicle rears taken from different points of
view, and 3900 images extracted from road sequences
not containing vehicles [50].
e. Accident images analysis data set For special purpose
of accident analysis, we collected a new data set. The
images have been collected by UAV and some resources
of Google images from different scenes at different times
and locations. The size of these images was transformed
to 28×28 pixels. The resolution of these images was
also transformed to 96 dpi by a MATLAB program.
The images are labelled by human experts. This data
set includes three parts:
13
Journal of Real-Time Image Processing
• Accident-detection
• Vehicles-in-accidents
• Accident-severity
The aim of “Accident-detection” data set is to detect the
occurrence of accidents, by image processing. It includes
two subfolders with labels “without-accident” and “with-
accident” consisting of 2500 and 2398 images, respectively.
The goal of “Vehicles-in-accidents” data set is to clas-
sify the vehicles involved in an accident, by image process-
ing. It includes three subfolders with labels “light vehicle”,
“heavy vehicle” and “motorcycle” including 892, 876, and
868 images, respectively.
The purpose of “Accident-severity” data set is to classify
the severity of accidents, by image processing. It includes
three subfolders with labels “low dangerous”, “medium dan-
gerous” and “high dangerous” consisting of 118, 1603, and
1225 images. These three data sets are accessible in [51].
Considering these five data sets, in this section, we first
show the quality of features extracted by traditional methods.
Then, we obtain the best configuration of CNN for feature
extraction and show the results of accident analysis.
4.1 Comparison between the different classifiers
In this subsection, we analyze the performance of the differ-
ent classifiers that work on the extracted features by CNN.
We present the results on accident images analysis data set
[51]. We pursue the following experimental studies:
1. For networks except for ELM, we examine the following
cases:
• SVM is considered as the linear support vector
machine and the polynomial function is used as its
kernel.
• We consider radial basis function neural network
(RBF) that is trained in a fully supervised manner
by BFGS minimizing method.1 For this network,
the different batch sizes and ridge parameters have
been evaluated in Table 5 and the minimum and the
maximum accuracies for Accident images’ analysis
data set are obtained. Finally, the best parameters are
determined in the last column of this table. In Fig. 5,
the accuracy of RBF for different batch sizes is illus-
trated. As one can see, for the fixed ridge = 20, the
batch size = 150 is the best option. Similar analysis
has been done for XGBoost in Table 5 to determine
the best batch size and seed number.
1
 Broyden–Fletcher–Goldfarb–Shanno algorithm (BFGS) is one of
the best quasi-Newton methods that has been proposed for uncon-
straint nonlinear programming.
Journal of Real-Time Image Processing
Table 5  Parameters’ analysis for different classification algorithms that are used on the extracted features from accident images analysis data set
Classifier Parameters
RBF Batch size ∈ {50,100,150,200}, ridge ∈
{0.1,0.01,0.001}
XGBoost Batch size ∈ {50,100,150}, seed ∈
{0.5,1,1.5}
MLP Num-neurons ∈ {15,20,25}, learning-
rate ∈ {0.1,0.2,0.3,0.4}
ELM Num-neurons ∈{10,20,30}
KELM Num-neurons ∈{160 ,180,200,220}
OSELM Num-neurons∈{160,180,200,220}
Initial-training-data ∈{260,280,300,320}
Size-data-block ∈ {5,10,20,30}
CELM Num-neurons ∈{150,200,250}
Fig. 5  RBF accuracy with respect to the different batch sizes
Fig. 6  MLP accuracy with respect to the different learning rates
Range of accuracy Best configuration
Accuracy ∈ [86.9,89.30] Batch size = 150, ridge = 0.01
Accuracy ∈[87.02,88.61] Batch size = 100, seed = 1
Accuracy ∈ [94.04,94.8] Num-neurons = 20, learning-
rate = 0.2
Accuracy = 92.03 Num-neurons = 20
Accuracy ∈[92.29,92.54] Num-neurons = 180
Accuracy ∈ [89.38, 91.28] Num-neurons = 180
Initial-training-data = 300
Size-data-block = 10
Accuracy ∈ [48.29, 69.03] Num-neurons = 200
• Stacking applies SVM, RBF, and XGBoost as sub-
classifiers and SVM is used as meta-classifier.
• In MLP, the number of hidden layers is 3. The num-
ber of nodes in the hidden layer and the learning rate
were evaluated in Table 5. As one can see in the last
column of Table 5, the number of hidden neurons
equalling 20 and the learning rate equalling 0.2 pro-
vide the best results for MLP with sigmoid function.
In Fig. 6 the trend of MLP accuracy with respect to
the different learning rate are presented where the
best learning rate is recognized.
• For ELM-based networks, we studied the following
cases:
• We consider KELM with RBF function as the ker-
nel with different numbers of neurons in the hidden
layer, see Table 5.
• For OSELM, the number of hidden nodes changes
when the activation function is sigmoid function.
Fig. 7  OSELM accuracy with respect to the size of data block
13
 Journal of Real-Time Image Processing

Table 6  Comparison between accuracies of different hybrids of feature selectors and classifiers on accident images’ analysis data set [62]
Pre-processing on the features Classifier (%)
Feature extractor Feature selector Stacking XGBoost MLP RBF SVM ELM OSELM CELM KELM FC*

LBP + HOG +  PCA [45] 57.49 53.88 52.02 52.49 53.28 36.19 38.68 49.61 48.62 −
SURF+MSER CSF [46] 55.73 55.20 57.6 52.57 54.65 57.8 55.91 33.3 58.65 –
BBF [47] 47.20 47.76 48.59 46 47 56.65 56.34 55.64 52.76 –
CNN Max-pooling 89.4 87.02 94.8 89.30 86.83 92.03 91.28 69.03 92.29 –
layer
Baseline CNN – – – – – – – – – 73.8

Bold values are the best obtained results

*FC fully connected layer of baseline CNN

Table 7  Finding the best configuration for CNN for accident images’ analysis data set [62]
Con- C M C M C M C FC Acc.
figuration (%)
index

1 C (fi (2,16), M (Pol (2,2), C (fi (2,32), M (Pol (2,2), C (fi (2,64), M (Pol (2,2), C (fi (2,128), F(O(2)) 63.46
P(3,3)) Stir(2,2)) P(3,3)) Stir(2,2)) P(3,3)) Stir(2,2)) P(3,3))
2 C (fi (3,16), M (Pol (2,2)), C (fi (3,32), M (Pol (2,2), C (fi (3,64), – – F(O(2)) 70.16
P(1,1)) Stir(2,2)) P(1,1)) Stir(2,2)) P(1,1))
3 C (fi (3,16), M (Pol (2,2)), C (fi (3,32), M (Pol (2,2), C (fi (3,64), M (Pol (2,2), C (fi (3,128), F(O(2)) 73.8
P(1,1)) Stir(2,2)) P(1,1)) Stir(2,2)) P(1,1)) Stir(2,2)) P(1,1))

*C Convolution layer, fi: filter, P padding, M max-pooling layer, Pol poll-size, Stir stride, FC fully connected layer, O output, ACC​accuracy

Table 5 shows the results of OSELM when the num-
ber of initial training data and the size of data block
in each step are varying. The best values are given in
the last column of Table 5. In Fig. 7, the behaviour
of OSELM is presented when the size of data block
changes.
• In CELM, the number of hidden nodes is also ana-
lyzed in Table  5 when the activation function is
sigmoid function. In KELM, the number of hidden
nodes is also analyzed in Table 5 when the kernel
function is RBF.
As a conclusion for this subsection, the range of accuracy
of MLP, ELM, and KELM for accident analysis is better
than the other classifiers when the features are extracted by
the last pooling layer of CNN. We discuss on this topic in
the next subsections.
data as training and testing data, respectively. In addition, we
used fivefold cross validation. Table 6 shows the accuracy
of different combinations of feature selectors and classifiers.
Bold values show the best obtained results throughout the
paper. As one can see on the traditional feature extractors,
PCA as the feature selector and stacking as the classifier
provides the best accuracy 57.49%. While, CSF as the clas-
sifier and KELM as the classifier produce 58.65% accuracy.
BBF as the feature selector and ELM classifier give the best
results while its accuracy is less than the previous combina-
tions. However, the accuracies of all of these experiments
are worse than baseline CNN with 73.8% accuracy. It is
worthwhile to note that hybrid of CNN features and KELM
classifier provides 92.29% accuracy, which is the best result
among all of the methods.
4.3 Best configuration for CNN

To find the best configuration of CNN, we considered the

4.2 Quality of features
To test the performance of the feature extractor and feature
selector methods, we applied these methods on the acci-
dent images’ analysis data set [51] to detect involved vehicle
types in the accident. We considered 80% and 20% of the
accident images’ analysis data set [51]. In Table 7, three
different configurations of baseline CNN were compared.
Trivially, the third configuration provides the best accuracy.
Thus, this configuration is used for all of the experiments
in our study. However, the best accuracy of CNN is 73.8%.
Since baseline CNN applies MLP for classification in the

13
Journal of Real-Time Image Processing

Table 8  The learning Training algorithm for MLP Cross entropy Accuracy (%) Training time Accuracy/
algorithms for MLP as classifier (seconds) training
to classify the features extracted time
by the pooling layer of CNN
Scale-conjugate-gradient 0.062 94.42 25 3.78
Levenberg–Marquardt 0.029 94.8 27 3.51
BFGS quasi-newton 0.062 94.49 20 4.72
Resilient backpropagation 0.065 94.57 18 5.25
Conjugate gradient with Powell restart 0.064 94.38 18 5.24
Fletcher–Powell conjugate gradient 0.063 94.42 15 6.29
Polka–Ribiere conjugate gradient 0.062 94.49 15 6.30
One step secant 0.063 94.49 12 7.87
Gradient-descent with momentum 0.064 94.42 17 5.55

Bold values are the best obtained results

Table 9  Comparison between different gating networks for CNN– implementation. We checked different MLP with different
MELM on accident images analysis data set [62] hidden layers and different number of neurons and finally
Feature extractor Classifier Gating network Accuracy we used an MLP with 3 hidden layers and 20 neurons in
(%) each hidden layer for our experiment. Then, the accuracy
of different learning algorithms was compared. As one can
CNN MELM Plurality voting 92.66
see in Table 8, the best cross entropy has been determined
BSK 92.52
for Levenberg–Marquardt algorithm. Also, the accuracies
Decision templates 92
of the different learning algorithms are between 94.38 and
94.8. Again, Levenberg–Marquardt gets the best accuracy.
last layer, in what follows, we focus on its MLP configura- Since the processing time is important for real-time
tion to improve the results. applications, we used a new measure by determining the
rate of accuracy to the processing time. Because, we need
to maximize the accuracy and minimize the processing
4.4 Best configuration for MLP time, the maximal rate shows the best learning algorithm.
As one can see in the last column of Table 8, “one step
In this subsection, the accident images’ analysis data set [51] secant” algorithm gets the best score. In what follows, we
is used to find the best configuration for MLP. For this aim, consider this algorithm for MLP training process.
the features of the last pooling layer of CNN are considered
as the inputs for MLP. We have used MATLAB software for

Table 10  Comparison between Feature selector Classifier MNIST Air pollution Brain tumor data set Vehicle
the percent of accuracy of the image data
hybrids of CNN with different set
classifiers
CNN KELM 99.62 66.92 93.68 98.91
ELM 99.75 63.91 88.51 98.84
CELM 99.51 63.75 69.83 82.05
OSELM 99.60 63.75 90.2 98.77
MLP 99.76 62.11 88.80 99.7
Stacking 99.73 59.42 86.91 99.76
XGBoost 99.71 62.73 87.33 99.64
SVM 99.8 58.59 87.51 99.53
RBF 77.50 60.24 86.84 99.75
MELM 99.70 68.42 92.81 97.66
Baseline CNN 99.35 61.41 82.28 98.84
Best reported result 99.79 [67] 59.38 [64] 91.28 [65] 92.78

Bold values are the best obtained results

13

Fig. 8  Some samples of accident images (raw data)
4.5 Best gating network
In this experiment, we compared three approaches for gat-
ing network. We determine the performance of the proposed
MELM with respect to these networks. As one can see in
Table 9, there are not significant differences between the
presented gating networks. However, the plurality voting
method is the best option for gating network for classifying
the accident images.
4.6 Effectiveness of CNN–MELM
Now the features extracted by CNN are directly passed
to MELM. The results of CNN–MELM on four stand-
ard benchmarks are compared with various classifiers in
Table 10. This table shows that hybrid of CNN with SVM
has the best result on the MNIST data set, but CNN–MELM
is close to this classifier. On air pollution, the best results
are obtained by CNN–MELM. On brain tumor data set,
the best classifier is CNN–KELM and then CNN–MELM.
Finally, for vehicle image data set, the best result is pro-
vided by CNN-Stacking while the difference between the
accuracies of MELM and this method is 2.1%, which shows
that CNN–MELM is still a reasonable classifier. As a final
point, the results of Table 10 show that MNIST and vehicle
image data sets are not as much as complex to distinguish
the differences of the presented classifiers while the other
mentioned data sets are better to distinguish the differences
between the classifiers. In these types of complex data sets,
CNN–MELM and CNN–KELM could classify the samples,
successfully in a low processing time.
As a conclusion, using mixture of ELM as the classifier,
we can obtain the best results from CNN–MELM. Therefore,
to detect the accidents and to distinguish the characteristics
of crashes, we can use this ensemble method on the accident
images, perfectly.
13
Journal of Real-Time Image Processing
Fig. 9  The transformed 28 × 28 pixels of accident images presented in
Fig.  8 (They are provided in accident images’ analysis data set [62]
and used as the input samples of our proposed system)
4.7 Accident analysis by CNN–MELM
In this section, we evaluate our proposed system for accident
detection. If an accident happens, we distinguish the charac-
teristics of this accident by classifying the accident severities
and involved vehicles. To evaluate the performance of the
different classifiers, we used the accident images’ analysis
data set [51]. In Figs. 8 and 9, some samples of these images
are illustrated. Since the severity of the accident is a rela-
tional concept or is defined with respect to the severity of
the injury, number of injuries, number of deaths, and num-
ber of damaged cars, it is very hard to propose a common
measure for severity evaluation based on the images. Usually
the police officers judge about severity of accidents based
on their experiences and status of vehicles and injured peo-
ple. On the other hand, intelligent sensors can collect some
details about these statuses. Since this paper focuses just
on accident images, these approaches cannot be followed
and we need to get an overall description about severity
for accident management in the initial time steps (golden
time). Thus, we invited some human experts to assign the
labels “low dangerous”, “medium dangerous” and “high
dangerous” to each image. The severity classification by
CNN–MELM is stated in Table 3.
We used MATLAB-2017b and WEKA open-software to
compare the ELM-based methods with the other classifiers.
We implemented all of the classifiers on a system with Intel
i5-2430M CPU @ 2.4 GHz processor and 4G of memory.
Journal of Real-Time Image Processing

Table 11  Running time of Running time of feature selection using CNN (s) Classifier Running time(s)
classifiers to classify the vehicle
types in [62] Train Test

20 MELM 1.65 0.95

CELM 0.20 0.07
ELM 0.38 0.03
OSELM 0.0469 About 0
KELM 0.3127 0.1451
MLP 12 4.52
Stacking 3830 1000
XGBoost 2.6 0.7
SVM 0.42 0.15
RBF 7.96 0.2

The number of the epochs in our CNN model is 20 for all
of the training iterations. In addition, the learning rate in our
model is 0.05. In Tables 2, 3 and 4, Precision, Recall, and
F-measure of CNN–MELM for accident detection, accident
severity classification, and involved vehicles’ classification
are presented, respectively. In all of these tables, the best val-
ues are shown with bold numbers. To compare the results of
CNN–MELM with that of the other algorithms, we present
the same measures for all of the classifiers. Table 2 shows
that CNN–MELM has the best results among the other algo-
rithms to detect the accidents in the images.
Table 3 reveals that the precision of CNN–KELM is the
best for accident severity classification. Its behaviour is
close to CNN–MLP. After these algorithms, CNN–MELM
has the third rank. However, the recall, the F-measure,
and the accuracy of CNN–MLP are better than the others.
The recall of CNN–ELM is the third rank, the F-measure
of CNN–OSELM is the second rank and the accuracy of
CNN–MELM has the second rank in this experiment. These
results show that based on all of the measures, the results of
the hybrid of CNN and ELM variants can be used to classify
the accident severities efficiently.
Table 4 compares the different classifiers for vehicles
that are involved in the accidents. As one can see, the dif-
ferent measures of CNN–MLP are superior compared with
the other methods. However, the differences between these
measures from the corresponding measures of CNN–MELM
are 1.55% for precision, 2.34% for recall, 2% for F-measure
and 1.83% for accuracy. Thus, one can say that it is better to
use CNN–MLP for involved-vehicle classification.
However, the training of MLP is very time-consuming.
Since both of the accuracy and training time are important
for real-time applications, we define the rate of accuracy to
processing time similar to Subsection 4.4. In the last column
of Table 4, this measure is evaluated for all of the classifiers.
As one can see, the hybrid of CNN and OSELM, CELM,
and KELM are the best algorithms for this examination.
Additionally, MELM is better than MLP at least 7 times.
Thus, we cannot propose CNN–MLP for accident analyzing
in real-time, while the hybrid of CNN and the variants of
ELM is still defensible.
Now, we compare our classifier with one of the best
CNN-based works to recognize vehicle types [52]. In the
latter reference, the average of precision to detect the differ-
ent vehicle types was between 66.36 and 90.65% and their
total average was 81.05%. This is worse than the result of our
proposed MELM with averaged precision 92.73%.
Besides, in Table 4, it is shown that the processing times
of ELM, CELM, OSELM, and KELM to detect the vehi-
cle type are 0.38, 0.2, 0.0469, and 0.3127s, respectively,
while the processing times of MLP, Stacking, XGBoost and
RBF are 12, 3830, 2.6 and 7.96 s, which are worse than
ELM-based classifiers. Just SVM classifies this data set
with 0.72 s while its precision is 88.66%, which is worse
than ELM, KELM, and OSELM as presented in Table 4.
Also, the processing times of KELM and ELM are not as
good as the same time for OSELM. This shows that they
can be neglected from MELM for real-time implementa-
tion. However, as the last column of Table 4 shows, the rate
of accuracy of KELM to its training time has the third rank
between the mentioned classifiers which shows its capability
for classification of accident images.
Table 11 also shows the running time of CNN–MELM
for involved-vehicle classification in [51]. The running time
of CNN for feature extraction is 20 s but this time is for
a data set with 2398 images. Therefore, the average run-
ning time for feature extraction for a single image is 0.008 s.
After feature extraction, the next step is classification. The
running time for training and testing on this data set is
shown in Table 11. The average of running time for testing
a single image is 0.0003 s. This shows that after training,
CNN–MELM consumes 0.0083 s for analyzing the type of
involved vehicles in any accident image and so this classifi-
cation can be done in real-time.

13

5 Conclusion
Understanding the characteristics of an accident provides a
great help for accident management centers. To recognize
these characteristics, the images are good sources of infor-
mation. When these images are collected by various sources
such as UAVs, surveillance cameras, and residences and are
sent to a processing center, we need to process them in real-
time to call a supporting procedure for accident manage-
ment. This paper tried to detect the accidents, to classify
the severity of accident and finally to recognize the type of
involved vehicles in real-time.
For this aim, we proposed a hybrid system consisting
of a CNN for feature extraction and a mixture of ELM
(MELM) for classification purposes. To evaluate this sys-
tem, first, traditional feature extraction methods such as
LBP, HOG, SURF, and MSER were applied. We compared
the features of these methods with the features of pooling
layer of CNN and showed that the results of CNN were
preferable. Then, to select the best classifier, we compared
CNN–MELM with some shallow networks and baseline
CNN on MNIST, air pollution, brain tumor data set, vehi-
cle image data set and accident images’ analysis data set
[51]. The results showed that the accuracy, the precision,
the recall and the F-measure of CNN–MELM were accept-
able while the processing times were close to real-time.
Really, by CNN–MELM on accident images [62], we
found out 99.31% precision for accident detection, 90.27%
precision for severity classification and 92.73% precision
for vehicle classification. By considering accident images’
analysis data set [62], we also showed that the accident
characteristics could be recognized in real-time.
Finally note that, our ensemble mixture of expert
method consists of four learning algorithms containing
ELM, KELM, CELM, and OSELM. We used KELM
because of its perfect accuracy. But as the experiments
showed, KELM is almost time-consuming compared
with the other variants of ELM or some other learning
algorithms such as SVM and XGBoost. However, their
differences can be neglected. By the way, for real-time
applications, we can remove KELM from the MELM and
the experimental results showed that the accuracy remains
almost fixed. The reason is that the difference between the
accuracies of KELM and OSELM is not significant. Since,
in the proposed ensemble method, we want to provide an
opportunity to receive a rapid solution associate with accu-
rate solution, we have not neglected from KELM. Trivially
if the real-time computation gets the most priority in an
application, the user can ignore from KELM from mixture
of ELM.
For the future works, we try to combine the power
of ELM for feature selection inside CNN model. Also,
13
Journal of Real-Time Image Processing
optimization of the weights of ELM can lead us to find
the accuracy similar to MLP in less computation time.
Besides, with respect to the technological viewpoint, since
the images can be sent to our proposed system by different
sources such as CCTV cameras or UAV images, we need a
robust algorithm for data fusion before feature extraction
and accident classification. For fusion purposes, maybe
some kinds of neural networks or some layers of deep net-
works can be utilized. In addition, the readers can define
an ensemble of CNN and the traditional feature extraction
methods such as LBP or HOG for feature extraction. In
such approaches, the ensemble method will extract the
features instead of classification and again the different
classifiers can be applied. In addition, the theoretical anal-
ysis should be stated for combination of CNN and ELM
variations in a single system.
References
1. Bisht, N., Siddhi, P., Kashyap, H.: Monitoring road accidents
using sensors and providing medical facilities. Treatise Electr
Magn 2, 68–73 (2012)
2. Hoose, N., Vicencio, M., Zhang, X.: Incident detection in urban
roads using computer image processing. Traffic Eng Control
33(4), 236–244 (1992)
3. Zifeng, J.: Macro and micro freeway automatic incident detec-
tion (aid) methods based on image processing. In: Intelligent
Transportation System, ITSC’97 (1997)
4. Coifman, B., McCord, M., Mishalani, R., Iswalt, M., Ji, Y.:
Roadway traffic monitoring from an unmanned aerial vehicle.
In: IEE Proceedings-Intelligent Transport Systems (2006)
5. Cao, X., Lan, J., Yan, P., Li, X.: Vehicle detection and tracking
in airborne videos by multi-motion layer analysis. Mach. Vis.
Appl. 23(5), 921–935 (2012)
6. Kim, N., Chervonenkis, M.: Situation control of unmanned
aerial vehicles for road traffic monitoring. Modern Appl. Sci.
9(5), 1 (2015)
7. Srinivasan, D., Jin, X., Cheu, R.: Evaluation of adaptive neural
network models for freeway incident detection. IEEE Trans.
Intell. Transp. Syst. 5(1), 1–11 (2004)
8. Chiou, Y.-C., Fu, C.: Modeling crash frequency and severity
using multinomial-generalized poisson model with error com-
ponents. Accid. Anal. Prev. 50, 73–82 (2013)
9. Anderson, J., Govada, M., Steffen, T., Thorne, C., Varvarigou,
V., Kales, S., Burks, S.: Obesity is associated with the future
risk of heavy truck crashes among newly recruited commercial
drivers. Accid. Anal. Prev. 49, 378–384 (2012)
10. Ahonen, T., Hadid, A., Pietikainen M.: Face description with
local binary patterns: application to face recognition. IEEE
Trans. Pattern Anal. Mach. Intell. 28(12), 2037–2041 (2006)
11. Wang, X., Han, T., Yan, S.: An hog-Lbp human detector with
partial occlusion handling. In: International Conference on
Computer Vision (2009)
12. Chen, H., Tsai, S., Schroth, G., Chen, D., Grzeszczuk, R., Girod,
B.: Robust text detection in natural images with edge-enhanced
maximally stable extremal regions. In: 18th IEEE International
Conference on Image Processing (ICIP) (2011)
13. Bay, H., Ess, A., Tuytelaars, T., Van Gool, L.: Speeded-up
robust features (surf). Comput Vis. Image Underst. 110(3),
346–359 (2008)
Journal of Real-Time Image Processing
14. Krizhevsky, A., Sutskever, I., Hinton, G.: Imagenet classifica-
tion with deep convolutional neural networks. Adv. Neural Inf.
Process. Syst. 25, 1097–1105 (2012)
15. Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only
look once: unified, real-time object detection. In: Conference
on computer vision and pattern recognition, pp. 779–788 (2016)
16. Ahn, B.: Real-time video object recognition using convolutional
neural network. In: Neural Networks (IJCNN). pp. 1–7 (2015)
17. LeCun, Y., Kavukcuoglu, K., Farabet, C.: Convolutional net-
works and applications in vision. In: International Symposium
on Circuits and Systems (ISCAS) (2010)
18. Lee, K., Park, D.C.: Image classification using fast learning
convolutional neural networks. Adv. Sci. Technol. Lett. 113,
50–55 (2015)
19. Sadeky, S., Al-Hamadiy, A., Michaelisy, B., Sayed, U.: “Real-
time automatic traffic accident recognition using Hfg. In: 20th
International Conference on Pattern Recognition (ICPR) (2010)
20. Nejjari, F., Benhlima, L., Bah S.: Event traffic detection using
heterogenous wireless sensors network. In: 13th International
Conference of Computer Systems and Applications (AICCSA)
(2016)
21. Kahaki, S., Nordin, M.: Highway traffic incident detection using
high-resolution aerial remote sensing imagery. J. Comput. Sci.
7(6), 949 (2011)
22. Jiansheng, F.: Vision-based real-time traffic accident detection.
In: 11th World Congress on Intelligent Control and Automation,
WCICA (2014)
23. Chen, L., Cao, Y., Ji, R.: Automatic incident detection algorithm
based on support vector machine. In: Sixth International Con-
ference on Natural Computation (ICNC) (2010)
24. Prabha, C., Sunitha, R., Anitha, R.: Automatic vehicle accident
detection and messaging system using GSM and GPS modem.
Int. J. Adv. Res Electr. Electron. Instrum. Eng. 3(7), 10723–
10727 (2014)
25. Kagesawa, M., Nakamura, A., Ikeuchi, K., Saito H.: Vehicle
type classification in infra-red image using parallel vision board.
ITSWC (2000)
26. Zhou, Y., Nejati, H., Do, T.-T., Cheung, N.-M., Cheah, L.: Image-
based vehicle analysis using deep neural network: a systematic
study. In: International Conference on Digital Signal Processing
(DSP) (2016)
27. Chen, Z., Ellis, T.: Semi-automatic annotation samples for vehicle
type classification in urban environments. IET Intel. Transp. Syst.
3(9), 240–249 (2014)
28. Wang, X., Zhang, W., Wu, X., Xiao, L., Qian, Y., Fang, Z.:
Real-time vehicle type classification with deep convolutional
neural networks. J. Real-Time Image Proc. (2017). https​://doi.
org/10.1007/s1155​4-017-0712-5
29. Zheng, Z., Lu, P., Lantz, B.: Commercial truck crash injury sever-
ity analysis using gradient boosting data mining model. J. Saf.
Res. 65, 115–124 (2018)
30. Delen, D., Sharda, R., Bessonov, M.: Identifying significant pre-
dictors of injury severity in traffic accidents using a series of arti-
ficial neural networks. Accid. Anal. Prev. 38(3), 434–444 (2006)
31. Chang, L.-Y., Wang, H.-W.: Analysis of traffic injury severity:
an application of non-parametric classification tree techniques.
Accid. Anal. Prev. 38(5), 1019–1027 (2006)
32. Nguyen, C.H., Cai, Chen, F.: Automatic classification of traffic
incident’s severity using machine learning approaches. IET Intel.
Transp. Syst. 11, 615–623 (2017)
33. Kheradpisheh, S., Sharifizadeh, F., Nowzari-Dalini, A., Gan-
jtabesh, M., Ebrahimpour, R.: Mixture of feature specified experts.
Inf. Fusion 20, 242–251 (2014)
34. Li, L., Zou, B., Hu, Q., Wu, X., Yu, D.: Dynamic classifier ensem-
ble using classification confidence. Neurocomputing 99, 581–591
(2013)
35. Yu, J.S., Chen, J., Xiang, Z.Q., Zou, Y.X.: A hybrid convolutional
neural networks with extreme learning machine for WCE image
classification. In: IEEE International Conference on Robotics and
Biomimetics (ROBIO) (2015)
36. McDonnell, M.D., Tissera, M.D., Vladusich, T., Van Schaik,
A., Tapson, J.: Fast, simple and accurate handwritten digit clas-
sification by training shallow neural network classifiers with the
‘extreme learning machine’algorithm. PLoS One 10(8), e0134254
(2015)
37. Liu, Y., Yao, X.: Ensemble learning via negative correlation. Neu-
ral Netw. 12(10), 1399–1404 (1999)
38. Masoudnia, S., Ebrahimpour, R., Arani, S.: Incorporation of a
regularization term to control negative correlation in mixture of
experts. Neural Process. Lett. 36(1), 31–47 (2012)
39. Islam, M., Yao, X., Nirjon, S., Islam, M., Murase, K.: Bagging
and boosting negatively correlated neural networks. IEEE Trans.
Syst. Man Cybern. Part B (Cybern.) 38(3), 771–784 (2008)
40. Liu, Y., Yao, X.: Simultaneous training of negatively correlated
neural networks in an ensemble. IEEE Trans.Syst. Man Cybern.
Part B (Cybern.) 29(6), 716–725 (1999)
41. Ebrahimpour, R., Sadeghnejad, N., Masoudnia, S., Arani, S.:
Boosted pre-loaded mixture of experts for low-resolution face
recognition. Int. J. Hybrid Intell. Syst. 9(3), 145–158 (2012)
42. Lotfi, M., Motamedi, S., Sharifian, S.: Time-based feedback-
control framework for real-time video surveillance systems with
utilization control. J. Real-Time Image Proc. (2016). https​://doi.
org/10.1007/s1155​4-016-0637-4
43. Zarándy, Á, Nemeth, M., Nagy, Z., Kiss, A., Santha, L., Zsedro-
vits: A real-time multi-camera vision system for UAV collision
warning and navigation. J. Real-Time Image Proc. 4, 709–724
(2016)
44. Puri, A.: A survey of unmanned aerial vehicles (UAV) for traffic
surveillance. Department of computer science and engineering,
University of South Florida (2005)
45. Pearson, K.: Liii. On lines and planes of closest fit to systems of
points in space. Lond. Edinb. Dublin Philos. Mag. J. Sci. 2(11),
559–572 (1901)
46. Hall, M.: Correlation-based feature selection for machine learn-
ing. PhD thesis, Department of Computer Science, University
of Waikato Hamilton (1999)
47. Hamon, J.: Optimisation Combinatoire Pour La Sélection De
Variables En Régression En Grande Dimension: Application
En Génétique Animale. Université des Sciences et Technologie
de Lille-Lille I. (2013)
48. Mofarreh-Bonab, M., Mofarreh-Bonab, M.: Color image com-
pression using PCA. Int. J. Comput. Appl. 111(5):16–19 (2015)
49. Zhang, L., Yang, F., Zhang, Y.D., Zhu, Y.J.: Road crack detec-
tion using deep convolutional neural network. In: International
Conference on Image Processing (ICIP), pp. 3708–3712 (2016)
50. Weng, Q., Mao, Z., Lin, J., Liao, X.: Land-use scene classi-
fication based on a cnn using a constrained extreme learning
machine. Int. J. Remote Sens. pp. 1–19 (2018)
51. Martinel, N., Piciarelli, C., Foresti, G., Micheloni C.: Mobile
food recognition with an extreme deep tree. In: Proceedings of
the 10th International Conference on Distributed Smart Camera
(2016)
52. Pan, S.J., Yang, Q.: A survey on transfer learning. IEEE Trans.
Knowl. Data Eng. 22(10), 1345–1359 (2010)
53. Abbasi, E., Shiri, M., Ghatee, M.: A regularized root–quartic
mixture of experts for complex classification problems. Knowl.-
Based Syst. 110, 98–109 (2016)
54. Huang, G.-B., Zhu, Q.-Y., Siew, C.-K.: Extreme learning
machine: a new learning scheme of feedforward neural net-
works. In: International Joint Conference on Neural Networks
(2004)
13

55. Huang, G.-B., Zhou, H., Ding, X., Zhang, R.: Extreme learning
machine for regression and multiclass classification. IEEE Trans.
Syst. Man Cybern. Part B (Cybern.) 42(2), 513–529 (2012)
56. Huang, G.-B., Liang, N.-Y., Rong, H.-J., Saratchandran, P., Sunda-
rarajan N.: “On-line sequential extreme learning machine. Com-
put. Intell. 2005, 232–237 (2005)
57. Zhu, W., Miao, J., Qing, L.: Constrained extreme learning
machine: a novel highly discriminative random feedforward neural
network. In: International Joint Conference on Neural Networks
(IJCNN) (2014)
58. Tian, H.X., Mao, Z.Z.: An ensemble ELM based on modified
AdaBoost. RT algorithm for predicting the temperature of molten
steel in ladle furnace. IEEE Trans. Autom. Sci. Eng. 7(1), 73–80
(2010)
59. Huang, Y., Suen, C.: The behavior-knowledge space method for
combination of multiple classifiers. In: IEEE Computer Society
Conference on Computer Vision and Pattern Recognition (1993)
60. Kuncheva, L., Bezdek, J., Duin, R.: Decision templates for mul-
tiple classifier fusion: an experimental comparison. Pattern Rec-
ognit. 34(2), 299–314 (2001)
61. Kuncheva, L.I.: Combining Pattern Classifiers: Methods and
Algorithms. John Wiley & Sons (2004)
62. Pashaei, A., Ghatee, M., Sajedi, H.: Accident images analysis
dataset. Amirkabir University of Technology, 2018. (Online).
https​://githu​b.com/mghat​ee/Accid​ent-Image​s-Analy​sis-Datas​et.
Accessed 2018
63. Yann, L., Corinna, C., Christopher, J.: The Mnist Database of
Handwritten Digits (Online). http://yhann​.lecun​.com/exdb/mnist​
(1998)
64. Vahdatpour, M., Sajedi, H., Ramezani, F.: Air pollution forecast-
ing from sky images with shallow and deep classifiers. Earth Sci.
Inf. 11(3), 413–422 (2018)
65. Cheng, J., Huang, W., Cao, S., Yang, R., Yang, W., Yun, Z., Wang,
Z., Feng, Q.: Enhanced performance of brain tumor classification
via tumor region augmentation and partition. PLoS One 10(10),
0140381 (2015)
66. Arróspide, J., Salgado, L., Nieto, M.: Video analysis based vehi-
cle detection and tracking using an MCMC sampling framework.
EURASIP J. Adv. Signal Process. (2012)
13
Journal of Real-Time Image Processing
67. Wan, L., Zeiler, M., Zhang, S., Le Cun, Y., Fergus R.: Regu-
larization of neural networks using dropconnect. In: International
Conference on Machine Learning (2013)
68. Haut, J., Paoletti, M., Plaza, J., Plaza, A.: Fast dimensionality
reduction and classification of hyperspectral images with extreme
learning machines. J. Real-Time Image Proc. 15(3), 439–462
(2018)
Publisher’s Note Springer Nature remains neutral with regard to
jurisdictional claims in published maps and institutional affiliations.
Ali Pashaei  is an M.Sc. student in Department of Computer Science
of Amirkabir University of Technology, Tehran, Iran. He works on
image processing, deep learning and extreme learning algorithms. He
has written and presented two papers in two international conferences
on Intelligent Transportation Systems.
Mehdi Ghatee  is an Associate Professor with Department of Com-
puter Science, Amirkabir University of Technology, Tehran, Iran. His
major is ITS, Smartphone-based ITS Systems, Neural Network and
Fuzzy Systems. He has written more than 100 papers on national and
international journals and conferences. Previously, he was Chairman of
the Department of Computer Science and Project Manager of Iranian
Nation Plan of Intelligent Transportation Systems and currently is an
Associate Dean for Undergraduate Affairs of Faculty of Mathematics
and Computer Science. He is also a member of the Board of Directors
of the ITS-RI and director of NORC.
Hedieh Sajedi  received a B.Sc. degree in computer engineering from
Amirkabir University of Technology in 2003, and M.Sc. and Ph.D.
degrees in computer engineering (artificial intelligence) from Sharif
University of Technology, Tehran, Iran in 2006 and 2010, respectively.
She is currently an Assistant Professor at the Department of Computer
Science, Tehran University, Iran. She has written more than 80 papers
on national and international journals and conferences. Her research
interests include multimedia data hiding, steganography and steganaly-
sis methods, pattern recognition, and machine learning.