1

Behavioral Recognition of Fish Using

Accelerometer Data
Areej Habib, Department of Biomedical Engineering

that challenge by using visual aids to deduce critical

Abstract—Bio-logging and telemetry of animals in the wild helps associations. Through optimal synchronization between
collect critical data that can be used to study their behavior, track accelerometer data and camera recordings we created a labelled
daily activity and quantify energy expenditures. The goal of the dataset to train traditional classifiers on acceleration metrics
paper is to use 3-axis accelerometer recordings (> 50 Hz) of red such as range, mean and standard deviation from 2 sec long
drum fish to recognize a characteristic escape movement, triggered
during experiments as well as potentially classify other behavioral
overlapping windows to predict different behavioral states. The
patterns of interest. The time-series accelerometer data was remote nature of telemetry makes visual observation a
manually labelled using video recording. For the purpose of this challenging task, thus, creating a need for an intelligent system
study, it was analytically concluded at an early stage that out of the that relies on algorithms which when deployed to
three sensor locations only the tail sensor recordings along the microelectronics/sensors activity can be remotely monitored
lateral axis is distinctive in characterizing the escape movement. through quantitative and qualitative metrics [3]. Accelerometers
The escape motion entails a sudden caudal fin acceleration
followed by an escape turn. This limited dataset of fast-start events alone are inefficient at tracking complete 3D movements of
(or escape motion) were identified with 100% accuracy using animals because they lack rotational information, such as
Linear SVM and kNN (binary classification models). angular velocity and the direction of movement, gyroscopes
To increase complexity, we reduced any potential bias in the provide these elucidated details and are now being used in
dataset by further labeling the routine behavior into events with mobile phones and microelectronics to provide quality
turns and events without turns creating a multi-class labelled information.
dataset. The algorithms showed biased classification towards the
Studies have proven to show that activity trackers i.e.
no-turn events with SVM-gaussian kernel attaining close to 96.8%
accuracy and Linear SVM models showing close to 96.7% accelerometers can be used in understanding/detecting autistic
accuracy. The statistical features used for classification were mean, behaviors; self-injurious behaviors (SIB) [10]. Such behaviors
range and standard-deviation. are characterized by anomalies that involve characteristic
The data collection, activity tracking and behavioral behavior and by recognizing these repetitive behavioral patterns
experiments for the study were conducted on Red Drum Fish the qualitative feedback generated can be used by caregivers to
(Scianeops occellatus) at the Whitney Laboratory of Marine
diagnose early and develop therapies that reverse them [10].
Biology Sciences, University of Florida and the processing and
analysis was achieved in Matlab. We explore methods to detect the escape movement of the red
drum fish often referred to as a fast-start event which consists of
Index Terms—bio-logging, activity tracking, telemetry, a sequential C-shape contraction of body, brief, sudden
accelerometer sensor, red drum fish, time, Linear SVM, kNN, acceleration of caudal fin followed by a turn within a few
Gaussian kernel performance metric, caudal fin, standard- milliseconds to escape the threat posed. Fig 2 is a sequence of
deviation, range, mean frames from the video depicting first 150 milliseconds of the
escape behavior. From the three spatially located sensors on the
K nowledge from accelerometer tags helps quantify
behavioral states of animals in the wild which can
fish body; head, mid and tail, we identified the tail as the most
critical sensory location of the fish along the lateral axis
provide vital insight on their habitats, their interaction with
recordings through signal comparison to achieve our goal of
members of the same species as well as other species (prey-
classifying this escape motion (Fig 1.)
predator relationships) and can be further extrapolated for clinical
Overall, this work can be further optimized to develop a
applications in diagnostics [1]. Field biologists use this
technique for wireless accelerometer tags to track marine
information to undertake individual and population animal
studies, eventually making informed decisions on management of
ecosystems and animal conservation. By approximating linkages
between physiological and ecological processes in the context of
movement we can identify the survival strategies of animals when
they come across a predator or possible danger [3]. This study
focuses on identifying this behavior in red drum fish i.e. recognize
and discriminate escape movements from routine swimming.
Additionally, provide scope to establish a mathematically relevant
link between this survival behavior and parameters from activity
tracking data i.e. accelerometer data [8]. Studies have shown to
link acceleration derived parameters to heart rate, energy
expenditure and behavioral patterns such as feeding and breeding
[4].
It is challenging to analyze accelerometer data because of its Fig 1. Schematic diagram of the fish coordinate frame. The directions of
underlying non-intuitive nature [3]. This study aims to overcome the acceleration (Ax, Ay, Az) on the X-, Y- and Z-axes are shown [11].
animals in the wild with fine spatial and temporal resolution and 2
study their predatory behaviors and responses. This can also be
implemented to recognize anomalies in the ecology and provide Behavior Description Time
critical data for animal conservation. (s)
Swimming Steady motion while moving the 6412
A.Data loggerand experimental setup caudal fin
Drifting Gliding through the water without 172
Accelerometer tags MicroDaq MSR145 [12] moving the caudal fin
continuously record data at a constant frequency of 51.2 Hz for a
Resting Stationary position while the 232
total of 3.45 hours along all three axes (x = forward, y = lateral caudal fin continues to beat
and z = vertical) with maximum values of ±2G. These sensors Escape Sudden beating of the caudal fin 78
were surgically mounted on the fish before the experiment and while the fish bends its body to a
manually programmed to begin data logging 24 hours after C-shape
surgery so the fish is fully recovered from surgery. GoPro Turning Bending of mid body to take a 87
Cameras were used for 2.23 hours of the activity trials. To get a 360° turn
complete (fully labelled) dataset the accelerometer signal was
reduced to merely 2.23 h consisting of at least three distinct Table 1. Summary of the behaviors exhibited by the red drum, and the total duration, in
behavioral events with maximum duration of 3-5 seconds. seconds (6953 sec).
Prior to data processing and labelling, the three sensor datasets
were synchronized across all three channels (x, y and z) first and B. Dataset
then optimally synced with video recordings. Localization of For labelling purposes, the dataset was segmented into 3-sec long
calibration events along the sensor channels was challenging, non-overlapping windows with a sampling frequency of
spectral analysis or STFT (short-time Fourier transform) of the 51.2 Hz generating windows of 154 data points. The limited 2.23 hours
signals were employed to identify the high frequency high of video recordings led to reducing the accelerometer dataset to 2374
amplitude calibration point. On analyzing the signals, the lateral labelled events for binary classification and 2312 events for multiclass
channel of the tail sensor proved to be the most significant one for classification (remaining were discarded).
the escape event because of its characteristic lateral tail motion. In binary dataset, the events were labelled as FSS and NFSS
(Non-fast-start swimming) which consisted of all the routine swimming
events (summarized in table 1 above). For multiple classification we
segmented the data into FSS, Turns and clubbed the remaining routine
swimming movements under a single class. An important observation to
make is that although a significantly large portion of the fish behavior
comprised of resting (i.e. a highly biased dataset) the tail beating
frequencies were prominent features to cluster fast start swimming.

Fig 3. Separation of data into two clusters using two features (units: g)
where feature 1 corresponds to mean and feature 2 corresponds to range

We employed 60:20:20 segmentation for training, validation and testing

purposes and used the computed accuracies for parameter tuning of
Fig 2. Image sequences of typical escape movement with rapid tail movement models later.
 3
C. Feature engineering
Feature extraction was achieved using a window-based model;
we used axis-specific parameters only such as RMS (root mean
square), range, standard deviation and mean- their average
values are summarized in Table 1 for each event. However, for
final predictions we discarded RMS as it didn’t influence the
prediction accuracy. Moreover, we observed that for escape
movement the range was an extremely good discriminator in
predicting the event (see Fig 3.) as the data was easily separable
into two clusters.
The turns unlike escape events are not characterized with
extremely high amplitude acceleration. Thus, discriminating
them with just three features was unlikely. We extracted
autocorrelation coefficients previously proven to be good
predictors for time-series data and increased the dimensionality
of the dataset by 8 times from 3 to 24. However, PCA of all 24
dimensions showed that the 95.19% of the data’s variance can
be explained by our first three features (see Table 2b). Thus,
these coefficients were discarded and only the statistical
parameters were selected for multi-class classification.

Fig 4. First three principal components of 24-dimensional data where the lines
VARIABLES (M S-2) ESCAPE ROUTINE indicate the 24 variables
MeanAx -5.43 -6.39
SDAx 6.307 0.258
RangeAx 40.289 1.218
RMSAx 8.343 6.409 D. Implementation of traditional machine learning models
Table 2a. Average values of the first four features selected initially Because of the small dataset available we used traditional
machine learning models- SVM and k-NN. The objective of
multi-class classification was solely to observe whether a similar
accuracy can be achieved in discriminating turns from routine
movements by training models with exact the same features.

• Binary Classification- SVM models with linear and

Gaussian kernels and k-NN were trained using the three
selected features and implemented on training set of
474 events out of which only 4 were Escape events

• Multi-class Classification The same features were used

to train classifiers for multiple classes. To identify the
most accurate SVM kernel classifier, we tuned the
polynomial order of kernel function and plotted them
against their performance metrics; (macro-precision
computed for training, validation and test sets). Linear
kernel function was identified as the most efficient
model among the three polynomial functions (1,2 and
3).

• Gaussian Kernel vs Linear Kernel- Gaussian and linear

Table 2b. Dimensionality reduction using PCA, the principal
variability of the first three principal components accounts to
95.19%
SVM models were compared using ROC curves for
classifying Turns. While both the models showed high
accuracy in classifying FSS (Fast-start) events, the
AUC of linear kernel ROC was close to 0.5 for Turns.
The Gaussian Kernel AUC for the same class of events
was 0.82.
 4

E. Results and performance metrics

• kNN at n=3 proved to perform the best for both binary Kernel Function Validation Testing Accuracy
Accuracy
and multi-class classification in identifying FSS (Fast-
Linear 96.8% 96.7%
start) and Turn (Fig 6.). Gaussian 96.7% 96.8%

• SVM with Gaussian kernel was a better Table 3. Summary of accuracy results for multi-class SVM models
discriminator for classifying turn events with 96.8%
accuracy on test-dataset.

• ROC plots for both SVM and k-NN models trained

for binary classification had AUC values= 1 which
points to the high discriminatory power of features
selected to classify even an extremely biased
dataset.

• From PCA we observed that although range

feature did not contribute to the variance of the
dataset its absence as a feature dramatically reduced
the performance of classifiers.

Fig 6. Confusion matrices computed for k-NN at n=3

II. CONCLUSION, SIGNIFICANCE AND FUTURE SCOPE

This study allowed us to make significant observations and

conclusions to help direct future work in understanding
predatory behaviors and responses. Through optimization and
wide-scale implementation of this techniques wireless
accelerometer tags can be used to track and predict marine
animal activity in the wild. Below are the derived conclusions:

• The sensory region of the red drum fish with maximum

discriminatory power to recognize escape movement
patterns is the tail region as opposed to the mid and
head regions.

• kNN demonstrated good predictive performance even

for an extremely biased dataset. It allowed us to
Fig 5. kNN ROC plots for classifying Turns (top) and Escape discriminate small short-timed swimming variations
movements (below) with AUC as 0.985 and 1, respectively.
using acceleration signal only.
 5
• Precision and recall rates for identifying complex
motions will be better if the data is obtained from the data
logger, which combines an accelerometer, a gyroscope
and a magnetometer.

• Larger datasets and shorter windows containing precise

detailed motion can maximize performance of the
classifiers.

The contribution of this paper for future work is extending this

method to be used as a diagnostic tool for detecting abnormal
fish behavior and ecological changes. With computer vision
gaining high prevalence the videos can be used for trajectory
analysis and the two datasets can be combined to build a more
robust method for activity tracking of marine animals [5].

REFERENCES
[1] Brownscombe JW, Gutowsky LFG, Danylchuk AJ, Cooke SJ (2014)
Foraging behaviour and activity of a marine benthivorous fish estimated
using tri-axial accelerometer biologgers. Mar Ecol Prog Ser 505:241-
251.
[2] Kai Kuspa, Tony Pratkanis, Classification of Mobile Device
Accelerometer Data for Unique Activity Identification (2013)
http://cs229.stanford.edu/proj2013/PratkanisKuspa-
[3] https://www.youtube.com/watch?v=7KRz_yKskbQ&ab_channel=MAS
TS
[4] Franziska Broell, Takuji Noda, Serena Wright et al.,Accelerometer tags:
detecting and identifying activities in fish and the effect of sampling
frequency, Journal of Experimental Biology (2013) 216: 1255-1264;
doi: 10.1242/jeb.077396
[5] C. Beyan, R. B. Fisher, (2013), Detection of Abnormal Fish Trajectories
Using a Clustering Based Hierarchical Classifier, Proc. of British
Machine Vision Conference (BMVC), Bristol, UK.
[6] Wang, X., Smith, K. & Hyndman, Characteristic-Based Clustering for
Time Series Data, Data Min Knowl Disc (2006)
[7] Borell et.al,(2013), J Exp Biology Accelerometer tags: detecting and
identifying activities in fish and the effect of sampling frequency.
[8] (a) Monitoring escape and feeding behaviors of cruiser fish by initial
and magnetic sensors Fish by Inertial and Magnetic Sensor
[9] (b) The Kinematics and Performance of the Escape Response in the
Angelfish (Pterophyllum Eimekei)
[10] Cheol-Hong Min and Ahmed H. Tewfik, Automatic Characterization and
Detection of Behavioral Patterns Using Linear Predictive Coding of
Accelerometer Sensor Data, 32nd Annual International Conference of
the IEEE EMBS 2010.
[11] doi:10.1371/journal.pone.0079392.g001
[12] https://www.microdaq.com/msr145-multifunction-data-logger.php