Human Activity

Recognition using
Smartphone
Amin Rasekh, Chien-An Chen, Yan Lu
CSCE 666 Project Presentation

Outline

Introduction
Human Activity Recognition
Active Learning

Goals
Literature Review
Methods
Data Collection and Feature Extraction
Classification Techniques
Query Strategies of active learning

Results
Conclusions

Introduction: Activity Recognition

Using sensors to identify human activities such as walking,
jogging, limping.

Motivation
Human survey (study human daily activities)
Medical care (diabetes, elderly, rehabilitation)

Sensors types
Inertial sensors (accelerometer, gyroscope)
Camera
GPS

Smartphone is small and convenient to carry around and its

computational resource is powerful enough for our purpose.

Introduction: Active Learning

Passive Learning: What we have studied in class
We can achieve greater accuracy with fewer training labels if
we choose the data from which we learn

Motivation: To minimize the time and labor for labeling

abundant data

Goals

Design a simple, light weight, and accurate

system that can learn human activity with
minimum user interaction.
Compare and find a model that best fit our
system in terms of accuracy and efficiency.
Reduce the labeling time and labor works using
active learning.

Literature review

Use one or multiple camera to do a vision-based

recognition [5,6].

Install multiple inertial sensors on the body. [1, 2, 3,4]

A mixture between vision-based and inertial sensor system.

[7]

Classifiers such as Bayesian Decision Making, KNN, SVM,

ANN were studied before. [10,11]

Features from time domain, frequency domain and wavelet

analysis have been studied.[8,9]

Methods

Data Collection

Smartphone: HTC EVO 4G

Sensor: 3D accelerometer,50 Hz
Cellphone in pockets around waist
3 people 5 activities: walking, biking, walking upstairs,
walking downstairs, jogging, limping

Feature Generation (Total 31 features)

Sampling Window: 256 samples (5.12 seconds)
Time Domain:
Variance, Mean, 25% Percentile, 75% Percentile, Correlation,
Average Resultant Acceleration

Frequency Domain:
Energy, Entropy, Centroid Frequency, Peak Frequency

Methods

Classification Techniques

Quadratic
K-Nearest Neighbors
Support Vector Machines
Artificial Neural Networks

Query Strategies based on Uncertainty

Quadratic: Distance from discriminant curve

KNN:
Entropy
SVM:
Distance from the boundary
ANN
Discriminant function values

Query Strategy: Distance Measure

Query is performed for the unlabeled
instance that is nearest to the discriminant
curve or SVM boundary

Random
Query

Active
Query

Query Strategy: Entropy Measure

Query is performed for the unlabeled
instance that has the maximum entropy:

Results: LDA Subspace

walking
limping
jogging
downstair
upstair

-1

LDA component 2

-1.2
-1.4
-1.6
-1.8
-2

8.5

LDA component 4

-0.8

walking
limping
jogging
downstair
upstair

7.5

-2.2
-2.4
-2.6
-1.8

-1.6

-1.4

-1.2

-1

-0.8

-0.6

LDA component 1

-0.4

-0.2

7
8.4

8.6

8.8

9.2

9.4

9.6

LDA component 3

9.8

10

10.2

10.4

Results: Passive Learning

Results: Feature Subset Selection

Sequential Forward Selection (Wrapper)
Algorithm: SVM
10-Fold Cross Validation for each feature subset
Best Features
Variance, 25% Percentile, Frequency-Domain
Entropy, Peak Frequency

Classification Rate of SVM+LDA:

78%

Classification Rate of SVM+SFS:

84%

Results:

LDA Space for Hw2 Data

Second LDA Component

30

20

10

-10

-20

-30
-50

-45

-40

-35

-30

First LDA Component

-25

Results:

Active Learning on Hw2

Data
Quadratic

KNN

SVM

0.7

Active Learning
Random Sampling

Classifcation rate

0.65

0.6

0.55

0.5

0.45

0.4

0.35

50

100

Number of Instance Queries

150

Results:

Active Learning on Activity

Data
-0.8

-0.8

-1

-1

Active learning
with SVM

-1.6
-1.8
-2

-1.4
-1.6
-1.8
-2

-2.2

-2.2

-2.4

-2.4

-2.6
-1.8

-1.6

-1.4

-1.2

-1

-0.8

-0.6

-0.4

LDA component 1

-0.2

KNN

Random sampling
with SVM

-1.2

LDA component 2

-1.4

-2.6
-1.8

-1.6

SVM
0.75
0.7
0.65
0.6
0.55
0.5
0.45
Active Learning
Random Sampling

0.4
0.35

50

100

150

-1.4

-1.2

-1

-0.8

-0.6

LDA component 1

-0.4

Quadratic

0.8

Classifcation rate

LDA component 2

-1.2

200

Number of instance queries

250

300

-0.2

Works in Progress

Improving the performance of active

learning for activity recognition problem
Clustering
Hybrid query strategies

Adding more activities such as biking

Conclusions

We achieved a classification rate of over

80% on 5 human activities using a
smartphone.

The result is robust to common positions

and orientations of cellphone.

SVM+SFS gives the best performance and is

promising to run on mobile devices.

Performance of active learning is highly

sensitive to the type of problem

Thank you!

Questions?

Reference
1)

L. Bao and S. S. Intille, Activity recognition from user-annotated acceleration data, Pers Comput., Lecture Notes in
computer Science, vol. 3001, pp. 117, 2004.

2)

U. Maurer, A. Rowe, A. Smailagic, and D. Siewiorek, Location and activity recognition using eWatch: A wearable sensor
platform, Ambient Intell. Everday Life, Lecture Notes in Computer Science, vol. 3864, pp. 86102, 2006.

3)

J. Parkka, M. Ermes, P. Korpipaa, J. Mantyjarvi, J. Peltola, and I. Korhonen, Activity classification using realistic data from
wearable sensors, IEEE Trans. Inf. Technol. Biomed., vol. 10, no. 1, pp. 119128, Jan. 2006.

4)

N.Wang, E. Ambikairajah,N.H. Lovell, and B.G. Celler, Accelerometry based classification of walking patterns using timefrequency analysis, in Proc. 29th Annu. Conf. IEEE Eng. Med. Biol. Soc., Lyon, France, 2007, pp. 48994902.

5)

T.B.Moeslund,A.Hilton,V.Kr uger, Asurveyofadvancesinvision-based human

motioncaptureandanalysis,Comput.VisionImageUnderstanding 104 (23)(2006)90126.

6)

T.B. Moeslund, E. Granum, A survey of computer vision-based human motion capture, Comput. Vision Image
Understanding 81 (3) (2001) 231268.

7)

Y. Tao, H. Hu, H. Zhou, Integration of vision and inertial sensors for 3D arm motion tracking in home-based rehabilitation,
Int. J. Robotics Res. 26 (6) (2007) 607624.

8)

Preece S J, Goulermas J Y, Kenney L P J and Howard D 2008b A comparison of feature extraction methods for the
classification of dynamic activities from accelerometer data IEEE Trans. Biomed. Eng. at press

9)

N. Ravi, N. Dandekar, P. Mysore, and M. L. Littman. Activity recognition from accelerometer data. In AAAI, pages 1541
1546, 2005.

10)

S.J. Preece, J.Y. Goulermas, L.P.J. Kenney, D. Howard, K. Meijer and R. Crompton, Activity identification using bodymounted sensorsa review of classification techniques. Physiol Meas, 30 (2009), pp. R1R33.

11)

Altun, K., Barshan, B., Tuncel, O.: Comparative study on classifying human activities with miniature inertial and
magnetic sensors. Pattern Recogn. 43(10), 36053620 (2010), doi:10.1016/j.patcog.2010.04.019

Results (can be removed)

Support Vector Machine

Results:

Active Learning on Hw2

Data

30

30

20

20

10

10

-10

-10

-20

-20

-30
-50

-45

-40

-35

-30

-30
-25 -50

-45

-40

-35

-30

-25