Sidney Harris

Deep Learning
Explaining Step 2

Andre Cianflone
What is Deep Learning (DL)?
Large neural networks
Can have a huge number of hidden layers
DL gets better with more data, bigger models, more computation, better
algorithms
Most breakthroughs with supervised learning
Automatic feature extraction
Deep learning algorithms seek to exploit the unknown structure in the
input distribution in order to discover good representations, often at
multiple levels, with higher-level learned features defined in terms of
lower-level features

Yoshua Bengio. (2012). Deep Learning of Representations for Unsupervised and Transfer
Learning. JMLR
 Big Data

graph by: Andrew Ng

Why the name? Why now?
The name seems to come from Geoffrey Hinton's work on Deep Boltzmann
Machines and Deep Autoencoders, 2006
Hinton mentions in a 2006 Science article that only now we have the key
ingredients for deep neural nets: fast computers, big data and smart
initialization.
New wave: Speech Recognition (2009), Image Recognition (2012), Machine
Translation (2014), AlphaGo (2015). Image captioning (2015), Art, Self-driving
cars, many others
Learning
Minimum requirements
Patience
Machine learning knowledge

Recommended
Probability theory
Multivariate calculus
Read. Read. Read.
Today
Neural Networks
Training
Setting up a machine
Coding
ConvNets
LSTM
You, after the
workshop:

You, reading a deep learning paper for the

first time:
Resources
Machine Learning course: Andrew Ng, Coursera
Machine Learning review: Doina Precup, DL Summer School
Neural Networks course: Hugo Larochelle (UdS):
Deep Learning book: http://www.deeplearningbook.org/
ConvNets: Andrej Karpathy (Stanford)
LSTM: Chris Olah blog
Datasets
Online demo
Neural network playground