DEEP LEARNING IN PYTHON

Introduction to
deep learning
 Deep Learning in Python

Imagine you work for a bank

You need to predict how many transactions each customer
will make next year
 Deep Learning in Python

Example as seen by linear regression

Age

Bank Balance Number of

Transactions

Retirement Status

 Deep Learning in Python

Example as seen by linear regression

Model with no interactions Model with interactions

Predicted Predicted
Transactions Not Retired Transactions Not Retired

Retired Retired

Bank Balance Bank Balance

 Deep Learning in Python

Interactions
Neural networks account for interactions really well
Deep learning uses especially powerful neural networks
Text
Images
Videos
Audio
Source code
 Deep Learning in Python

Course structure
First two chapters focus on conceptual knowledge
Debug and tune deep learning models on conventional
prediction problems
Lay the foundation for progressing towards modern
applications
This will pay o in the third and fourth chapters
 Deep Learning in Python

Build deep learning models with keras

In [1]: import numpy as np

In [2]: from keras.layers import Dense

In [3]: from keras.models import Sequential

In [4]: predictors = np.loadtxt('predictors_data.csv', delimiter=',')

In [5]: n_cols = predictors.shape[1]

In [6]: model = Sequential()

In [7]: model.add(Dense(100, activation='relu', input_shape = (n_cols,)))

In [8]: model.add(Dense(100, activation='relu')

In [9]: model.add(Dense(1))
 Deep Learning in Python

Deep learning models capture interactions

Age

Number of
Bank Balance Transactions

Retirement Status

 Deep Learning in Python

Interactions in neural network

Input Layer
Hidden Layer
Age
Output Layer
Income
Number of
# Accounts Transactions
 DEEP LEARNING IN PYTHON

Lets practice!
 DEEP LEARNING IN PYTHON

Forward
propagation
 Course Title

Bank transactions example

Make predictions based on:
Number of children
Number of existing accounts
 Deep Learning in Python

Forward propagation
Input Hidden Layer
1
# Children 2 5
1 2
Output
9 # Transactions
-1
-1
3 1 1
# Accounts
 Deep Learning in Python

Forward propagation
Input Hidden Layer
1
# Children 2 5
2
1 Output
9 # Transactions
-1
-1

3 1 1
# Accounts
 Deep Learning in Python

Forward propagation
Input Hidden Layer
1
# Children 2 5
2
1 Output
9 # Transactions
-1
-1

3 1 1
# Accounts
 Deep Learning in Python

Forward propagation
Input Hidden Layer
1
# Children 2 5
2
1 Output
9 # Transactions
-1
-1

3 1 1
# Accounts
 Course Title

Forward propagation
Multiply - add process
Dot product
Forward propagation for one data point at a time
Output is the prediction for that data point
 Deep Learning in Python

Forward propagation code

In [1]: import numpy as np

In [2]: input_data = np.array([2, 3])

In [3]: weights = {'node_0': np.array([1, 1]),

...: 'node_1': np.array([-1, 1]),
...: 'output': np.array([2, -1])}

In [4]: node_0_value = (input_data * weights['node_0']).sum()

In [5]: node_1_value = (input_data * weights['node_1']).sum()

Input Hidden Layer Output

1
2 5 2
1

3 -1
1 -1
1
 Deep Learning in Python

Forward propagation code

In [6]: hidden_layer_values = np.array([node_0_value, node_1_value])

In [7]: print(hidden_layer_values)
[5, 1]

In [8]: output = (hidden_layer_values * weights['output']).sum()

In [9]: print(output)
9

Input Hidden Layer Output

1
2 5 2
1
9
3 -1
1 -1
1
 DEEP LEARNING IN PYTHON

Lets practice!
DEEP LEARNING IN PYTHON

Activation
functions
 Deep Learning in Python

Linear vs Nonlinear Functions

Linear Functions Nonlinear Functions

 Deep Learning in Python

Activation functions
Applied to node inputs to produce node output
 Deep Learning in Python

Improving our neural network

Input Hidden Layer
1
2 5
1 2
Output
9
-1
-1

3 1 1
 Deep Learning in Python

Activation functions
Input Hidden Layer

1 tanh(2+3)
2
2
1 Output

9
-1
-1

3 1
tanh(-2+3)
 Deep Learning in Python

ReLU (Rectified Linear Activation)

Rectifier
 Deep Learning in Python

Activation functions
In [1]: import numpy as np

In [2]: input_data = np.array([-1, 2])

In [3]: weights = {'node_0': np.array([3, 3]),

...: 'node_1': np.array([1, 5]),
...: 'output': np.array([2, -1])}

In [4]: node_0_input = (input_data * weights['node_0']).sum()

In [5]: node_0_output = np.tanh(node_0_input)

In [6]: node_1_input = (input_data * weights['node_1']).sum()

In [7]: node_1_output = np.tanh(node_1_input)

In [8]: hidden_layer_outputs = np.array([node_0_output, node_1_output])

In [9]: output = (hidden_layer_output * weights['output']).sum()

In [10]: print(output)
1.2382242525694254
 DEEP LEARNING IN PYTHON

Lets practice!
 DEEP LEARNING IN PYTHON

Deeper networks
 Deep Learning in Python

Multiple hidden layers

2 -1 Age
3
-3
4 1

4
2 7
-5 2
55

Calculate with ReLU Activation Function

 Deep Learning in Python

Multiple hidden layers

2 -1 Age
3
-3
4 1

4
2 7
-5 2
55

Calculate with ReLU Activation Function

 Deep Learning in Python

Multiple hidden layers

2 -1 Age
3
-3
4 1

4
2 7
-5 2
55

Calculate with ReLU Activation Function

 Deep Learning in Python

Multiple hidden layers

2 -1 Age
3
-3
4 1

4
2 7
-5 2
55

Calculate with ReLU Activation Function

 Deep Learning in Python

Multiple hidden layers

2 -1 Age
3
-3
4 1

4
2 7
-5 2
55

Calculate with ReLU Activation Function

 Deep Learning in Python

Multiple hidden layers

2 Age
3

55

Calculate with ReLU Activation Function

 Deep Learning in Python

Multiple hidden layers

2 Age
3

55

Calculate with ReLU Activation Function

 Deep Learning in Python

Multiple hidden layers

2 Age
3

55

Calculate with ReLU Activation Function

 Deep Learning in Python

Multiple hidden layers

2 Age
3 26

55

Calculate with ReLU Activation Function

 Deep Learning in Python

Multiple hidden layers

Age
3 26

-5
55

Calculate with ReLU Activation Function

 Deep Learning in Python

Multiple hidden layers

Age
3 26

-5
55

Calculate with ReLU Activation Function

 Deep Learning in Python

Multiple hidden layers

Age
3 26

-5
55

Calculate with ReLU Activation Function

 Deep Learning in Python

Multiple hidden layers

2 -1 Age
3 26
-3
4 1

4
2 7
-5 2
55 0

Calculate with ReLU Activation Function

 Deep Learning in Python

Multiple hidden layers

2 -1 Age
3 26 0
-3
4 1

364
4
2 7
-5 2
55 0 52

Calculate with ReLU Activation Function

 Deep Learning in Python

Representation learning
Deep networks internally build representations of pa"erns in
the data
Partially replace the need for feature engineering
Subsequent layers build increasingly sophisticated
representations of raw data
 Deep Learning in Python

Representation learning
 Deep Learning in Python

Deep learning
Modeler doesnt need to specify the interactions
When you train the model, the neural network gets weights
that find the relevant pa"erns to make be"er predictions
 DEEP LEARNING IN PYTHON

Lets practice!