Deep Learning

Subir Varma and Sanjiv Das

2017-10-17
2
Contents

Preface 5

1 Introduction 7

2 Pattern Recognition 9

3 Supervised Learning 11

4 Linear Learning Models 13

5 Neural Networks with Deep Learning 15

6 Training Neural Networks 17

7 Techniques to Improve Gradient Descent 19

8 Techniques to Improve Model Generalization Ability 21

9 Hyper-Parameter Selection 23

10 Deep Learning with R 25

11 Convolutional Neural Networks 27

12 Deep Learning with Python 29

13 Recurrent Neural Networks 33

14 Glossary 35

15 Placeholder 37

3
4 CONTENTS
Preface

5
6 CONTENTS
Chapter 1

Introduction

7
8 CHAPTER 1. INTRODUCTION
Chapter 2

Pattern Recognition

9
10 CHAPTER 2. PATTERN RECOGNITION
Chapter 3

Supervised Learning

11
12 CHAPTER 3. SUPERVISED LEARNING
Chapter 4

Linear Learning Models

13
14 CHAPTER 4. LINEAR LEARNING MODELS
Chapter 5

Neural Networks with Deep Learning

15
16 CHAPTER 5. NEURAL NETWORKS WITH DEEP LEARNING
Chapter 6

Training Neural Networks

17
18 CHAPTER 6. TRAINING NEURAL NETWORKS
Chapter 7

Techniques to Improve Gradient

Descent

19
20 CHAPTER 7. TECHNIQUES TO IMPROVE GRADIENT DESCENT
Chapter 8

Techniques to Improve Model

Generalization Ability

21
22 CHAPTER 8. TECHNIQUES TO IMPROVE MODEL GENERALIZATION ABILITY
Chapter 9

Hyper-Parameter Selection

23
24 CHAPTER 9. HYPER-PARAMETER SELECTION
Chapter 10

Deep Learning with R

25
26 CHAPTER 10. DEEP LEARNING WITH R
Chapter 11

Convolutional Neural Networks

27
28 CHAPTER 11. CONVOLUTIONAL NEURAL NETWORKS
Chapter 12

Deep Learning with Python

import pylab

from keras.models import Sequential

from keras.layers import Dense, Dropout, Activation
from keras.layers.advanced_activations import LeakyReLU
from keras import backend
from keras.utils import to_categorical

#Read in data
import pandas as pd
tf_train = pd.read_csv("BreastCancer.csv")
tf_test = pd.read_csv("BreastCancer.csv")
n = len(tf_train)

X_train = tf_train.iloc[:,1:10].values
X_test = tf_test.iloc[:,1:10].values
y_train = tf_train.iloc[:,10].values
y_test = tf_test.iloc[:,10].values

idx = y_train; y_train = zeros(len(idx));

y_train[idx=='malignant']=1; y_train = y_train.astype(int)

idx = y_test; y_test = zeros(len(idx));

y_test[idx=='malignant']=1; y_test = y_test.astype(int)

Y_train = to_categorical(y_train,2)

#Set up and compile the model

model = Sequential()
n_units = 512
data_dim = X_train.shape[1]

model.add(Dense(n_units, input_dim=data_dim))
model.add(LeakyReLU())
model.add(Dropout(0.25))

model.add(Dense(n_units))

29
30 CHAPTER 12. DEEP LEARNING WITH PYTHON

Figure 12.1: Loss functions and accuracy for the training and validation data sets

model.add(LeakyReLU())
model.add(Dropout(0.25))

model.add(Dense(n_units))
model.add(LeakyReLU())
model.add(Dropout(0.25))

model.add(Dense(n_units))
model.add(LeakyReLU())
model.add(Dropout(0.25))

model.add(Dense(n_units))
model.add(LeakyReLU())
model.add(Dropout(0.25))

model.add(Dense(2))
model.add(Activation('sigmoid'))

model.compile(optimizer='rmsprop',
loss='binary_crossentropy',
metrics=['accuracy'])

#Fit the model

bsize = 32
model.fit(X_train, Y_train, batch_size=bsize, epochs=15, validation_split=0.1, verbose=2)

## OUT-SAMPLE ACCURACY
from sklearn.metrics import confusion_matrix
yhat = model.predict_classes(X_test,batch_size=bsize)
cm = confusion_matrix(y_test,yhat)
print("Confusion matrix = "); print(cm)
 31

acc = sum(diag(cm))/sum(cm)
print("Accuracy = ",acc)
32 CHAPTER 12. DEEP LEARNING WITH PYTHON
Chapter 13

Recurrent Neural Networks

33
34 CHAPTER 13. RECURRENT NEURAL NETWORKS
Chapter 14

Glossary

35
36 CHAPTER 14. GLOSSARY
Chapter 15

Placeholder

37
38 CHAPTER 15. PLACEHOLDER
Bibliography

39