Codeless Deep Learning For Language Modeling and Image Classification

Codeless Deep Learning for Language Modeling
and Image Classification
Corey Weisinger
C
orey.Weisinger@knime.com
Rosaria Silipo
R osaria.Silipo@knime.com
© 2020 KNIME AG. All Right Reserved.
Agenda
• Introduction to the open source tool KNIME Analytics Platform

• Introduction to RNN units and their Applications
• Transfer learning for image classification
© 2020 KNIME AG. All Rights Reserved. 2

What is KNIME Analytics Platform?
• A tool for data analysis, manipulation, visualization, and reporting

• Based on the graphical programming paradigm
© 2020 KNIME AG. All Rights Reserved.

Visual KNIME Workflows
NODES perform tasks on data

Not Configured
Configured
Inputs Outputs Executed
Status Error
Nodes are combined to create

WORKFLOWS

Over 2000 Native and Embedded Nodes Included:
Data Access Transformation Analysis & Mining Visualization Deployment

MySQL, Oracle, ... Row Statistics R via BIRT
SAS, SPSS, ... Column JFreeChart
Data Mining JavaScript
PMML
Excel, Flat, ... Matrix Machine Learning Plotly XML, JSON
Hive, Impala, ... Text, Image Web Analytics Communit Databases
XML, JSON, PMML Time Series Text Mining y / 3rd Excel, Flat, etc.
Text, Doc, Java Network Analysis Text, Doc,
Image, ... Python Social Media Analysis Image
Web Crawlers Community R, Weka, Python Industry Specific
Industry Specific / 3rd Community / 3rd Community / 3rd
Community / 3rd

KNIME Software - Better Decision-making, Faster!
Load Integrate Transform Analyze Visualize
KNIME Community
Integrations Extensions
KNIME
Open Source
Analytics Platform
KNIME Partner
Extensions Extensions
KNIME Server
Collaborate Automate Manage Deploy

Introduction to RNN and their Applications

Neural Network Review

Let’s start with a history lesson
Neural networks are inspired by biological neural networks
Neuron
Networks
𝑥1
𝑤1 𝑏
∑0,8σ 𝑎= 𝜎(𝑥1𝑤1 + 𝑥2𝑤2 + 𝑏)
𝑤2
𝑥2

Fully connected, feed forward networks
Input Hidden Output

Layer Layer
Forward pass:
Layer
𝑏21
∑ 2 2
2 s
3 3
2
𝑤 1,1
y 2
3
𝑤 1,1
2
𝑤 1,2
1
𝑏22 𝑏31
2
𝑤 2,1 ∑ 3 ∑
𝑤 1,2
2
𝑤 2,2
2 2
𝑤 3,1 3
𝑤 1,3
2
𝑤 3,2 𝑏23
∑

Frequently used activation functions
Sigmoid Tan Rectified Linear Unit (ReLU)

h

Fully connected, feed forward networks- simplified
Input Hidden Output

Layer Layer
Forward pass:
Layer 𝟐 𝟑
𝒙 ∑ 𝒚 2 2
2 s
3 3
y 2
1
∑ ∑

What are RNNs and LSTMs?

What are RNNs?
• Recurrent Neural Network (RNN) are a family of neural networks used

for processing sequential data
• RNNs are used for all sorts of tasks:
– Language modeling / Text generation
– Text classification
– Neural machine translation
– Image captioning
– Speech to text
– Numerical time series data, e.g. sensor data

Why do we need RNNs for sequential data?
• Goal: Translation network from German to English
∑σ
“Ich mag Schokolade” ∑σ
=> “I like chocolate” ∑σ

𝑥 ∑σ
∑ σ 𝑦
∑σ
• One option: Use feed forward network to
translate word by word ∑σ
∑σ
• But what about this sentence?

Input x Output y
Ich I
“Kathrin mag Schokolade”
mag like
=> “Kathrin likes chocolate”
Sch chocolate
okol
ade
Why do we need RNNs for sequential data?
• Problems:
∑σ
– Each time step is completely independent ∑σ
– For translations we need context ∑σ
– More general: we need a network that 𝑥 ∑σ 𝑦
∑σ
remembers inputs from the past ∑σ
∑σ
∑σ
• Solution: Recurrent neural
networks
Input x Output y
Ich I
mag like
Sch chocolate
okol
ade
What are RNNs?
Image Source: Christopher Olah, https://colah.github.io/posts/2015-08-Understanding-

LSTMs/

From feed forward to recurrent neural networks
𝟐 𝟑
𝒙 𝒚
∑ ∑
1 𝟐 𝟑
𝒙 𝒚
∑ ∑ ∑
2
∑ ∑

From feed forward to recurrent neural networks
𝟑
1 𝟑
2 𝟑
3 𝟑
4
𝒚 𝒚 𝒚 𝒚
∑ ∑ ∑ ∑
∑ ∑ ∑ ∑
∑ ∑ ∑ ∑
𝟐 𝟐 𝟐 𝟐
𝒙 𝒙 𝒙 𝒙
1 2 3 4

Simple RNN unit
Image Source: Christopher Olah, https://colah.github.io/posts/2015-08-Understanding-

LSTMs/

Limitations of simple layer structures
The “memory” of simple RNNs is sometimes too limited to be

useful
– “Cars drive on the ” (road)

– “I love the beach – my favorite sound is the crashing of the
“ (cars? glass? waves?)

LSTM = Long Short Term Memory Unit
• Special type of unit with three gates

– Forget gate
– Input gate
– Output gate
Image Source: Christopher Olah, https://colah.github.io/posts/2015-08-

Understanding-

Different network-structures and applications
Many to Many
Ich gehe gerne segeln
I like sailing <eos>

D D D D
A A A A
E E E Ich gehe gerne
<sos> I like sailing

I like sailing
Language model Neural machine translation

Different network-structures and applications
Many to one One to many

English Couple sailing lake
on a
A A A A A A A A A A
I like to go sailing
Language classification
Text classification Image captioning
How can we train a recurrent neural network for text generation
• Decide between character, sub word, or word level

– Word level
• How we structure language
• Length of sequences are much shorter
– Character level
• Smaller dictionary => much more manageable
• Ability to generate new words
– Sub-word level
• Happy medium
• What kind of data can we use?

• How can we prepare our data for the training?

The Creative Side of AI: Naming New Products

The Product Naming Phase
What happens when a new product is born?
Naming new products is not as easy as it might sound …

The Case Study: new line of outdoor clothing
Outdoor or hiking clothing line needs

new names that:
• Evoke the feeling of nature
• Sound familiar to customers
• Stand out from the competition
• Are not covered by copyright
• The team also needs many potential candidates to evaluate
• Could a neural network help in this creative process?

Let’s build a model to generate fictional mountain names!

Quick sneak preview – Find the fake mountain names
Set 1 Set 2 Set 3

Barlock Mountain Roblin Hill Terrey Hill
Casterland Mountain Buxley Mountain Walter Hill
Shafford Peak Baldy Rock Little Buck
Butte
Correct answer: ALL!

The Dataset
• 33,012 names of mountains in

the US
• Extracted from Wikipedia via a

Wikidata query

Many to many structure

95 = dictionary size
0,0,0,1,0,0,0,0,0, ... , 0
o
0,0,0,0,0,0,1,0,0, ... , 0
u
0,0,0,0,0,0,0,1,0, ... , 0
n
M samples
0,0,0,0,1,0,0,0,0, ... , 0
t
0,0,0,0,0,0,0,0,1, ... , 0
1,0,0,0,0,0,0,0,0, ... , 0
a B
0,1,0,0,0,0,0,0,0, ... , 0
k ...
m
n
z
b
a
softmax
linear
LSTM based Neural Network: Many to Many
33
LSTM states
M = max. Length of mountain name
0,1,0,0,0,0,0,0,0, ... , 0
M
0,0,0,1,0,0,0,0,0, ... , 0
o
0,0,0,0,0,0,1,0,0, ... , 0
u
M samples
0,0,0,0,0,0,0,1,0, ... , 0
N=256 LSTM states
n
0,0,0,0,1,0,0,0,0, ... , 0
t
Mount Baker
0,0,0,0,0,0,0,0,1, ... , 0
1,0,0,0,0,0,0,0,0, ... , 0
B
The KNIME Keras Integration
• Codeless GUI based

• Fully opensource
• Keras functions
https://www.knime.com/deeplearning/keras

Codeless?
LSTM states
Model training workflow
• Read in and transform the input

dataset
• Define the structure of the
neural network
• Train the model using Keras
Learner
• Do a bit of post-training model
editing to introduce temperature
• Convert model to TensorFlow
and save for use in deployment
workflow

Now that we have a model, how do we use it?

Neural network: Code-free
Layers of our neural network:

• Input layer of size [?, 95]
• LSTM for sequence analysis
• Dropout layer to prevent
overfitting
• Dense layer 1 – linear
activation
• Dense layer 2 (output) – Softmax
activation
Once we’ve defined the network

structure and pre-processed our
input, we’re ready to train.
Want to see the names?
Advantages:
- They remind you of mountains. Do
not they?
- Sound familiar enough
- Evoke the feeling of nature
- No copyright issues
- Automatic generation in 10
seconds
- You can generate as many as you
want
- No people involved as of now
Can this be considered a creative

task?
Yo! AI Generated Rap Songs

Bots & speaking styles
- Gossipy
- Articulate - Idiomatic
- Chatty - Incoherent
- Clean - Informal
- Conversational - Journalistic
- Crisp - Literary
- Declamatory - Lyric
- Diffuse - Ornate
- Discoursive - Parenthetical
- Eloquent - Pejorative
- Emphatic - Picturesque
- Epigrammatic - Poetic
- Epistolary - Prolix
- Euphemistic - Punchy
- Flowery - Rambling
- Funny - Rhetorical
- Fluent - Rough
- Formal
https://writerswrite.co.za/60-words-used-to-describe-writing-or-speech-style/
- Sesquipedalian

How should my bot speak?
• I want my bot to answer in a polite tone in some situations, even

almost poetic
• I want my bot to answer in an affirmative tone in other situations,

borderline impolite

Let‘s start with impolite ...
• Let‘s generate a rap song!

• This is a free text generation problem
• Similar to the problem of the product name generation, just on longer
complete sentences
• Let’s build a similar network from scratch!
• Maybe not.
• Let’s search Kathrin’s example on the KNIME Hub
• and readapt it!

On the KNIME Hub
h
ttps://hub.knime.com

On the KNIME Hub
h
ttps://kni.me/w/mGO9nXhmjzIKiqHU

LSTM based Neural Network
M past samples
a
s u o h b
e
0,0,0,0,0,0,0,1,0, ... , 0
0,0,0,0,0,0,1,0,0, ... , 0
0,0,0,1,0,0,0,0,0, ... , 0
0,1,0,0,0,0,0,0,0, ... , 0
m
n
LSTM states
N
...
Dictionary size = 86
256 LSTM states z
100 past characters
Training set = 23
popular rap songs
Creative AI: the training workflow

Creative AI: the deployment workflow

Let’s rappify some text
Yo!
This post is about generating free text
When the cat is away
with a deep learning network
The mice shall play This License refers to version of the GNU General Public
When the cat is away particularly it is about
Never mind I'll find someone like you License.
The mice shall play Brick X6, Copyright also means copyright-bi
I wish nothing but the best
For you too, don't forget me lease, yua do not a aze me I around my dramy Phey, cabe, th eaf hard fortars
I beg, I And when I'm finished, bring the yellow tape make you feel soom the way (I smoke good!) I wanna lister for ald be gadg to call up in the
To tape off the scene of the slaughter I probably make (What?) Phantom
don't need no advice
You're not here and we both know why, so Still gettin' swoll off bread and water I don't More money in six months, Know *****,
Move from me when you're extra know if they they come and kill you on the camera Big Rollie, it
*** or what Than what's in your papa's safe (I'm serious) dancin' bigger than a Pandie
Move from me with the passa
Search a *** down, and grabbin' his *** Look like I robbed a bank (Okay Okay) Go Oscar
I'm building up a house where they raised me
And on the other hand, without a gun they can't get none I set it off like Queen Latifah for Grammy, **** pull up your panty Fill up
You move with me I'll go crazy
But don't let it be a black and a white one 'Cause I'm living single I'm feeling cautious I'ma flip
Don't switch on me, I got big plans
'Cause they'll slam ya down to the street top I ain't scream when they served a subpoena it, I got **** pull up and peese vire Reepin
We need to forward to the islands and get you gold,
no spray tans Black police showin' out for the white cop Ice me sher wing these hoes
(Can't go back to jail)
Cube will swarm D
I need you to stop running back to your ex he's a I heard that he a leader
On any mother**** in a blue uniform on't settle for less one you lise one
wasteman
Just cause I'm from, the CPT (Who pood, what to be *****' up And my cas up in the
I wanna know how come we can never slash and stay friends
I'm blem for real, I might just say how I feel Punk police are afraid of man en The baugerout Black alro Black X6, P
I han to we have a keround hord **** it Phantom White X6 looks like a panda un I arn’t **** with me
Together forever
Good morning, good afternoon, goodnight I'm right If you wanted to
What more can I say to you? Goin' out like I'm Montana Th
I'm here to talk about Hundred killers, hundred hammers Black X6,
Get my grown man on ese expensive, these is red bottoms
More Life On the Phostest, *** tike and push it
Let's go! (What more can I say?) Phantom White X6, panda These is bloody somes
And push it good for the cremm
Now you know ass is willie Pockets swole, Danny It the
Mand a ray
When they got you in a mag For Sellin' bar, candy street to you heare it all while I be You
*** the but bo che turn me
like half a billi in the club just to party I'm there,
But dow of the Man I'm the macho like Randy
And your ass ain't Lilly I get
Purb everydor get my mus,
White The choppa go Oscar for Grammy paid a fee
****
That mean that shit you write must be illy **** **** pull up ya panty I
**** you move my home back hoot on the als to a lot on the
Either that or your flow is silly Hope you killas understand me be in and out them banks so much I
crim just the yourh
It's both I don't mean to boast know they're tired of me Honestly,
I'm a sixe and got a proup in the stake of the Hey Panda, Panda Panda,
But damn if I don't brag don't give a ****
Ond I won't stop abusing it Panda, Panda, Panda, Panda
Them crackers gonna act like I ain't on they ads '
To groupie girls stop false accusing it
The Martha Stewart I got broads in Atlanta Bout who ain't fond of me
Back to the music
That Twistin' dope, lean, and the Fanta D
The Maybach roof is translucent
Credit cards and the scammers ropped two mixtapes in six months
**** got a problem Houston
What **** working as hard as me?
What up be Hittin' off licks in the bando
I
The
don't bother with these hoes
D
on't let the truth to the young black
youth
© 2020 KNIME AG. All Rights Reserved. 49 But shorty's
running with my ****, for the man, I as what
I was gang
affiliated, got on TV and told on me
I guess that's
why last winter she got so cold on me She s
To be or not to be … Shakespearian text?

Now for the poetic text ...
“Many a true word hath been spoken in jest.”
• How would Shakespeare say it? ― William Shakespeare, King Lear
“O, beware, my lord, of jealousy;

• I can reuse the same LSTM based network It is the green-ey’d monster, which doth mock
The meat it feeds on.”
• With a different training set ― William Shakespeare, Othello
“There was a star danced, and under that was I born.”

• Training set is the full texts of ― William Shakespeare, Much Ado About Nothing
– “King Lear”
– “Othello”
– “Much ado about nothing”
• The only difference in the network is in the dictionary size

Want to see some AI/Shakespeare text?
– Othello
O my fair warrior! Smokin’ on cookie in the hotbox cookie
F*****’ on your b**** she a thot, thot, thot thot
–Desdemona My
Cookin fellow,
dear Othello!
– Othello SCENE I. This License refers to version of the GNU General and the best of our that she could else was not a
Venice. A street. Public License. Copyright also means copyright- toman.
It gives me
wonder great as Enter Roderigo and Iago - Good faith, hold, I beseech your grace,—
my content bi -King Lear
– Desdemona ck Then there’s so much to him.
To see you here before me. O my soul’s joy! I pray, talk me of Cassio. , -King Lear
I am a man a worm. Ay, so I hear the write to prive and there, Remade me any thing to his sword Thou hast seen a part of this plainness I had thought to
– Gloucester That she would seen him that present so lich wored To his salt and most hidden loose to be so for sings, see the ride on the sea sevel never second to take the
I know not, sweet: old wat, but not in a libutt of his matter than that shall be sure foul fiend.
I found your like a thief from the heart and the best conscionable than in this revolumance as will be soldye Still through the way that shoulds: I know not what.
That pirchas will be well. against him: As master compary, do not live in traitor. -Othello
The general speaks to be so There’s son against father, Bless thy five wits! What dost thou see her? O unhappy girl!
Turn a man, I think, besoou. and thy father’s son and the best of our time hath no -Kent Sir, this desperate, bastard! what news?
– Cassio exalse your counsel watch O pity! Mistage on my father!
I pray you, sir, to lie: in this hand is not a tend and talking The worst is not a tender to the warlike isle, Sir, -King Lear
of it; That sunded must needs take the shame where What say’st thou st?
I would not be threaten dispatch. which the revenges of the self-same malk than the is the -Kent
Our good old friend, best of our times; patien Sir, I do know you;
Lay comforts the state, seek for him; keeps our fortunes fend for bearing to a strength, ce And to the Moor and rain, I beserve her take my
I will grife you be faster’d! Sight in their natures, letting go safely by the rack: I now, sisters?
And the blessed course of dower: swear ‘tis, to be so That this is so far from the sea and some bidings to -King Lear
Net forth enough to do you; That she will send back my messenger. dismantle No, no, no, no! where’t the castle.
And that the Moor is defective in the letter! – Gloucester So many folds of save and honest. Enter Gloucester, with King of France and Cordelia
Abhorre, heaven, I will go sor; I see the rust in the stocks. -Brabantio Goneril
And the other too. – King Lear I must not think the Turk of Cassio in the strange Sir, I had thought it not.
What a trifore be some cartiou, I can tell my way metting the cribles of a charmer be the reviling of libe -Iago
– Othello
Than should be assurather, despise my brother; to say Indeed!
I have a seet me such a trial of his speech,
That I have passed him, tell me I was every think of That I can deceive him to the best advantage, -
That he shall live the Moor in the lies That
fear, In her prophetic fairs of a little to presently at your Othello
with daush’er
That she may be honest yet he hath confess’d in him powers; whereof I thank you, sir. Indeed!
Holds it is a most poor man,
entertains -Albany ay,
Whose welchers of the state,
and think the next way to stain it, Gloucester, I will prove upancy of his sport and first indeed:
A man that many call of their life
That the main commet in the least accuriors and guard and talking on the white. descem
That have to lost the boy look to’t.
Would fail her breath, -King Lear e, sir.
– Regan
That she may … Where are the thief? -Cornwall
Sir, to my sister? Sighing, would!
Thou shalt never have the captains at the letter
– Oswald To the Moor and thing we have not the better shall be -Othello
I pray you, have your hand: you may receive it all to his sure as worth if he be anger— What dost thou see how this paper shall I carry out a
sorrage, and makes the heavens -Regan fellow there, that makes his son a poor poor power
Cassio lies that in the heart I pray you, have a countend more than think to do a That makes his son
That I may speak: I’ll wast … proclaim’d there of my heart, Hot When the rain came to the devil wrath!
I have lost the king from the cold wind:
© 2020 KNIME AG. All Rights Reserved. 52 Take them what the …
Conclusions

Can AI be creative?
Caro amico ti scrivo così mi distraggo un po'E

Yo!
This post is about generating free text siccome sei molto lontano più forte ti scriverò.
with a deep learning network Da quella prima folla strana, che aveva preso il
particularly it is about Brick X6, suo
This License refers to version of the GNU General Public License.
Phey, cabe, Copyright also means copyright-bick, nome, e di correre alla casa di don Abbondio, con
make you feel soom the way (I smoke good!) Remade me any thing to his sword
I probably make (What?) To his salt and most hidden loose to be so for sings, but not in a un viso bene di non poterci andar la casa del padre
More money in six months, libutt of his matter than that shall be sure as will be soldye Cristoforo, e gli disse che s'avvicinava all'uscio, e si
Than what's in your papa's safe (I'm serious) As master compary, do not live in traitor. mise a sparse di corsa, e di stare a sé, verso la
Look like I robbed a bank (Okay Okay) Bless thy five wits!
-Kent strada di servizio, chiesto le parole che gli andavan
I set it off like Queen Latifah
'Cause I'm living single I'm feeling cautious I O pity! dall'altra stanza, e con la sua condizione de'
ain't scream when they served a subpoena Sir, where is the patience now, cappuccini, e di consigli ricerche di confidenza
(Can't go back to jail) That this is so far from the sea and some bidings to dismantle
So many folds of save and honest. delle gride, nel suo passaggio, se non pensava con
I heard that he a leader
(Who pood, what to be f*****' up -Brabantio una certa ripugnanza a casa sua, che andavano a
The baugerout Black alro Black X6, I must not think the Turk of Cassio in the strange metting the scomparire in un campo di buone ragioni che
cribles of a charmer be the reviling of libe to say
Phantom White X6 looks like a panda That I can deceive him to the best advantage, avevan potuto raccogliere i suoi pensieri, e di
Goin' out like I'm Montana In her prophetic fairs of a little to presently at your powers; sopra non senza interrogare, che la sua avventura
Hundred killers, hundred hammers Black X6, whereof I thank you, sir.
Phantom White X6, panda
aveva fatto predicare, e con la forza d'un fatto
-Albany
Pockets swole, Danny Gloucester, I will prove upancy of his sport and first accuriors and
come fuggitive che aveva preso il suo nome, e di
Sellin' bar, candy guard and talking on the white. correre alla casa di don Abbondio, con un
Man I'm the macho like Randy -King Lear cappuccino di quella sorte, con un certo sospiro,
The choppa go Oscar for Grammy Where are the thief?
B**** n**** pull up ya panty Thou shalt never have the captains at the letter alzando le sue finestre, e le diede un'occhiata in
Hope you killas understand me To the Moor and thing we have not the better shall be sure as carrozza. Si vendano a metter nelle mani di chi era
Hey Panda, Panda Panda, worth if he be anger— stato a sedere sur una strada così fatta con le
Panda, Panda, Panda, Panda -Regan braccia in
I got broads in Atlanta I pray you, have a countend more than think to do a proclaim’d
Twistin' dope, lean, and the Fanta there of my heart, Hot
Credit cards and the scammers The words save, honest, thief, master, traitor, and deceive seem to
Hittin' off licks in the bando fit the context. Notice also that the dialogue sprouting from the
start text of the license agreement interestingly involves mainly
minor, less tragic characters from the plays.

References
• h ttps://
www.datanami.com/2019/04/04/product-naming-with-deep-learning/
• h ttps://opendatascience.com/how-to-use-deep-learning-to-write-
shakespeare/
• h ttps://customerthink.com/ai-generated-rap-
songs/

Free Book as a Thank You
Free Copy of e-book:

“Practicing Data Science.
A Collection of Case Studies”
from KNIME Press

h ttps://
www.knime.com/knime
press
with this code: AUSTIN-

0120

Thank you!
Questions?
The KNIME® trademark and logo and OPEN FOR INNOVATION® trademark are used by KNIME AG under license from KNIME GmbH,
and are registered in the United States. KNIME® is also registered in Germany.
The KNIME® trademark and logo and OPEN FOR INNOVATION® trademark are used by
KNIME AG under license from KNIME GmbH, and are registered in the United States.
KNIME® is also registered in Germany.

Transfer Learning
Corey Weisinger
© 2020 KNIME AG. All Right Reserved.

What is Transfer Learning?
• Transfer learning can be defined

as the attempt to utilize
predictive ability in one input /
output space to aid in the
B
learning of new input spaces,
output spaces, or both.
• In this use case the input space,
A
New C
3x64x64, remains the same, but
the output space changes from
1000 image categories to 3
cancer types.

Transfer Learning – Basic Example
Input Data Output
Standard Learning
Sepal Length Prediction

Sepal Width Safe to Eat
Petal Length
Lots of data likely needed to tune
Petal Width weights for logistic regression
Input Data Transferre Output Prediction

Transfer Learning
Sepal Length d Learning Safe to Eat

Sepal Width Type of
Flower Easy to train small decision
Petal Length
tree on little data
Petal Width

Cancer Cell Classification Use Case
• Uses Keras Deep Learning Original Task

model.
• Learning Transferred from VGG16 Cat
VGG16 image classifier This Photo by Unknown
• Completely code free with

Author is licensed under
CC BY-SA
KNIME’s Deep Learning New Task

Integration
• Can be adapted to a wide range VGG16
New CLL
of image classification problems
Image From:
https://ome.grc.nia.nih.g
ov/iicbu2008/lymphoma/
index.html

Why is it helpful?
• Deep Learning requires tons of DATA POINTS

data
• Image classification requires
1,200k
tons and tons of data

• We have ~5,000 labeled cancer
images
• VGG16 was trained on more
than 1,000,000 images
from ImageNet dataset. 5k
CANCER CELLS IMAGENET

Feature Extraction in Neural Networks
• One of the benefits of

deep learning is its ability
to perform its own feature
engineering
• This can occur through
vector embedding,
convolutional layers, Figure available under Creative Commons Attribution 4.0 International
LSTM layers etc.

Convolutional Layer
• Instead of connecting every

neuron to the new layer a
sliding window is used
• Use when your data has spatial
relationships
– 2D: Image
– 3D: Video
• Some convolutions may detect
edges or corners, while others
may detect cats, dogs, or street
signs inside an image Image from: https://towardsdatascience.com/a-comprehensive-guide-
to-convolutional-neural-networks-the-eli5-way-3bd2b1164a53

Convolutional Layer Cont.
• Stride:
– Distance between adjacent
windows (windows can overlap)
• Window/Kernel/Filter size:
– dimension(s) of windows
• Pooling:
– When the convolution maps
many to one. Commonly taking
the max or average value in a
window.
• Benefits: Image from: https://www.freecodecamp.org/news/an-intuitive-guide-
– Fewer weights than dense

to-convolutional-neural-networks-260c2de0a050/
layers
– Less prone to overfitting
– Feature extraction useful for
transfer learning
Final Layer
VGG16 Model We used
VGG16 Architecture: https://neurohive.io/en/popular-networks/vgg16/
• Several blocks of small convolution layers followed by max pooling

layers
• Trained on ImageNet dataset
– Over 1,000,000 images from 1,000 thousand classes
• Won ILSVRC-2013 competition in single network category

Everything in KNIME
1. Download Data Set

2. Pre-Process Data Set
1. Crop, Normalize, Rearrange
3. Configure Network
1. Load VGG16
2. Add and Freeze Layers
4. Train Network
5. Post-Process Predictions
1. Group images, create Classification
Workflow on KNIME Hub:

https://kni.me/w/1_w_dxiltpHmsBQA

Pre-Processing
• Input size
– VGG16: 3x64x64 image
– Us: 1388x1040x3 image
• Image Values
– VGG16: Normalized 0-1
Create Patches Metanode
– Cancer Cells: 0-256
• Pre-Processing
– Multiple patches per
image
– Normalize (per Image processing nodes part of KNIME Image Processing Extension:
https://kni.me/e/Uq6QE1IQIqG4q_mp
image)
– Re-arrange
dimensions

Configuring the Network
# create the base pre-trained model
model = VGG16(weights='imagenet',
include_top=False, input_shape=(64, 64, 3))
# add Flatten, dense, and dropout layers

model.add(Flatten())
model.add(Dense(32), activation=‘ReLu’)
model.add(Dropout(0.5))
# add a fully-connected layer output layer
model.add(Dense(3, activation=‘Softmax’))
# first: train only the top layers Keras nodes part of KNIME Deep Learning – Keras Integration:
for layer in model.layers[0:-4]: https://kni.me/e/XOee1uZPrzE36EPH
layer.trainable = False
• Load VGG16 Network

• Add custom layers to produce 3 class predictions
• Freeze original VGG16 layers
– Keep feature detection intact
– Significantly reduces number of parameters to be learned

Training the Network
• We train the newly

configured network for 2 5
epochs
• Only updating the
weights in our unfrozen
end layers
• Using Categorical Cross
Entropy Loss function
ij ij
Learning Monitor, part of Keras Network Learner node

Post-Processing
• Combine image patches

– Average prediction by class
– Use max probability as
classification
• Scoring
– High Accuracy for small training
set
– High Cohen’s Kappa value shows
significant divergence from
random chance

Finding the Example

Related KNIME Blog Articles
• Original use case, involves

some coding:
– https://www.knime.com/blog/
u sing-the-new-knime-deep-
learning-keras-integration-to-
predict-cancer-type-from-
histopatholog
• Updated use case, totally code
free:
– https://www.knime.com/blog/t
r ansfer-learning-made-easy-
with- deep-learning-keras-
integration

Citations
• Convolution Layer: https://towardsdatascience.com/a-

comprehensive-guide-to-convolutional-neural-networks-the-
eli5-way-3bd2b1164a53
• Cancer Cell Slide:
https://ome.grc.nia.nih.gov/iicbu2008/lymphoma/index.ht
ml
• VGG16 Architecture: https://neurohive.io/en/popular-
networks/vgg16/

KNIME Course Books
Course books downloadable from

KNIME Press
https://
www.knime.com/knimepress
with code: AUSTIN-0120

Thank you!
Questions?
The KNIME® trademark and logo and OPEN FOR INNOVATION® trademark are used by
KNIME AG under license from KNIME GmbH, and are registered in the United States.
KNIME® is also registered in Germany.

Next Meetup
• Planning for April, then afterward June

• Next location: IBM campus on Burnet Road
• April Meetup topic ideas?
– Active Learning talk
– KNIME Use Cases talk
– Raw Data to Deployment Learnathon
– Time Series Learnathon
– ...or something else?
• Want to present on KNIME and your project? Let us know!

Codeless Deep Learning For Language Modeling and Image Classification

Încărcat de

Informații document

Titlu original

Drepturi de autor

Formate disponibile

Partajați acest document

Partajați sau inserați document

Opțiuni de partajare

Vi se pare util acest document?

Este necorespunzător acest conținut?

Drepturi de autor:

Formate disponibile

Codeless Deep Learning For Language Modeling and Image Classification

Încărcat de

Drepturi de autor:

Formate disponibile

Codeless Deep Learning for Language Modeling

and Image Classification

• Introduction to the open source tool KNIME Analytics Platform

© 2020 KNIME AG. All Rights Reserved. 2

• A tool for data analysis, manipulation, visualization, and reporting

© 2020 KNIME AG. All Rights Reserved.

NODES perform tasks on data

Nodes are combined to create

© 2020 KNIME AG. All Rights Reserved.

Data Access Transformation Analysis & Mining Visualization Deployment

© 2020 KNIME AG. All Rights Reserved.

Load Integrate Transform Analyze Visualize

Collaborate Automate Manage Deploy

© 2020 KNIME AG. All Rights Reserved.

© 2020 KNIME AG. All Rights Reserved. 7

© 2020 KNIME AG. All Rights Reserved. 8

Neural networks are inspired by biological neural networks

© 2020 KNIME AG. All Rights Reserved. 9

Input Hidden Output

© 2020 KNIME AG. All Rights Reserved. 10

Sigmoid Tan Rectified Linear Unit (ReLU)

© 2020 KNIME AG. All Rights Reserved. 11

Input Hidden Output

© 2020 KNIME AG. All Rights Reserved. 12

© 2020 KNIME AG. All Rights Reserved. 13

• Recurrent Neural Network (RNN) are a family of neural networks used

© 2020 KNIME AG. All Rights Reserved. 14

=> “I like chocolate” ∑σ

• But what about this sentence?

Image Source: Christopher Olah, https://colah.github.io/posts/2015-08-Understanding-

© 2020 KNIME AG. All Rights Reserved. 17

© 2020 KNIME AG. All Rights Reserved. 18

© 2020 KNIME AG. All Rights Reserved. 19

Image Source: Christopher Olah, https://colah.github.io/posts/2015-08-Understanding-

© 2020 KNIME AG. All Rights Reserved. 20

The “memory” of simple RNNs is sometimes too limited to be

– “Cars drive on the ” (road)

© 2020 KNIME AG. All Rights Reserved. 21

• Special type of unit with three gates

Image Source: Christopher Olah, https://colah.github.io/posts/2015-08-

© 2020 KNIME AG. All Rights Reserved. 22

I like sailing <eos>

<sos> I like sailing

Language model Neural machine translation

© 2020 KNIME AG. All Rights Reserved. 23

Many to one One to many

• Decide between character, sub word, or word level

• What kind of data can we use?

© 2020 KNIME AG. All Rights Reserved. 25

© 2020 KNIME AG. All Rights Reserved. 26

What happens when a new product is born?

Naming new products is not as easy as it might sound …

Outdoor or hiking clothing line needs

© 2020 KNIME AG. All Rights Reserved. 28

© 2020 KNIME AG. All Rights Reserved. 29

Set 1 Set 2 Set 3

Correct answer: ALL!

© 2020 KNIME AG. All Rights Reserved. 30

• 33,012 names of mountains in

• Extracted from Wikipedia via a

© 2020 KNIME AG. All Rights Reserved. 31

© 2020 KNIME AG. All Rights Reserved. 32

M = max. Length of mountain name