Sunteți pe pagina 1din 8

9/7/2014 Convolutional Neural Network Workbench - CodeProject

http://www.codeproject.com/Articles/140631/Convolutional-Neural-Network-MNIST-Workbench 1/8
10,720,671 members (82,205 online) 590 Sign out


Member 9960792
home quick answers discussions features community help Search f or articles, questions, tips
Articles General Programming Algorithms & Recipes Neural Networks

Article
Browse Code
Bugs / Suggestions
Stats
Revisions (195)
Alternatives
Comments (173)
View this article's
Workspace
Fork this Workspace
Add your own
alternative version
About Article
Type Article
Licence CPOL
First Posted 28 Dec 2010
Views 200,281
Bookmarked 260 times




, +
Related Articles
Neural Network for Recognition
of Handwritten Digits
A Neural Network on GPU
Online handwriting recognition
using multi convolution neural
networks
Large pattern recognition

Rate: Filip D'haene, 8 Jul 2014
Convolutional Neural Network Workbench
A workbench to create, train, and test convolutional neural networks against the MNIST and CIFAR-10 datasets
Download CNNWB Sources
Download Setup
Introduction
This article is about a framework in C# 4.0 that allows to create, train, and test convolutional neural networks against
the MNIST and the CIFAR-10 dataset of 10 different natural objects. I initially based me on an article by Mike O'Neill
on the The Code Project and gradually added new features that I've found interesting in research documents found on
the internet. Dr. Yann LeCun's paper: Gradient-Based Learning Applied to Document Recognition is a great paper to
get a better understanding of the principles of convolutional neural networks and the reason why they are so
successful in the area of machine vision.
The Code
The main goal of this project was to build a more flexible and extendable managed version of Mike O'Neill's excellent
C++ project. I've included and used the splendid WPF TaskDialog Wrapper from Sean A. Hanley, the Extended WPF
Toolkit and for unzipping the CIFAR-10 dataset the open-source SharpDevelop SharpZipLib module. Visual Studio
2012/2013 and Windows 7 are the minimum requirements. I made maximal use of the parallel functionality offered in
C# 4.0 by letting the user at all times choose how many logical cores are used in the parallel optimized code parts with
a simple manipulation of a sliderbar next to the View combobox.
Using the Code
Here is the example code to construct a LeNet-5 network in code (see the InitializeDefaultNeuralNetwork()
function in MainViewWindows.xaml.cs):
Collapse | Copy Code
NeuralNetwork cnn = new NeuralNetwork
(DataProvider, "LeNet-5", 10, 0.8D, LossFunctions.MeanSquareError,
DataProviderSets.MNIST, TrainingStrategy.SGDLevenbergMarquardt, 0.02D);
cnn.AddLayer(LayerTypes.Input, 1, 32, 32);
cnn.AddLayer(LayerTypes.Convolutional, ActivationFunctions.Tanh, 6, 28, 28, 5, 5);
cnn.AddLayer(LayerTypes.AveragePooling, ActivationFunctions.Tanh, 6, 14, 14, 2, 2);
bool[] maps = new bool[6 * 16]
{
true, false,false,false,true, true, true, false,false,true, true, true, true, false,true, true,
true, true, false,false,false,true, true, true, false,false,true, true, true, true, false,true,
true, true, true, false,false,false,true, true, true, false,false,true, false,true, true, true,
false,true, true, true, false,false,true, true, true, true, false,false,true, false,true, true,
false,false,true, true, true, false,false,true, true, true, true, false,true, true, false,true,
false,false,false,true, true, true, false,false,true, true, true, true, false,true, true, true
};
cnn.AddLayer(LayerTypes.Convolutional, ActivationFunctions.Tanh, 16, 10, 10, 5, 5, new
Mappings(maps));
cnn.AddLayer(LayerTypes.AveragePooling, ActivationFunctions.Tanh, 16, 5, 5, 2, 2);
cnn.AddLayer(LayerTypes.Convolutional, ActivationFunctions.Tanh, 120, 1, 1, 5, 5);
cnn.AddLayer(LayerTypes.FullyConnected, ActivationFunctions.Tanh, 10);
cnn.InitializeWeights();
4.93 (87 votes)
C# Windows
.NET Visual-Studio
Dev QA XAML
WPF Beginner
Intermediate
articles
9/7/2014 Convolutional Neural Network Workbench - CodeProject
http://www.codeproject.com/Articles/140631/Convolutional-Neural-Network-MNIST-Workbench 2/8
system using multi neural
networks
Multiple convolution neural
networks approach for online
handwriting recognition
Neural Network for Recognition
of Handwritten Digits in C#
Neural Networks on C#
An Introduction to Encog
Neural Networks for C#
AForge.NET open source
framework
Designing And Implementing A
Neural Network Library For
Handwriting Detection, Image
Analysis etc.- The BrainNet
Library - Full Code, Simplified
Theory, Full Illustration, And
Examples
AI: Neural Network for
Beginners (Part 3 of 3)
Brainnet 1 - A Neural Netwok
Project - With Illustration And
Code - Learn Neural Network
Programming Step By Step And
Develop a Simple Handwriting
Detection System
C# Application to Create and
Recognize Mouse Gestures
(.NET)
An Introduction to Encog
Neural Networks for Java
Library for online handwriting
recognition system using
UNIPEN database.
AI : Neural Network for
beginners (Part 1 of 3)
Mouse gestures recognition
Creating Optical Character
Recognition (OCR) applications
using Neural Networks
Neural Cryptography
AI : Neural Network for
beginners (Part 2 of 3)
Related Research
Gartner: Magic Quadrant for
On-Premises Application
Platforms
Expanding active decision-
making: The power of
integrating business rules and
events
Toad Oracle: Tips to Simplify
Database Administration and
Development
Design View
In Design View, you can see how your network is defined and get a good picture of the current distribution of weight
values in all the layers concerned.
Training View
In Training View you can train the network. The 'Play' button gives you the 'Select Training Parameters' dialog where
you define all the training parameters. The 'Training Scheme Editor' button gives you the possibility to make training
schemes to experiment with. At any time the training can be paused or aborted. The 'Star' button will forget (reset) all
9/7/2014 Convolutional Neural Network Workbench - CodeProject
http://www.codeproject.com/Articles/140631/Convolutional-Neural-Network-MNIST-Workbench 3/8
Learn Agile: Ten Tips for
Launching and Testing High
Quality Apps for the American
Market
the learned weight values.

Testing View
9/7/2014 Convolutional Neural Network Workbench - CodeProject
http://www.codeproject.com/Articles/140631/Convolutional-Neural-Network-MNIST-Workbench 4/8
In Testing View you get a better picture of the testing (or training samples) which are not recognized correctly .
Calculate View
In Calculate View we test a single testing or training sample with the desired properties and get a graphical view of all
the output values in every layer.
9/7/2014 Convolutional Neural Network Workbench - CodeProject
http://www.codeproject.com/Articles/140631/Convolutional-Neural-Network-MNIST-Workbench 5/8
Final Words
I would love to see a GPU integration for offloading the highly parallel task of learning the neural network I made an
attempt to use a simple MVVM structure in this WPF application. In the Model folder you find
the NeuralNetwork and DataProvider class which provide all the neural network code and deals with loading and
providing the necessary MNIST and CIFAR-10 training and testing samples. Also a NeuralNetworkDataSet class is
used to load and save neural network definitions. The View folder contains four different PageViews and a global
PageView which acts as the container for all the different views (Design, Training, Testing and Calculate). Hope
there's someone out there who can actually use the code and improve on it. Extend it with an unsupervised learning
stage for example (encoder/decoder construction), implement better loss-functions and more training strategies
(conjugate gradient, l-bgfs, ...), add more datasets and use new activation fuctions, etc.
History
1.0.3.7: (07-08-14)
BugFix: slow speed resolved in Testing view
Added SGDLevenbergMarquardtModA training strategy. This can be used with a softmax output layer
Posibility to save the weights while training. Just click on Pause and then Save/Save as...
Various smaller fixes and optimizations
1.0.3.6: (05-02-14)
Choice between four Training Strategies:
SGDLevenbergMarquardt
SGDLevenbergMarquardtMiniBatch
SGD
SGDMiniBatch
BugFix: Derivative of ReLU's activation functions
Added SoftSign activation function
Overall 20% more training performance over previous version
Faster binary save of the network weights
Various smaller fixes and optimizations
1.0.3.5: (03-13-14)
StochasticPooling and L2Pooling layers now correctly implemented
Native C++ implementation + managed C++/CLI wrapper
1.0.3.4: (12-10-13)
Support for Stochastic Pooling layers
Much faster binary load and save of a cnn
ReLU activation function now working as expected
1.0.3.3: (11-26-2013)
Bugfix: Download datasets now working as expected
Bugfix: Softmax function corrected
Bugfix: DropOut function corrected
1.0.3.2: (11-15-2013)
Bugfix: Local layer and Convolutional layer now works properly
Bugfix: Cross Entropy loss now works better (in combination with a SoftMax activation function in a final fully
connected layer)
Added LeCun's standard LeNet-5 training scheme
1.0.3.1: (11-09-2013)
Now the last min/max display preference is saved
Added some extra predefined training parameters and schemes
Bugfix: Average error not showing correctly after switching between neural networks with a different objective
function
Bugfix: Sample not always showing correctly in calculate view
Bugfix: The end result in testing view is not displaying the correct values
1.0.3.0: (11-06-2013)
Supports dropout
Supports the Local layer type (Same as the convolution layer but with non-shared kernel weights)
Supports padding in the Local and Convolution layers (gives ability to create deep networks)
Supports overlapping receptive fields in the pooling layers
Supports weightless pooling layers
Supports all the commonly used activation functions
Supports Cross Entropy objective function in combination with a SoftMax activation function in the output layer
9/7/2014 Convolutional Neural Network Workbench - CodeProject
http://www.codeproject.com/Articles/140631/Convolutional-Neural-Network-MNIST-Workbench 6/8
Filip D'haene
Software Developer
Belgium
No Biography provided
Add a Comment or Question
Search this forum Go

Ability to specify a density percentage to generate mappings between layers far more easily
Improved DataProvider class with much reduced memory footprint and a common logical functionality
shared across all datasets (easier to add datasets)
Much improved UI speed and general functionality
License
This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)
About the Author
Article Top
Comments and Discussions
Profile popups Spacing Relaxed Noise Medium Layout Normal Per page 25 Update
First Prev Next
CatchExAs
3hrs 40mins ago
Filip D'haene
2hrs 34mins ago
MikeMNN
13-Jun-14 7:24
Filip D'haene
13-Jun-14 16:41
MikeMNN
16-Jun-14 8:20
Filip D'haene
17-Jun-14 9:30
MikeMNN
18-Jun-14 4:09
Filip D'haene
18-Jun-14 18:08
Member 10627683
12-Apr-14 6:53
Filip D'haene
14-Apr-14 13:43
Member 10627683
14-Apr-14 21:40
wsnzone
20-Mar-14 5:47
My vote of 5
Re: My vote of 5
Change database [modified]
Re: Change database
Re: Change database
Re: Change database
Re: Change database [modified]
Re: Change database
Please guide me to training CNNs on Sound spectrum
Images
Re: Please guide me to training CNNs on Sound spectrum
Images [modified]
Re: Please guide me to training CNNs on Sound spectrum
Images
rbf wrong
9/7/2014 Convolutional Neural Network Workbench - CodeProject
http://www.codeproject.com/Articles/140631/Convolutional-Neural-Network-MNIST-Workbench 7/8
Permalink | Advertise | Privacy | Mobile
Web04 | 2.8.140705.1 | Last Updated 8 Jul 2014
Article Copyright 2010 by Filip D'haene
Everything else Copyright CodeProject, 1999-2014
Layout: fixed | fluid
Filip D'haene
20-Mar-14 13:10
wsnzone
24-Mar-14 15:25
Ido Freeman
17-Mar-14 8:27
Filip D'haene
17-Mar-14 8:43
Member 10642225
8-Mar-14 4:53
Filip D'haene
11-Mar-14 15:37
Member 10372081
2-Mar-14 1:06
Filip D'haene
6-Mar-14 17:41
Member 10334053
17-Dec-13 5:23
Filip D'haene
30-Dec-13 13:39
Member 10088555
21-Jan-14 19:47
tintinkool1988
27-Nov-13 22:59
Filip D'haene
29-Nov-13 7:49
Last Visit: 31-Dec-99 23:00 Last Update: 9-Jul-14 8:46 Refresh 1 2 3 4 5 6 7 Next
General News Suggestion Question Bug Answer Joke Rant Admin
Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.
Re: rbf wrong
Re: rbf wrong
Zip file corrupted
Re: Zip file corrupted
Please guide me to implement GPU accelerated version
Re: Please guide me to implement GPU accelerated version
Why didn't CNN-CIFAR-10 use higher image resolution on
input layer? [modified]
Re: Why didn't CNN-CIFAR-10 use higher image resolution on
input layer?
how can i store the training setting and training result in my
database?
Re: how can i store the training setting and training result in my
database?
Re: how can i store the training setting and training result in
my database?
Question about time for each epoch
Re: Question about time for each epoch
9/7/2014 Convolutional Neural Network Workbench - CodeProject
http://www.codeproject.com/Articles/140631/Convolutional-Neural-Network-MNIST-Workbench 8/8
Terms of Service

S-ar putea să vă placă și