1

CHAPTER 1
INTRODUCTION
The automated 3D reconstruction of neurons in brain EM image stacks remains
one of the most challenging problems in neuroscience. In such problems, neurons
spanning multiple adjacent image slices are expected to be consistently identified and
reconstructed. Conventionally, this problem has been approached as a 2D prediction
task, where each image slice is segmented individually and then, a post-processing
step was performed to generate 3D segmentation. The post-processing step usually
involves heuristic off-the-shelf classifiers that were trained to link similar segments
together across the entire image stack. These classifiers usually rely on hand-crafted
features which incorporate prior knowledge and understanding of the data. Thus,
classifiers that worked well on some problems/ datasets are not guaranteed to perform
similarly in different scenarios. It is thus desirable to design a fully trainable system
with minimal post-processing to perform the 3D segmentation task in an end-to-end
fashion. Currently, deep convolutional neural networks (CNNs) are one of the main
tools used for semantic segmentation. These models are very powerful and capable of
extracting hierarchical features from raw image data. They are characterized by their
ability to learn features directly from the raw images without relying on prior
knowledge.
CNNs have achieved success in different areas of machine learning and
computer vision. Improved performance has been achieved in image classification and
object detection tasks of real deconvolutional, and full-resolution features are not well
preserved. To address this limitation, deconvolutional networks have been proposed
recently by performing actual deconvolutional. The pooling layers are reversed in the
decoding stage by unpooling layers which keep track of the maximum activation
position selected during the pooling operation.
CNN approach attempts to design novel deep models specifically for dense
prediction problems, they do not have explicit mechanisms to address the conflicting
goals in dense prediction problems so it does not suffer from loss of information due
to excessive reduction of resolution.
 2

1.1 RESIDUAL DECONVOLUTION NETWORKS

Accurate reconstruction of anatomical connections between neurons in the
brain using electron microscopy (EM) images is considered to be the gold standard for
circuit mapping. A key step in obtaining the reconstruction is the ability to
automatically segment neurons with a precision close to human-level performance.
Despite the recent technical advances in EM image segmentation, most of them rely
on hand-crafted features to some extent that are specific to the data, limiting their
ability to generalize. They propose a simple yet powerful technique for EM image
segmentation that is trained end-to-end and does not rely on prior knowledge of the
data. This proposed residual deconvolutional network consists of two information
pathways that capture full-resolution features and contextual information,
respectively. We showed that the proposed model is very effective in achieving the
conflicting goals in dense output prediction; namely preserving full-resolution
predictions and including sufficient contextual information. This method achieved one
of the top results. We demonstrated the generality of this technique by evaluating it on
the 2D neural segmentation challenge dataset where consistently high performance
was obtained. We thus expect this method to generalize well to other dense output
prediction problem.
1.2 RESIDUAL DECONVOLUTIONAL NETWORK MODEL
In the design of this model, they intend to achieve three goals:
(1) Generate dense predictions equal in size to any arbitrary-sized input.
(2) Increase the receptive fields of output maps to increase the contextual
information used to make pixel-level decisions.
(3) Achieve pixel-level accuracy by incorporating high resolution feature
information.
We build deconvolutional scheme to generate dense predictions and they
enhance the performance of deconvolutional networks by adding residual connections
between every several stacks of convolution or deconvolutional layers. These shortcut
connections perform projection mapping and are added to the output of the stacked
layers with minimum additional computation cost. It has been shown in that it is much
easier to optimize a residual mapping (with shortcut connections added) rather than
 3

the original plain one. Residual networks in also demonstrated a significant

performance gain as a result of increased network depth on tasks of image
classification and object detection.For this dense prediction network architecture,we
propose to introduce projection shortcuts not just on the convolutional stage
responsible for extracting the feature representations, but also on the deconvolutional
stage responsible for reconstructing the shape and producing the objects segmentation.
We believe that with this design, this network is able to acquire more multi scale
contextual information while reducing the effect of the degradation problem. They
also propose the use of a novel resolution-preserving path to facilitate the
reconstruction of full-resolution output.
The resolution-preserving paths are essentially the projection mapping of the
pooling layer outputs added to the output of the corresponding deconvolutional layer
before performing the unpooling operation. These paths are responsible for
transferring the missing high resolution information from the encoding stage to the
decoding stages. The context-growing and the resolution-preserving paths have
significantly boosted the performance of non residual deconvolutional networks.
1.3 NETWORK ARCHITECTURE AND TRAINING
The RDN architecture is mainly inspired by the ideas in, with three main
differences:
The convolution stage of the network has been mirrored maps instead of a
single value prediction for each training instance. Context-growing paths have been
added to the deconvolutional as well as the convolution layers. Also, resolution-
preserving paths have been added to transfer resolution-specific information from
encoding to decoding stages.
The input to the network is 3D patches extracted from consecutive slices to
exploit the 3D aspect of the data in a way similar to how a human annotator to
perform segmentation. Square patches were extracted randomly from the entire image
stack. The performed mirror padding for patches extracted from the first and last slices
to generate the 3D input to our network.
The minimal post-processing that involves handcrafted features throughout
the entire pipeline. The network contains 23 convolutional layers and 20
 4

deconvolutional layers in total as the network is not entirely symmetric. The kernel
sizes are either 3x3 or 1x1 when we performed branching before adding the residual
paths.
Zero padding was used whenever size preserving was needed in the learned
layers. They added a batch normalization layer after each learned layer and rectified
linear u units were used as the non-linearity transformation. We used a patch size of
128x128x3 in training while the entire image was used in testing. No pre-processing
was used on the raw input images. However, we modified the training labels to reduce
the segment sizes by increasing the border width in-between them. The widening of
borders was done using a minimum kernel of size 5x5 by assuming that all segments
are having a label of 1 and borders are having a label of 0 in the ground truth label
stack.
Any pixel that was in a neighbourhood of size 5x5 of a border pixel was
considered to be border as well. Label widening was crucial in allowing the network
to differentiate border from non-border pixels. This model implementation was based
on the publicly available MATLAB. They trained our RDN using back propagation
with stochastic gradient decent. The mini batch size used was 15 as the dense
prediction requires a lot of memory. However, the network requires roughly 15k
iterations to achieve its full potential due to the existence of residual paths which
speeds up the computations.
To improve the robustness of the resulting probability maps, we applied 8
variations to the testing images before passing them down through the network. A
reverse transformation was then applied to each resulting probability map before
taking the average across all variations. The transformations were combinations of
horizontal and vertical mirroring, and/or rotations by +90, -90 and 180 degrees.
1.4 EM IMAGE DENSE PREDICTION PROBLEM
In this experiments, we used two separate datasets for training and testing from
the ISBI 2013 challenge. Each dataset is a 3D stack of 100 sections from a serial
section scanning electron microscopy (SEM) of mouse cortex. The pixel resolution is
6x6x30 nm/pixel which covers a micro cube of approximately 6x6x3microns. Both
datasets have high x- and y-direction resolution whereas the resolution of z-direction
 5

is low. The neurites in the training stack have been manually delineated, generating a
corresponding label stack of 100 sections. The training stack contains 400 neurites
that have been labeled consistently across the 100 slices. Some neurites are split into
several segments in some slices while still required to preserve their unique label
across sections, which increases the complexity of the 3D segmentation task. The
labels of the testing stack are not available to challenge participants.
They formulated the 3D segmentation problem of 400 neurites in the training
stack as a single 2D segmentation problem. We built a pixel classifier that accepts
patches extracted from the raw input image to generate 2D probability maps. Each
resulting probability map indicates the probability of each pixel being either a
membrane (border) pixel or non-membrane (neurite). The probability maps have no
reference of which neurite a pixel belongs to, had it been identified as a non-
membrane pixel. The final 3D segmentation was obtained.
1.5 POST PROCESSING
Super-pixel level algorithms are commonly used as a building block in most
post-processing techniques for 2D and 3D segmentation tasks. They are used mainly
to generate an over-segmentation from probability maps or affinity graphs. Later,
another classifier is built on top of the results of the super-pixel level algorithms to
accurately merge some of the overly segmented regions. The key limitation of these
approaches is that they reduce the generality of the overall proposed techniques, since
they rely on hand-crafted features to build classifiers on top of super-pixel algorithms.
One of the fundamental advantages of the proposed method is the ability to learn
features from the data, hence their ability to generalize to many other datasets.It has
been shown before that relying heavily on the learned network while simplifying post-
processing could result in a dramatic increase in the speed of computations.While
maintaining the generalization of the proposed technique.
This model tends to reduce the predicted segment sizes due to the widening of
training label borders described. As a result, we applied a reverse transformation
which used a maximum kernel to increase segments sizes. The overall processing is
fast simple and minimal additional computation is required.
 6

CHAPTER 2
LITERATURE SURVEY
2.1 LITERATURE REVIEW-1: HYPERCOLUMNS FOR OBJECT
SEGMENTATION AND FINE-GRAINED LOCALIZATION, published by B.
Hariharan, P. Arbelaez, and R. Girshick, in IEEE Conf. Computer Vision
Pattern Recognition (CVPR), in the year 2015.
Recognition algorithms based on convolutional networks (CNNs) typically use the
output of the last layer as a feature representation. The methods/techniques used is
convolutional networks (CNN).
MERITS
The high resolution of the lower layers is obtained instead of up sample the
higher layers .
DEMERITS
The first problem is simultaneous detection and segmentation (SDS).
The Second problem deals with detecting an object and localizing its parts.
2.2 LITERATURE REVIEW-2: AN INTEGRATED REGION, BOUNDARY,
SHAPE- BASED ACTIVE CONTOUR FOR MULTIPLE OBJECT OVERLAP
RESOLUTION IN HISTOLOGICAL IMAGERY published by S. ALI AND A.
MADABHUSHI in the year 2011.
Synergistic boundary and region-based active contour model that incorporates shape
priors in a level set formulation with automated initialization based on watershed. The
methods/techniques used is Principle Component Analysis(PCA) .

MERITS

Segments all the overlapping and non overlapping objects within an image
simultaneously.
DEMERITS
Variability in digital slide appearance due to inconsistencies in histological
staining,
Poor image quality with tissue samples due to slide digitization
Tissue fixation.
 7

2.3 LITERATURE REVIEW-3:PARTITIONING HISTOPATHOLOGICAL

IMAGES: AN INTEGRATED FRAMEWORK FOR SUPERVISED COLOR-
TEXTURE SEGMENTATION AND CELL SPLITTING published by H.
KONG, M. GURCAN, AND K. BELKACEM-BOUSSAID, IEEE TRANS. MED.
IMAGING in the year 2011.

An efficient LFT extraction algorithm based on image shifting and image integral. For
the splitting part, given a connected component of the segmentation map, they initially
differentiate whether it is a touching-cell clump or a single non touching cell. The
methods/techniques used is local fourier transform.

MERITS
The advantage of LFT segmentation over the others cannot be fully reflected
by the marginal improvement if only based on the segmentation accuracy.
DEMERITS
Contours containing multiple overlapping objects pose.
 8

CHAPTER 3
SYSTEM METHODOLOGY
Systems methodology is the process of defining how the
information system should be built (i.e., physical system design), ensuring that the
information system is operational and used, ensuring theinformation system meets
quality standard .
3.1 METHODOLOGY OF SYSTEM IMPLEMENTATION

IMAGE IMAGE POOLING CONVOLUTION

ACQUISITION ENHANCEMENT

RESOLUTED DECONVOLUTION UNPOOLING

IMAGE

Figure 3.1 RESIDUAL DECONVOLUTIONAL NETWORK

3.1.1 IMAGE ACQUISITION

The first stage of any vision system is the image acquisition stage.After the image has
been obtained, various methods of processing can be applied to the image to perform
the many different vision tasks required .However, if the image has not been acquired
satisfactorily then the intended tasks may not be achievable, even with the aid of some
form of image enhancement.

Image acquisition considerations including currently available remotely sensed

data. Image characteristics in terms of spatial, spectral, radiometric, and temporal
resolutions. Preprocessing techniques such as geometric distortion removals,
atmospheric correction, algorithms, image registration, enhancement, masking, and
data transformations. Data reduction, fusion, and integration techniques. International
policies governing acquisition and distribution of remotely sensed data.
 9

3.1.2 IMAGE ENHANCEMENT

The common processing methods used to enhance the visual quality of the
image. Specifically, we will first look at contrast enhancement methods, and then at
spatial filtering methods that sharpen edges and remove much of the image blur. This
presents a short introduction to image processing and image filtering techniques. The
mathematical concepts of convolution and the kernel matrix are used to apply filters to
signals, to perform functions such as extracting edges and reducing unwanted noise.
The Sobel operator and Gaussian smoothing filter are implemented in MATLAB to
achieve the functions previously mentioned, and are evaluated on test images. The
effects of adding Gaussian and salt and pepper noise before filtering are then
presented as an approximation to signals that occur in real applications. Pre-
processing images before applying other filters is shown to produce improved results
when extracting edges from images with noise.
3.1.3 POOLING
In this work, we propose BOSSA NOVA, a novel representation for content-
based concept detection in images and videos, which enriches the Bag-of-Words
model. Relying on the quantization of highly discriminant local descriptors by a
codebook, and the aggregation of those quantized descriptors into a single pooled
feature vector, the Bag-of-Words model has emerged as the most promising approach
for concept detection on visual documents. BOSSA NOVA enhances that
representation by keeping a histogram of distances between the descriptors found in
the image and those in the codebook, preserving thus important information about the
distribution of the local descriptors around each code word. Contrarily to other
approaches found in the literature, the non-parametric histogram representation is
compact and simple to computer. BOSSA NOVA also shows good results in the
challenging real-world application of detection.
3.1.4 CONVOLUTION
This aims at approximating and covering the optimal local sparse structure of a
convolutional vision network by readily available locally dense components.
Assuming translation invariance suggests the use of convolutional building blocks:
finding the optimal local model and to repeat it spatially. A layer-by layer construction
 10

where one should analyze the correlation statistics of the preceding layer and cluster
them into groups of units with high correlation. In the layers close to the input
correlated units would concentrate in local regions. These reduction layers also
include the use of rectified linear activation which makes them dual-purpose by
increasing their representation power.
3.1.5 DECONVOLUTION
Image acquired on a digital fluorescence microscope can be deconvolved, and
several new applications are being developed that apply deconvolution techniques to
transmitted light images collected under a variety of contrast enhancing strategies.
Among the most suitable subjects for improvement by deconvolution are three-
dimensional montages constructed from a series of optical sections. The basic
concepts surrounding acquisition of serial optical sections for deconvolution analysis
are presented. During deconvolution analysis, the entire series of optical sections is
analyzed to create a three-dimensional montage.
As a technique, deconvolution is often suggested as a good alternative to the
confocal microscope. This is not strictly true because images acquired using a pinhole
aperture in a confocal microscope can also be analyzed by deconvolution techniques.
3.2 SOURCES OF IMAGE DEGRADATION
The sources of image degradation can be divided into four independent
phenomena: noise, scatter, glare, and blur. The principal task of deconvolution
methods is to remove the out-of-focus blur from images. Deconvolution algorithms
can and do remove noise, but this is a relatively simple aspect of their general overall
performance.
Blur is described by a non random spreading of light that occurs by passage
through the imaging system. The most significant source of blur is diffraction, and an
image whose resolution is limited only by blur is considered to be diffraction-limited.
This represents an intrinsic limit of any imaging system and is the determining factor
in assessing the resolution limit of an optical system. Optical theory proposes
sophisticated models of blur, which can be applied, with the assistance of modern
high-speed computers, to digital images captured in the optical microscope. This is the
basis for deconvolution. It is precisely this independence of optical blur from other
 11

types of degradation that enables the possibility of blur removal by deconvolution

techniques.
The interaction of light with matter is the primary physical origin of scatter,
glare, and blur. However, the composition and arrangement of molecules in a given
material has its own particular set of optical properties. For the purposes of
deconvolution, what distinguish scatter, glare, and blur are the location where they
occur and the possibility of generating a mathematical model for these phenomena.
Because scatter is a localized, irregular phenomenon occurring in the specimen, it has
proven difficult to model. This model renders it possible to reverse the blurring
process mathematically, and deconvolution employs this model to reverse or remove
blur.

3.3 THE POINT SPREAD FUNCTION

The model for blur that has evolved in theoretical optics is based on the concept
of a three-dimensional point spread function (PSF). This concept is of fundamental
importance to deconvolution and should be clearly understood in order to avoid
imaging artifacts. The point spread function is based on an infinitely small point
source of light originating in the specimen (object) space. Because the microscope
imaging system collects only a fraction of the light emitted by this point, it cannot
focus the light into a perfect three-dimensional image of the point. Instead, the point
appears widened and spread into a three-dimensional diffraction pattern. Thus, the
point spread function is formally defined as the three-dimensional diffraction pattern
generated by an ideal point source of light.
3.4 DEEP RESIDUAL LEARNING
Deeper neural networks are more difficult to train. We present a residual
learning framework to ease the training of networks that are substantially deeper than
those used previously. We explicitly reformulate the layers as learning residual
functions with reference to the layer inputs, instead of learning unreferenced
functions. Introduction to deep convolutional neural networks have led to a series of
breakthroughs for image classification. Deep networks naturally integrate
low/mid/high level features and classifiers in an end-to-end multilayer fashion, and
 12

the levels of features can be enriched by the number of stacked layers (depth).
Recent evidence reveals that network depth is of crucial importance, and the leading
results on the challenging Image Net dataset all exploit very deep modes, with a
depth of sixteen to thirty.
3.5 IMAGE RESOLUTION

Image resolution can be measured in various ways. Resolution quantifies how

close lines can be to each other and still be visibly resolved. Resolution units can be
tied to physical sizes (e.g. lines per mm, lines per inch), to the overall size of a picture
or to angular subtense. Line pairs are often used instead of lines; a line pair comprises
a dark line and an adjacent light line. A line is either a dark line or a light line. A
resolution of 10 lines per millimeter means 5 dark lines alternating with 5 light lines,
or 5 line pairs per millimeter (5 LP/mm). Photographic lens and film resolution are
most often quoted in line pairs per millimeter.

3.6 RESOLUTION OF DIGITAL IMAGES

The resolution of digital cameras can be described in many different ways.

3.6.1Pixel resolution

3.6.2Spatial resolution

3.6.3Spectral resolution

3.6.4Temporal resolution

3.6.5Radiometric resolution

Resolution in various media

3.6.1 Pixel resolution

The term resolution is often considered equivalent to pixel count in digital

imaging, though international standards in the digital camera field specify it should
instead be called "Number of Total Pixels" in relation to image sensors,and as
"Number of Recorded Pixels" for what is fully captured. Hence, JCIA & CIPA
suggest notation such as "Number of Recorded Pixels 1000 x 1500". According to the
 13

same standards, the "Number of Effective Pixels" that an image sensor or digital
camera has is the count of pixel sensors that contribute to the final image (including
pixels not in said image but nevertheless support the image filtering process), as
opposed to the number of total pixels, which includes unused or light-shielded pixels
around the edges.The count of pixels is not a real measure of the resolution of digital
camera images, because color image sensors are typically set up to alternate color
filter types over the light sensitive individual pixel sensors. Digital images ultimately
require a red, green, and blue value for each pixel to be displayed or printed, but one
individual pixel in the image sensor will only supply one of those three pieces of
information. The image has to be interpolated or decimated to produce all three colors
for each output pixel.

Figure 3.2 COMPARISON BETWEEN THE SEGMENTATIONS

 14

3.6.2 Spatial resolution

Spatial resolution in radiology refers to the ability of the imaging modality to

differentiate two objects. Low spatial resolution techniques will be unable to
differentiate between two objects that are relatively close together.

The measure of how closely lines can be resolved in an image is called spatial
resolution, and it depends on properties of the system creating the image, not just the
pixel resolution in pixels per inch. For practical purposes the clarity of the image is
decided by its spatial resolution, not the number of pixels in an image. In effect,
spatial resolution refers to the number of independent pixel values per unit length.

The spatial resolution of computer monitors is generally 72 to 100 lines per

inch, corresponding to pixel resolutions of 72 to 100 PPI. With scanners, optical
resolution is sometimes used to distinguish spatial resolution from the number of
pixels per inch.

In remote sensing, spatial resolution is typically limited by diffraction, as well

as by aberrations, imperfect focus, and atmospheric distortion. The ground sample
distance (GSD) of an image, the pixel spacing on the Earth's surface, is typically
considerably smaller than the resolvable spot size.

3.6.3 Spectral resolution

Spectral resolution is the ability to resolve spectral features and bands into their
separate components. Color images distinguish light of different spectra. Multispectral
images resolve even finer differences of spectrum or wavelength than is needed to
reproduce color. That is, multispectral images have higher spectral resolution than
normal color images. Spectral resolution is the ability to resolve spectral features and
bands into their separate components. The spectral resolution required by the analyst
or researcher depends upon the application involved.

3.6.4 Temporal resolution

Temporal resolution (TR) refers to the precision of a measurement with respect

to time. Often there is a trade-off between temporal resolution of a measurement and
its spatial resolution due to uncertainty principle which is an inherent property
 15

of Fourier transform. The time resolution used for movies is usually 24 to 48 frames
per second (frames/s).
3.6.5 RADIOMETRIC RESOLUTION
Radiometric resolution determines how finely a system can represent or
distinguish differences of intensity, and is usually expressed as a number of levels or a
number of bits, for example 8 bits or 256 levels that is typical of computer image files.
The higher the radiometric resolution, the better subtle differences of intensity
or reflectivity can be represented, at least in theory. In practice, the effective
radiometric resolution is typically limited by the noise level, rather than by the number
of bits of representation.

Electron microscopy is the most thoroughly developed approach for the dense
reconstruction of neural circuits. Because the wavelength of an electron under 10 kV
accelerating voltage is 10 pm, imaging with electrons can (in principle) reach spatial
resolutions in the sub-nanometer to nanometer range, more than sufficient to trace the
finest morphological sub-structures of neurons. The basic strategy employed by the
current EM approaches is to obtain many morphological images of thin tissue
sections, segmenting those images into regions corresponding to distinct neuronal
processes, and tracing individual axons from one image to another. Because axons are
thin, long, and densely interspersed with other neuronal processes, tracing their entire
lengths is a challenge.

3.7 EM DATA ACQUISITION: BASIC PROPERTIES

The physical constraints on large-scale electron microscopy for neural circuit

reconstruction were first studied in the 1980s, following the acquisition of the C. The
electron dose per pixel is one property which constrains the resolution and speed of an
imaging system. An exemplary recent connectomics study used roughly 14 electrons
per nm2, or 3812 electrons per 16.5nm16.5nm pixel. Due to Poisson counting
statistics, the fractional error in the estimate of the stain density in a voxel goes
roughly as 1/PN, where N is the number of electrons passing through the voxel, so the
analog bit precision in that study was roughly log2p3812 = 6 bits at each pixel.
 16

Merkle used the number of electrons per voxel, the number of parallel electron
microscopes available, and the total project time to estimate the beam current per
microscope: imaging a whole human brain in 3 years at 10nm10nm10nm voxel
size, with 7-bit precision and 1000 parallel microscopes, would give 0.1 mA beam
current, comparable with that of electron microscopes circa 1989.

TEM Vs SEM Transmission electron microscopy (TEM) involves passing

electrons through a sample, whereas scanning electron microscopy (SEM) relies on
back-scattered or secondary electrons emitted from the samples surface. High-
resolution EM analysis was originally limited to transmission electron microscopy,
which necessitated the use of ultra-thin (< 100 nm), grid-suspended sections to allow
electron penetration through the slice. Although TEM sections cannot easily be made
thinner than a few tens of nanometers, resolution can be improved by tilting the
sample and performing a tomographic reconstruction; only a handful of additional tilts
are required if sparse reconstruction techniques are used. Indeed, the first proposals
for whole-mouse-brain electron microscopy circuit tracing assumed a TEM
tomography strategy. Scaling up TEM Unfortunately, large-scale automation of
transmission electron microscopy has been difficult in practice due to the need to
isolate fragile ultra-thin sections which can be penetrated by the electron beam. TEM
is still used today, at rates approaching 10 megapixels per second using camera arrays,
but in a recent study, 30 of 4 000 thin sections were lost in the preparation process.
Thus, improvements in TEM sample handling are needed to trace connectivity at
whole-mouse-brain scale, and we focus on scanning electron microscopy techniques
below. Improvements in high-throughput, high-reliability automated TEM sample
preparation, coupled with camera arrays, could make TEM viable for large-scale
circuit reconstruction.Another major challenge to whole brain imaging will be
minimizing the material loss from vibratome section to vibratome section, and from
the sub-sectioning of the brain either before or after embedding
 17

3.8 METHODOLOGY OF RESIDUAL DECONVOLUTION

Figure 3.3 ARCHITECTURE OF THE RESIDUAL DECONVOLUTIONAL

NETWORK (RDN).
The network consists of two pathways, namely context growing path and
resolution preserving path. All convolution and deconvolution layers in the encoding
and decoding stages are of size 3x3. A kernel of size 1x1 is used to implement the
projection mappings. Max pooling is used to reduce the feature map sizes in the
convolution stage while unpooling is used to restore the original size in the decoding
stage. Our model implementation was based on the publicly available MATLAB.
RDN using back propagation with stochastic gradient decent. The mini batch
size used was 15 as the dense prediction requires a lot of memory. However, the
network requires roughly 15k iterations to achieve its full potential due to the
existence of residual paths which speeds up the computations. We used a momentum
of 0.9 and weight decay of 0.005. We started with a base learning rate of 102 with a
polynomial decay. Random initialization was used for all learned layers. The
experiments were carried out on an NVIDIA K80 GPU machine, taking roughly 2
days of training. To improve the robustness of the resulting probability maps, we
applied 8 variations to the testing images before passing them down through the
network. A reverse transformation was then applied to each resulting probability map
before taking the average across all variations. The transformations were
combinations of horizontal and vertical mirroring, and/or rotations by +90, -90 and
180 degrees.
 18

CHAPTER 4
SOFTWARE DESCRIPTION

4.1 GENERAL

MATLAB (matrix laboratory) is a multi-paradigm numerical

computing environment and fourth-generation programming language. A proprietary
programming language developed by MathWorks, MATLAB
allows matrix manipulations, plotting of functions and data, implementation
of algorithms, creation of user interfaces, and interfacing with programs written in
other languages, including C, C++, C#, Java, Fortran and Python.

Although MATLAB is intended primarily for numerical computing, an

optional toolbox uses the MuPAD symbolic engine, allowing access to symbolic
computing abilities. An additional package, Simulink, adds graphical multi-domain
simulation and model-based design for dynamic and embedded systems.

In 2004, MATLAB had around one million users across industry and
academia. MATLAB users come from various backgrounds of engineering, science,
and economics.

MATLAB was first adopted by researchers and practitioners in control

engineering, Little's specialty, but quickly spread to many other domains. It is now
also used in education, in particular the teaching of linear algebra, numerical analysis,
and is popular amongst scientists involved in image processing. The MATLAB
application is built around the MATLAB scripting language. Common usage of the
MATLAB application involves using the Command Window as an interactive
mathematical shell or executing text files containing MATLAB code.

4.2 FEATURES OF MATLAB

High-level language for numerical computation, visualization, and application

development.

Interactive environment for iterative exploration, design, and problem solving.

 19

Mathematical functions for linear algebra, statistics, Fourier analysis,

filtering,optimization, numerical integration, and solving ordinary differential
equations .

Built-in graphics for visualizing data and tools for creating custom plots

Development tools for improving code quality and maintainability and

maximizing performance .

Tools for building applications with custom graphical interfaces.

Functions for integrating MATLAB based algorithms with external

applications and languages such as C, Java, .NET, and Microsoft Excel

4.3 INTERFACING WITH OTHER LANGUAGES

MATLAB can call functions and subroutines written in the programming

languages C or Fortran. A wrapper function is created allowing MATLAB data types
to be passed and returned. The dynamically loadable object files created by compiling
such functions are termed "MEX-files"(for MATLAB executable). Since 2014
increasing two-way interfacing with Python is being added.

Libraries written in Perl, Java, ActiveX or .NET can be directly called from
MATLAB, and many MATLAB libraries (for example XML or SQL support) are
implemented as wrappers around Java or ActiveX libraries. Calling MATLAB from
Java is more complicated, but can be done with a MATLAB toolbox which is sold
separately by MathWorks, or using an undocumented mechanism called JMI (Java-to-
MATLAB Interface), (which should not be confused with the unrelated Java Metadata
Interface that is also called JMI). Official MATLAB API for Java was added in 2016.

As alternatives to the MuPAD based Symbolic Math Toolbox available from

MathWorks, MATLAB can be connected to Maple or Mathematica.Libraries also
exist to import and export MathML.

Development environment

Startup accelerator for faster MATLAB startup on windows, especially on

windows XP, and for network installations.
 20

Spreadsheet import tool that provides more options for selecting and loading
mixed textual and numeric data.
Readability and navigation improvements to warning and error message in the
MATLAB command window.
Automatic variable and function renaming in the MATLAB editor.

Developing algorithms and application

MATLAB provides a high level language development tools that let you
quickly develop and analyze your algorithms and applications.

The MATLAB language

MATLAB is a programming language developed by MathWorks. It started out

as a matrix programming language where linear algebra programming was simple. It
can be run both under interactive sessions and as a batch job. Alternatives to
MATLAB exist including open source software packages.

The MATLAB language provides native support for the vector and matrix
operations that are fundamental to solving engineering and scientific problems,

enabling fast development and execution. MATLAB provides features of traditional

programming languages, including flow control, error handling, and object-oriented
programming (OOP). You can use fundamental data types or advanced data
structures, or you can define custom data types.

MATLAB add-on products provide built-in algorithms for signal processing

and communications, image and video processing, control systems, and many other
domains. By combining these algorithms with your own, you can build complex
programs and applications.

MATLAB Editor

Matlab commands are executed either at the command prompt or by running

scripts or functions, which can be created and edited with the built in editor. To launch
the editor, if it is not already open, type edit or edit filename.
 21

Code Analyzer

The Code Analyzer checks your code for problems and recommends
modifications to maximize performance and maintainability.

MATLAB Profiler

Records the time spent executing each line of code.

Directory Reports

Scan all the files in the directory and report on the code efficiency, file
differences, file dependencies and code coverage.

Designing Graphical User Interfaces

By using this interactive tool GUIDE(graphical User Interface Development

Environment) to layout, design, and edit user interfaces. GUIDE lets you include list
boxes, pulldown menus,push buttons, radio buttons, and sliders, as well as MATLAB
plots and Microsoft ActiveX controls. Alternatively, you can create GUIs
programmatically using MATLAB functions.

4.4 ANALYSING AND ACCESSING DATA

MATLAB supports the entire data analysis process, from acquiring data from
external device and database, through preprocessing, visualization, and numerical
analysis, to producing presentation-quality output.

Data Analysis

MATLAB provides interactive tool and command-line functions for data

analysis operations, including:

Interpolating and decimating

Extracting sections of data, scaling, and averaging
Thresholding and smoothing
Correlation,fourier analysis and filtering
I D peak, valley, and zero findng
Basic statistics and curve fitting
Matrix analysis
 22

Data access

MATLAB is an efficient platform for accessing data from files, other

applications, databases, external devices. You can read data from popular file format,
such as Microsoft excel; ASCII text or binary files; images,sound, and video files; and
scientific files,such as HDF and HDF5. Low-level binary file I/O functions let you
work with data files in any format.

4.5 PERFORMING NUMERIC COMPUTATION

MATLAB contains mathematical, statistical, and engineering functions to

support all common engineering and science operations. These function, developed by
experts in mathematics, are the foundation of the MATLAB language. The core math
functions use the LAPACK and BLAS linear algebra subroutine libraries and the
FFTW discrete fourier transform library. Because thers processor-dependent libraries
are optimized to the different platforms that MATLAB support, these execute faster
than the equivalent c or c++ code.

MATLAB provides the following types of functions for performing

mathematical operations and analyzing data:

Matrix manipulation and linear algebra.

Polynomials and interpolation
Fourier analysis and filtering.
Data analysis and statistics.
Optimization and numerical integration.
Ordinary differential equations(ODEs)
Partial differential equation(PDEs)
Sparse matrix operations.

MATLAB can perform arithmetic on a wide range of data types, including

doubles, singles, and integers

ADVANTAGE

Its basic data element is the matrix. A simple integer is considered an

matrix of one row and one column. Several mathematical operations
 23

that work on arrays or matrices are built-in to the Matlab environment.

For example, cross-products, dot-products, determinants, inverse
matrices.
Vectorized operations. Adding two arrays together needs only one
command, instead of a for or while loop.
The graphical output is optimized for interaction. You can plot your data
very easily, and then change colors, sizes, scales, etc, by using the
graphical interactive tools.
Matlabs functionality can be greatly expanded by the addition of
toolboxes. These are sets of specific functions that provided more
specialized functionality. Ex: Excel link allows data to be written in a
format recognized by Excel, Statistics Toolbox allows more specialized
statistical manipulation of data (Anova, Basic Fits, etc)
DISADVANTAGE

It uses a large amount of memory and on slow computers it is very hard to use.
It sits on top of Windows, getting as much CPU time as Windows allows it to
have. This makes real-time applications very complicated.
 24

CHAPTER 5
RESULTS
In this, the proposed model is very effective.Achieving the conflicting
goals in dense output prediction.Full-resolution predictions and including
sufficient contextual information.
5.1 Input Image

Figure 5.1 Input Image.

The above figure shows the RAW image of Electron Microscopy image Segmentation
which has 1024x768 pixels
 25

5.2 Map Generated Image

Figure 5.2 Map Generated Image

The above figure shows the Colour Map Generation image. This uses
the color function to display a 16-by-16 matrix whose elements vary from 0 to
255 in a rectilinear spiral. The hsv color map starts with red in the center, then
passes through yellow, green, cyan, blue, and magenta before returning to red
at the outside end of the spiral. A color map is an m-by-3 matrix of real
numbers between 0.0 and 1.0. Each row is an RGB vector that defines one
color. The kth row of the color map defines the kth color,
where map(k,:) = [r(k) g(k) b(k)]) specifies the intensity of red, green, and blue.
 26

5.3 Neuron Segmentated Image

Figure 5.3 Neuron Segmentated Image

The above figure Shows 3D neuron Segmentation in EM Image. Segmentation

generated by our RDN and watershed. An illustration of the effect of resolution-
preserving paths on the final segmentation. We show the results generated for testing
slices number by our RDN with and without resolution-preserving paths on the left
and right respectively. Colored boxed have been placed on the compared segments.
The above figure is easy semi-automatic tracing of neurons or other tube-like
structures (e.g., blood vessels) through 3D image stacks.
 27

5.4 Signal Image Segmentation Of EM Image

Figure 5.4 Signal Image Segmentation Of EM Image

The above figure shows the Identification of electron microscopy disease using signal
image segmentation specific for one-dimensional signals and does not trivially
transfer to higher-dimensional images. Nevertheless, this general idea has inspired
several other authors to investigate coarse-to-fine schemes for image segmentation.
 28

5.5 Color Image Segmentation Of EM Image

Figure 5.5 Color Image Segmentation Of EM Image

The above figure shows the Identification of electron microscopy disease using color
image EM image segmentation MRI image.
 29

CHAPTER 6
CONCLUSION
We proposed a computational technique for EM image segmentation by
obtaining dense predictions that combined multi-scale contextual reasoning along with
full-resolution reconstruction. This approach achieved promising performance while
relying on minimum post-processing. They expect better probability maps be
generated with improvement in the z-dimension resolution of the data provided. A
limitation in the underlying post-processing techniques is that it requires a specific
parameter to control the level of over/under segmentation. Automatic tuning of this
parameter is not straightforward and can be data-specific even if it is tuned on the
validation dataset.They used semi automated visualization of the segmentations to
overcome this limitation. Nonetheless, our method can be paired with any other post-
processing techniques, leading to an overall performance improvement.

They did not use hand-crafted features either in the network training or post-
processing stages. Consequently, we demonstrated the ability of this model to
generalize by applying it to multiple datasets obtained from different species. This
method achieved consistently promising performance.They believe this method can
generalize well to other similar dense output prediction task.
 30

REFERENCES

[1] M. N. Gurcan, L. E. Boucheron, A. Can, A. Madabhushi, N. M. Rajpoot,and B.

Yener, Histopathological image analysis: A review, IEEE Rev.Biomed. Eng., vol. 2,
pp. 147171, 2009.
[2] G. Lin, U. Adiga, K. Olson, J. F. Guzowski, C. A. Barnes, andB. Roysam, A
hybrid 3D watershed algorithm incorporating gradientcues and object models for
automatic segmentation of nuclei in confocal image stacks, Cytometry A, vol. 56, no.
1, pp. 2336, 2003.
[3] X. Yang, H. Li, and X. Zhou, Nuclei segmentation using
markercontrolledwatershed, tracking using mean-shift, and kalman filter in time-lapse
microscopy, IEEE Trans. Ciruits Syst., vol. 53, no. 11, pp.24052414, 2006.
[4] X. Zhou, K.-Y. Liu, P. Bradley, N. Perrimon, and S. T. C. Wong,
Towards automated cellular image segmentation for RNAi genomewide
screening, in Int. Conf. Med. Image Comput. Comput. Assist.
Intervent. (MICCAI), vol. 3749, 2005, pp. 885892.
[5] J. Cheng and J. C. Rajapakse, Segmentation of clustered nuclei with
shape markers and marking function, IEEE Trans. Biomed. Eng.,
vol. 56, no. 3, pp. 741748, 2009.
[6] P. Soille, Morphological Image Analysis: Principles and Applications.
Berlin, Germany: Springer-Verlag, 1999.
[7] O. Schmitt and M. Hasse, Radial symmetries based decomposition
of cell clusters in binary and gray level images, Pattern Recognition,
vol. 41, no. 6, pp. 19051923, 2008.
[8] C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov,
D. Erhan, V. Vanhoucke, and A. Rabinovich, Going deeper with
convolutions, in Proceedings of the IEEE Conference on Computer
 31

Vision and Pattern Recognition, 2015, pp. 19.

[9] K. He, X. Zhang, S. Ren, and J. Sun, Deep residual learning
for image recognition, in Proceedings of the IEEE Conference on
Computer Vision and Pattern Recognition, 2016.
[10] P. Sermanet, D. Eigen, X. Zhang, M. Mathieu, R. Fergus, and Y. Le-
Cun, OverFeat: Integrated recognition, localization and detection
using convolutional networks, in Proceedings of the International
Conference on Learning Representations, April 2014.
[11] C. Farabet, C. Couprie, L. Najman, and Y. LeCun, Learning hierarchical
features for scene labeling, IEEE Transactions on Pattern
Analysis andMachine Intelligence, vol. 35, no. 8, pp. 19151929, 2013.
[12] H. Noh, S. Hong, and B. Han, Learning deconvolution network
for semantic segmentation, in Proceedings of the IEEE International
Conference on Computer Vision, 2015, pp. 15201528.
[13] J. Long, E. Shelhamer, and T. Darrell, Fully convolutional networks
for semantic segmentation, in Proceedings of the IEEE
Conference on Computer Vision and Pattern Recognition, 2015, pp.
34313440.
[14] H. Su, F. Xing, X. Kong, Y. Xie, S. Zhang, and L. Yang, Robust
cell detection and segmentation in histopathological images using
sparse reconstruction and stacked denoising autoencoders, in
International Conference on Medical Image Computing and Computer-
Assisted Intervention. Springer, 2015, pp. 383390.
[15] O. Ronneberger, P. Fischer, and T. Brox, U-net: Convolutional
networks for biomedical image segmentation, in International