A self-organizing map (SOM) or self-organizing feature map (SOFM) is a type of artificial

neural network (ANN) that is trained using unsuperised learning to produce a low-
dimensional (typically two-dimensional)! discretized representation of the input space of
the training samples! called a map" Self-organizing maps are different from other artificial
neural networks in the sense that they use a neigh#orhood function to presere the
topological properties of the input space"
A self-organizing map showing $"S" %ongress oting patterns isualized in Synapse"
&he first two #o'es show clustering and distances while the remaining ones show the
component planes" (ed means a yes ote while #lue means a no ote in the component
planes (e'cept the party component where red is (epu#lican and #lue is )emocratic)"
&his makes SOMs useful for isualizing low-dimensional iews of high-dimensional data!
akin to multidimensional scaling" &he model was first descri#ed as an artificial neural
network #y the Finnish professor &euo *ohonen! and is sometimes called a *ohonen
map or network"+,-+.-
/ike most artificial neural networks! SOMs operate in two modes0 training and mapping"
1&raining1 #uilds the map using input e'amples (a competitie process! also called
ector 2uantization)! while 1mapping1 automatically classifies a new input ector"
A self-organizing map consists of components called nodes or neurons" Associated with
each node is a weight ector of the same dimension as the input data ectors and a
position in the map space" &he usual arrangement of nodes is a two-dimensional regular
spacing in a he'agonal or rectangular grid" &he self-organizing map descri#es a
mapping from a higher-dimensional input space to a lower-dimensional map space" &he
procedure for placing a ector from data space onto the map is to find the node with the
closest (smallest distance metric) weight ector to the data space ector"
3hile it is typical to consider this type of network structure as related to feedforward
networks where the nodes are isualized as #eing attached! this type of architecture is
fundamentally different in arrangement and motiation"
$seful e'tensions include using toroidal grids where opposite edges are connected and
using large num#ers of nodes"
4t has #een shown that while self-organizing maps with a small num#er of nodes #ehae
in a way that is similar to *-means! larger self-organizing maps rearrange data in a way
that is fundamentally topological in character" +5-
4t is also common to use the $-Matri'"+6- &he $-Matri' alue of a particular node is the
aerage distance #etween the node and its closest neigh#ors"+7- 4n a s2uare grid! for
instance! we might consider the closest 6 or 8 nodes (the 9on Neumann and Moore
neigh#orhoods! respectiely)! or si' nodes in a he'agonal grid"
/arge SOMs display emergent properties" 4n maps consisting of thousands of nodes! it is
possi#le to perform cluster operations on the map itself"+:-
&he goal of learning in the self-organizing map is to cause different parts of the network
to respond similarly to certain input patterns" &his is partly motiated #y how isual!
auditory or other sensory information is handled in separate parts of the cere#ral corte'
in the human #rain"+;-
An illustration of the training of a self-organizing map" &he #lue #lo# is the distri#ution of
the training data! and the small white disc is the current training datum drawn from that
distri#ution" At first (left) the SOM nodes are ar#itrarily positioned in the data space" &he
node (highlighted in yellow) which is nearest to the training datum is selected" 4t is
moed towards the training datum! as (to a lesser e'tent) are its neigh#ors on the grid"
After many iterations the grid tends to appro'imate the data distri#ution (right)"
&he weights of the neurons are initialized either to small random alues or sampled
eenly from the su#space spanned #y the two largest principal component eigenectors"
3ith the latter alternatie! learning is much faster #ecause the initial weights already
gie a good appro'imation of SOM weights"+8-
&he network must #e fed a large num#er of e'ample ectors that represent! as close as
possi#le! the kinds of ectors e'pected during mapping" &he e'amples are usually
administered seeral times as iterations"
&he training utilizes competitie learning" 3hen a training e'ample is fed to the network!
its <uclidean distance to all weight ectors is computed" &he neuron whose weight
ector is most similar to the input is called the #est matching unit (=M$)" &he weights of
the =M$ and neurons close to it in the SOM lattice are ad>usted towards the input
ector" &he magnitude of the change decreases with time and with distance (within the
lattice) from the =M$" &he update formula for a neuron with weight ector 3(s) is
3(s ? ,) @ 3(s) ? A(u! ! s) B(s)()(t) - 3(s))!
where s is the step inde'! t an inde' into the training sample! u is the inde' of the =M$
for )(t)! B(s) is a monotonically decreasing learning coefficient and )(t) is the input
ectorC is assumed to isit all neurons for eery alue of s and t"+D- )epending on the
implementations! t can scan the training data set systematically (t is E! ,! ."""&-,! then
repeat! & #eing the training sampleFs size)! #e randomly drawn from the data set
(#ootstrap sampling)! or implement some other sampling method (such as >ackknifing)"
&he neigh#orhood function A(u! ! s) depends on the lattice distance #etween the =M$
(neuron u) and neuron " 4n the simplest form it is , for all neurons close enough to =M$
and E for others! #ut a Gaussian function is a common choice! too" (egardless of the
functional form! the neigh#orhood function shrinks with time"+;- At the #eginning when
the neigh#orhood is #road! the self-organizing takes place on the glo#al scale" 3hen the
neigh#orhood has shrunk to >ust a couple of neurons! the weights are conerging to local
estimates" 4n some implementations the learning coefficient B and the neigh#orhood
function A decrease steadily with increasing s! in others (in particular those where t
scans the training data set) they decrease in step-wise fashion! once eery & steps"
&his process is repeated for each input ector for a (usually large) num#er of cycles H"
&he network winds up associating output nodes with groups or patterns in the input data
set" 4f these patterns can #e named! the names can #e attached to the associated nodes
in the trained net"
)uring mapping! there will #e one single winning neuron0 the neuron whose weight
ector lies closest to the input ector" &his can #e simply determined #y calculating the
<uclidean distance #etween input ector and weight ector"
3hile representing input data as ectors has #een emphasized in this article! it should
#e noted that any kind of o#>ect which can #e represented digitally! which has an
appropriate distance measure associated with it! and in which the necessary operations
for training are possi#le can #e used to construct a self-organizing map" &his includes
matrices! continuous functions or een other self-organizing maps"
%onsider an nIm array of nodes! each of which contains a weight ector and is aware of
its location in the array" <ach weight ector is of the same dimension as the nodeFs input
ector" &he weights may initially #e set to random alues"
Now we need input to feed the map J&he generated map and the gien input e'ist in
separate su#spaces" 3e will create three ectors to represent colors" %olors can #e
represented #y their red! green! and #lue components" %onse2uently our input ectors
will hae three components! each corresponding to a color space" &he input ectors will
#e0
( @ K.77! E! EL
G @ KE! .77! EL
= @ KE! E! .77L
&he color training ector data sets used in SOM0
three%olors @ +.77! E! E-! +E! .77! E-! +E! E! .77-
eight%olors @ +E! E! E-! +.77! E! E-! +E! .77! E-! +E! E! .77-! +.77! .77! E-! +E! .77! .77-!
+.77! E! .77-! +.77! .77! .77-
&he data ectors should prefera#ly #e normalized (ector length is e2ual to one) #efore
training the SOM"
Self organizing map of FisherFs 4ris flower data"
Neurons (6EI6E s2uare grid) are trained for .7E iterations with a learning rate of E",
using the normalized 4ris flower data set which has four-dimensional data ectors"
Shown are0 a color image formed #y the first three dimensions of the four-dimensional
SOM weight ectors (top left)! a pseudo-color image of the magnitude of the SOM
weight ectors (top right)! a $-Matri' (<uclidean distance #etween weight ectors of
neigh#oring cells) of the SOM (#ottom left)! and an oerlay of data points (red0 4" setosa!
green0 4" ersicolor and #lue0 4" irginica) on the $-Matri' #ased on the minimum
<uclidean distance #etween data ectors and SOM weight ectors (#ottom right)"