CIS 311: Neural Networks

Implementing Kohonen Networks

1. Normalization

The training of Kohonen neural networks involves normalization of the input vectors and the weights.

The normalization of a vector x = ( x1,x2,...,xd ) is made by multiplying its components with a positive number c.

The formula for computing the positive normalizer c is:

c = 1.0 / sqrt( x12 + x22 + ... + xn2 )

The input vector x = ( 1.2, -2.3, 3.4, -4.55, 5.6 ) is normalized as follows:

c = 1.0 / sqrt( 1.22 + ( -2.3 )2 + 3.42 + ( -4.55 )2 + 5.62 ) = 0.1192

x = ( 1.2* 0.1192, (-2.3)*0.1192, 3.4*0.1192, (-4.55)*0.1192, 5.6*0.1192 )

x = ( 0.1430, -0.2742, 0.4053, -0.5424, 0.6675 )

2. Training

Suppose that a self-organizing neural network with two neurons in the Kohonen layer is given. Each neuron has 4 inputs.
The initial weight vectors are: w1 = ( -0.4, 1.2, 2, 1.3 ), and w2 = ( -1.5, 2, -0.8, 1 ).

These initial weights are normalized before training:

c1 = 1.0 / sqrt(( -0.4 )2 + 1.22 + 22 + 1.32 ) = 0.3704

w1 = ( -0.1482, 0.4445, 0.7408, 0.4815 )

c2 = 1.0 / sqrt(( -1.5 )2 + 22 + ( -0.8 )2 + 12 ) = 0.3560

w2 = ( -0.5340, 0.7120, -0.2848, 0.3560 )

Let the following the input vector be given: ( x1, x2, x3, x4 ) = ( 0.4, 0.3, 0.2, 0.1 )

The outputs of the neurons are computed to determine the index of the largest component as follows:


s = Wxt = | -0.1482, 0.4445, 0.7408, 0.4815  | * ( 0.4, 0.3, 0.2, 0.1 )t = |  0.2704 | 
                | -0.5340, 0.7120, -0.2848, 0.3560 |	  	                | -0.0214 |

Therefore the cluster index is: s( x ) = 1

The training of the Kohonen layer, using learning rate h = 0.15, updates the weights of the first neuron as follows:

w1 = ( -0.1482, 0.4445, 0.7408, 0.4815 ) + 0.15 * [( 0.4, 0.3, 0.2, 0.1 ) - ( -0.1482, 0.4445, 0.7408, 0.4815 )]

w1 = ( -0.1482, 0.4445, 0.7408, 0.4815 ) + 0.15 * ( 0.5482, -0.1445, -0.5408, -0.3815 )

w1 = ( -0.0660, 0.4228, 0.6597, 0.4243 )