CIS 311: Neural Networks

Implementing Radial - Basis Function Networks

1. The XOR Example

Consider the XOR example:

(x1,x2) | Y
------------
 (0,0)  | 0
 (0,1)  | 1
 (1,0)  | 1
 (1,1)  | 0

This XOR example can be learned by an RBF network with the following structure and characteristics:

- two hidden nodes and one output node;

- the output unit has a bias term b, that is the mapping is: F( x ) = Si=12 wi exp( - || x - xi ||2 / 2si2 ) + b

Solution: Training of this RBF to learn the XOE examples involves the following steps

----------------------------------------------------------------------------------------------------------------

Initialization:

- the network structure has two basis functions j1 and j2

j1= exp( - || x - x1 ||2 ) and j2 = exp( - || x - x2 ||2 )

- the centers are: x1 = (1,1) and x2 = (0,0)

- the basis are assumed to be one: s12 = 1 and s22 = 1

-------------------------------------------------------------------------------------------------------

Training:

j11 = exp( -(( 0-1 )2+( 0-1 )2 )) = exp( -2 ) = 0.1353

j21 = exp( -(( 0-1 )2+( 1-1 )2 )) = exp( -1 ) = 0.3678

j31 = exp( -(( 1-1 )2+( 0-1 )2 )) = exp( -1 ) = 0.3678

j41 = exp( -(( 1-1 )2+( 1-1 )2 )) = exp( 0 ) = 1

j12 = exp( -(( 0-0 )2+( 0-0 )2 )) = exp( 0 ) = 1

j22 = exp( -(( 0-0 )2+( 1-0 )2 )) = exp( -1 ) = 0.3678

j32 = exp( -(( 1-0 )2+( 0-0 )2 )) = exp( -1 ) = 0.3678

j42 = exp( -(( 1-0 )2+( 1-0 )2 )) = exp( -2 ) = 0.1353

---------------------------------------------------------------------------------------------------

        4   1.8709   1.8709
FTF =  0   1.2889   0.5412
        0   0        1.2889

            5.7274   -5.8553  -5.8553
( FTF )-1 = 5.8553  6.9281   5.5907
            -5.8553  5.5907   6.9281


FTy = 2   1.3678  0.5031

---------------------------------------------------------------------------------------------------

w0 = 0.5

w1 = 0.578235

w2 = -0.578236

----------------------------------------------------------------------------------------------------------------

Suggested Readings:

Bishop,C. (1995). Neural Networks for Pattern Recognition, Oxford University Press, Oxford, UK, pp.164-190 (Chapter 5).

Haykin, Simon. (1999). Neural Networks. A Comprehensive Foundation, Second Edition, Prentice-Hall, Inc., New Jersey, Section 5.