1. The Perceptron

The Perceptron is a network in which the neuron unit calculates the linear combination of its real-valued or boolean inputs and passes it through a threshold activation function, which in case of two input variables is: o = Threshold( w₀x₀ + w₁x₁ + w₂x₂ ), where the activation function could be defined as follows: Threshold ( s ) = 1 if s > 0 and 0 otherwise. Our single-layer Perceptron in this two-dimensional case looks like the following box where the inputs are multiplied by the weights when they enter the function.

       ----------------------------
      |               --------     |
      |  x0=1->w0--->|        |    |
      |              |        |    |
x1--->|------->w1--->| Neuron |--->|---> Y
      |              |        |    |
x2--->|------->w2--->|        |    |
      |               --------     |
       ----------------------------

Example: Suppose that we have to train the Perceptron to learn the following function of two boolean variables, assuming that the initial weights' values are: w₀=0.5, w₁=0.5, and w₂=0.5.

(x1,x2)| Y
---------
 (0,0) | 1
 (0,1) | 0
 (1,0) | 0
 (1,1) | 0

Solution:

------------------------------------------------------------------------------------
example:        (0,0) | 1
                o = 0.5 + 0 + 0 = 0.5 > 0 ok!

example:        (0,1) | 0
                o = 0.5 + 0 + 0.5 = 1 > 0, hence o = 1 Error, should be 0
         
weight update:  w0 = 0.5 + ( 0 - 1 ) * 1 = -0.5
                w1 = 0.5 + ( 0 - 1 ) * 0 =  0.5        
                w2 = 0.5 + ( 0 - 1 ) * 1 = -0.5

example:        (1,0) | 0
                o = -0.5 + 0.5 + 0 = 0 <= 0 ok!

example:        (1,1) | 0
                o = -0.5 + 0.5 + ( -0.5 * 1 ) = -0.5 < 0 ok!
------------------------------------------------------------------------------------
example:        (0,0) | 1
                o = -0.5 + 0 + 0 = -0.5 < 0, hence o = 0 Error, should be 1
         
weight update:  w0 = -0.5 + ( 1 - 0 ) * 1 = 0.5
                
example:        (0,1) | 0
                o = 0.5 + 0 + ( -0.5 ) = 0 <= 0 ok!

example:        (1,0) | 0
                o = 0.5 + 0.5 + 0 = 1 > 0  hence o = 1 Error, should be 0

weight update:  w0 = 0.5 + ( 0 - 1 ) * 1 = -0.5
                w1 = 0.5 + ( 0 - 1 ) * 1 = -0.5        

example:        (1,1) | 0
                o = -0.5 + ( -0.5 * 1 ) + ( -0.5 * 1 ) = -1.5 < 0 ok!
------------------------------------------------------------------------------------
example:        (0,0) | 1
                o = -0.5 + 0 + 0 = -0.5 < 0, hence o = 0 Error, should be 1
                w0 = -0.5 + ( 1 - 0 ) * 1 = 0.5
------------------------------------------------------------------------------------

Therefore, the learned function is: o = ( 0.5 - 0.5x₁ - 0.5x₂ ). The Perceptron has learned the function which is "true" for the coordinates (x₁,x₂) = (0,0) but "false" for coordinates (0,1), (1,0), and (1,1). This agrees with the truth-table for the so called "NOR" function.

2. Learning the XOR Examples by the Perceptron:

Tthe Perceptron cannot learn the examples of the boolean XOR problem (or, generally, the parity function of its inputs).

What happens then to the weights of a two input Perceptron using A step threshold activation function, beginning with all weights set to 0.5, as the XOR examples are provided for training?

Input Output
(0,0) 0
(0,1) 1
(1,0) 1
(1,1) 0

The perceptron weights oscillate, they change to the same values, here: 0.5, -0.5, 1.5 without converging finally to a weight vector with which all examples are classified correctly.