In machine learning and neural networks theory the most common idea used is that of the perceptron. This is a concept that its first apperance goes back to the distant 1943. It was conceived by two scientists, a neuroscientists called Warren S. McCulloch and a logician called Walter Pitts. They published a paper " A logical calculus of the ideas immanent in nervous activity". In this paper they tried to understand how the brain conducts complex procedures by using basic interconnected cells. These brain cells are cold neurons.

So what exactly is a perceptron?

Actually, the perceptron is an immitation of these neurons. Scientists have created a model that resembles the mechanism of neurons and has the exact same functionality. In order to have a better look we shall devide the perceptron into 4 main parts and observe its core components.

1. Inputs

2. Weights

3. Net-Sum

4. Activation Function

(1) inputs : Every single input is an array of n elements. Where n refers to the dimension of the array [ x1, x2, ... ,xn ]. Every element of the array refers to a characteristic of a specific object. For example consider our objects refer on two distinguished categories:

--> Category1 = Cars & Category2 = Vans.

(2) weights : Weights like inputs are n dimensional arrays [ w1, w2, ... , wn ]. Every element of the weights array correlates to the according element of the inputs array. Each of these elements denotes the importance of the specific characteristic. So the bigger the weight is, the bigger the impact will be on the final decision making for the category justification.

There is another weight that is irrelevant to inputs and gives a biased value for our estimamtion. It is called bias and is often symbolized as b or w0.

In fact weights give us parameter to calculate a decision boundary between category. That according to dimension of the our problem would be:

If 2 dimemsions, we have a line in the form: w1*x + w2*x + b = 0

If 3 dimemsions, we have a plane in the form: w1*x + w2*x + w3*x + b = 0

If n dimemsions, we have a hyperplane in the form: w1*x + w2*x + w3*x+ ... + wn*xn + b = 0

(3) net-sum : inputs and weights should be array multiplied in order to have a decision making result. This is feaseble using the mathematical formula:
inputs[ ] x weights [ ] + b = [ w1*x1 + w2*x2 + w3*x3+ ... + wn*xn ]+ b

(4) activation function : according to the net-sum result the activation function triggers a category decision based on this result, usually | 0 or 1 | or |-1 or 1|.

So if:

-> net-sum < 0 we assume that input values are under the decision boundary, thus activation function output equals 0 and perceptron decides the input is of type cars

and if:

-> net-sum >= 0 we assume that input values are above the decision boundary, thus activation function output equals 1 and perceptron decides the input is of type vans.

1. Presenting perceptron procedure

On summary

The Perceptron is a classifation algoritm. Its main funtion is to collect data and determine for any single piece of data the category it belongs. This is feasible by using weights and calculating the decision boundary. Finally the activation function is the last tool for completing the classification proccess.

Follow next article to learn more about how we train the perceptron in order to bring reliable results.