2.2 Perceptron Algorithm in Machine Learning

Perceptron Algorithm in Machine Learning

Perceptron: -

A perceptron is a simple supervised learning algorithm used for binary classification that separates data using a a straight line (linear decision boundary) and produces output 0 or 1.

It was introduced in 1957 by Frank Rosenblatt.

It is used for binary classification, which means it decides between two possible outputs, like:

0 or 1
Yes or No
Pass or Fail
Spam or Not Spam

It works only when the data is linearly separable.

Imagine a Perceptron as a single "neuron" in a robot's brain. It is the simplest version of an Artificial Neural Network.

It was designed to take a number of binary inputs and produce one binary output (0 or 1).

This algorithm enables neurons to learn and processes elements in the training set one at a time.

Frank Rosenblatt proposed a Perceptron learning rule based on the original MCP (McCulloch-Pitts) neuron.

It has five main basic parts or components:

Inputs or features (x₁, x₂, x₃...) – the data we give
Weights (w₁, w₂, w₃...) – importance of each input
Bias (b) – extra value to adjust decision
Weighted Sum – calculation of inputs and weights
Step Function – decides final output (0 or 1)

1) Inputs (x₁, x₂, x₃…)

Inputs are the values we give to the perceptron.They represent the characteristics or features of the data.

For example, if we want to predict whether a student will pass or fail, the inputs can be:

x₁ = number of hours studied
x₂ = attendance percentage
x₃ = number of assignments completed

These inputs are just numbers that describe the situation.

2) Weights (w₁, w₂, w₃…)

Weights show how important each input is.

Every input has its own weight.
If an input is more important for making the decision, its weight will be higher.

For example:

If studying hours are very important, w₁ will be large.
If attendance is less important, w₂ will be smaller.

So weights control how much influence each input has on the final result.

3) Bias (b)

Bias is an extra value added to the calculation.

It helps the perceptron adjust the final decision.

You can think of bias as a starting point or shift.

Without bias, the perceptron decision would always start from zero.

Bias allows the model to move the decision boundary.

For example, even if a student studies a little, the teacher may still require a minimum level to pass.
Bias helps set that minimum level.

4) Weighted Sum

The weighted sum is the main calculation step.

The perceptron multiplies each input by its weight and then adds them all together. After that, it adds the bias.

Formula: Weighted Sum = (x₁ × w₁) + (x₂ × w₂) + (x₃ × w₃) + b

This value decides what happens next. It combines all the information into one single number.

5) Step Function

The step function gives the final answer.

It checks the weighted sum and compares it with zero.

If the value is greater than or equal to zero, the output is 1.
If the value is less than zero, the output is 0.

So the step function converts the calculated number into a simple decision: yes or no.

Example: -

Training Algorithm: - The perceptron is a trained using a supervised learning algorithm such as the perceptron learning algorithm or backpropagation. During training, the perceptron adjusts its weights and biases using Perceptron learning algorithm to minimize the error between the predicted output and the true output for a given set of training examples.

Types of Perceptron

1) Single-Layer Perceptron is a type of perceptron is limited to learning linearly separable patterns. It is effective for tasks where the data can be divided into distinct categories through a straight line. it struggles with more complex problems where the relationship between inputs and outputs is non-linear.

2) Multi-Layer Perceptron has enhanced processing capabilities as they consist of two or more layers, adept at handling more complex patterns and relationships within the data. It is mainly similar to a single-layer perceptron model but has more hidden layers

Working process of Perceptron: -

step 1: - Definition

A perceptron tries to learn a line (in 2D), plane (in 3D), or hyperplane (in higher dimensions) that separates two classes.

Mathematically, it computes:

$y = f (w_{1} x_{1} + w_{2} x_{2} + . . . + w_{n} x_{n} + b)$

Where:

$x_{1}, x_{2}, . . ., x$ → input features
$w_{1}, w_{2}, . . ., w_{n}$ → weights
$b$ → bias
$f$ → activation function (step function)

Step 2: - Steps inside perceptron:

Multiply inputs with weights
Add bias
Apply step activation function
Output 0 or 1

step 3: - Step Activation Function

$f (z) = {\begin{cases} 1 & if z \geq 0 (Threshold) \end{cases}$

$\begin{array}{ll} 0 & if z < 0 (Threshold) \end{array}$

It converts the weighted sum into a binary output.

step 4: -Perceptron Learning Algorithm (Training Steps)

Given training data $(x, y)$ :

Initialize

Set weights $w$ and bias $b$ to small values (usually 0).

For each training example

Compute output:
$\hat{y} = f (w \cdot x + b)$
Update weights:
$w = w + η (y - \hat{y}) x$
Update bias:
$b = b + η (y - \hat{y})$

Where:

$\eta$ = learning rate
$y$ = actual output
$\hat{y}$ = predicted output

Repeat until no errors or maximum iterations reached.

Advantages

Simple and fast
Works well for linearly separable data
Foundation for neural networks

Limitations

Cannot solve non-linear problems (like XOR)
Only binary classification
Sensitive to learning rate

Simple Example

Problem:

Classify whether a student passes (1) or fails (0) based on study hours.

Study Hours (x)	Output (y)
1	0
2	0
3	1
4	1

We want a line that separates fail and pass.

Assume:

Initial weight $w = 0$
Bias $b = 0$
Learning rate $\eta = 1$

Iteration 1 (x = 1, y = 0)

$z = 0 (1) + 0 = 0$

Predicted = 1 (since z ≥ 0)

Error = 0 − 1 = −1

Update:

$w = 0 + (-1)(1) = -1$ $b = 0 + (- 1) = - 1$

After several iterations, weights adjust and the model learns a boundary like:

$w x + b = 0$

Example final model:

$x - 2.5 = 0$

Meaning:

If study hours ≥ 2.5 → Pass
If study hours < 2.5 → Fail

Search This Blog

ROHIT's Smart Class Room