ROHIT's Smart Class Room

Perceptron Convergence and Linear Separability The perceptron algorithm converges only when the dataset is linearly separable . If the data cannot be separated by a straight line or hyperplane, the algorithm will not converge. Algorithm:   The perceptron convergence theorem states that, for any data set which is linearly separable, the perception learning rule or algorithm will converge to a solution in finite no of iterations or find a solution in a finite number of steps. The perceptron learning algorithm updates its weight vector  w  using the following rule:   The update is performed only when the perceptron misclassifies a data point. The theorem guarantees that:  If the data is linearly separable , the perceptron algorithm will converge in a finite number of steps . If the data is not linearly separable , the perceptron will continue updating indefinitely. Linear Separability A dataset is linearly separable if we can separate the two classe...

Linear Separability The training data must be linearly separable. This means there exists a hyperplane that can separate the positive and negative examples without any errors.   Linearly Separable data:  It refers to a set of data points can be perfectly divided into two distinct classes using a straight line (in two dimensions), a plane (in three dimensions), or a hyperplane (in higher dimensions). This means that there exists a linear boundary that can clearly separate all the data points of one class from those of the other class without any overlap. 2. Non-linearly separable data:  It refers to a set of data points cannot be divided into two distinct classes using a straight line (in two dimensions), a plane (in three dimensions), or a hyperplane (in higher dimensions). In other words, no single linear boundary can perfectly separate the two classes. Limitations of perceptron: - Linearly separable data : Perceptrons can only classify data that is linearl...

Geometric interpretation The perceptron can be functionally visualized as In classification, the model tries to separate classes. Geometrically: In 2D → Decision boundary is a line In 3D → Decision boundary is a plane In nD → Decision boundary is a hyperplane Example: Perceptron Equation: w 1 x 1 + w 2 x 2 + b = 0 w_1x_1 + w_2x_2 + b = 0 w 1 ​ x 1 ​ + w 2 ​ x 2 ​ + b = 0 This represents a line in 2D . In this case, the vector X is an input vector of dimensionality n, x⊂R, whose elements are the values x 1 ,x 2 ...x n .   Note that a ‘1’ is concatenated to the front of any input vector. These values are all multiplied by a corresponding  weight , w 0  to w n . w 1  to w n are called  correlating weights , and w 0 , which is multiplied against the ‘1,’ is called the  bias . These products are all summed, generating a  weighted sum  of the inputs. ...

Perceptron Learning Rule in Machine Learning The Perceptron Learning Rule is the weight update rule used to train a single layer perceptron for binary classification . It adjusts the weights whenever the model makes a mistake. The goal is to move the decision boundary so that misclassified points are correctly classified.  Perceptron Model Structure Mathematical Model z = w 1 x 1 + w 2 x 2 + . . . + w n x n + b z = w_1x_1 + w_2x_2 + ... + w_nx_n + b y ^ = f ( z ) Where: x 1 , x 2 , . . . , x n x_1, x_2, ..., x_n  → input features w 1 , w 2 , . . . , w n w_1, w_2, ..., w_n  → weights b b  → bias f ( z ) f(z)  → Step activation function Step Function f ( z ) = { 1 z ≥ 0 0 z < 0 f(z) = \begin{cases} 1 & z \ge 0 \\ 0 & z < 0 \end{cases} When prediction is wrong, update weights using: w i = w i + η ( y − y ^ ) x i w_i = w_i + \eta (y - \hat{y}) x_i ​ b = b + η ( y − y ^ ) b = b + \eta (y - \hat{y}) Where: y y  → actual output ...