The Delta Learning Rule, also known as the LMS (Least Mean Squares) algorithm or the Widrow-Hoff rule, is a method for adjusting the weights of a linear neuron in a neural network during training. Below is a graphical representation of the Delta Learning Rule model:
Components of the Delta Learning Rule Model:
- Input Features:
- Represents the input data features (e.g., pixel values in an image, numerical attributes in a dataset).
- Weighted Summation:
- Each input feature is multiplied by its corresponding weight.
- The weighted inputs are summed up to produce a weighted sum.
- Activation Function:
- The weighted sum is passed through an activation function.
- Common activation functions include the step function, sigmoid function, or rectified linear unit (ReLU).
- Output:
- Represents the output of the neuron after applying the activation function.
- The output can be binary (0 or 1) for binary classification tasks or continuous for regression tasks.
Operation of the Delta Learning Rule:
- Initialization:
- Initialize the weights of the neuron with random or predefined values.
- Forward Propagation:
- Pass the input features through the neuron.
- Calculate the weighted sum of inputs.
- Apply the activation function to obtain the output.
- Error Calculation:
- Compare the predicted output to the true output (target).
- Calculate the error as the difference between the predicted output and the target.
- Weight Adjustment (Learning):
- Update the weights based on the error using the Delta Rule formula: Δwij=η⋅(di−yi)⋅xij Where:
- Δwij is the change in weight.
- η is the learning rate.
- di is the target output.
- yi is the predicted output.
- xij is the input feature.
- Adjust the weights to minimize the error and improve the accuracy of the model.
- Repeat:
- Repeat steps 2-4 for each training example in the dataset.
- Iterate through the entire dataset multiple times (epochs) until the model converges or until a stopping criterion is met.
