Back to the page

In the print window, choose “Save as PDF”.

doubtly doubtly.in

ML QuestionPaper Solution Dec 2023 – AI-DS/ML

Semester 6 AI-DS QP Solutions · Updated 30 May 2024 · Read online: www.doubtly.in/ml-questionpaper-solution-dec-2023-ai-ds-ml/

ML QuestionPaper Solution Dec 2023 – AI-DS/ML

Q.1 Solve any Four

A. Explain SVD and its applications. [05] ML

Singular Value Decomposition (SVD) is a powerful matrix factorization technique with various applications across different domains. Here are some common applications of SVD:

  1. Dimensionality Reduction:
  1. Image Compression:
  1. Recommender Systems:
  1. Data Denoising:
  1. Latent Semantic Analysis (LSA):
  1. Face Recognition:
  1. Principal Component Analysis (PCA):
  1. Signal Processing:

B. Differentiate between supervised and unsupervised learning. [05]

 Supervised LearningUnsupervised Learning
Input DataUses Known and Labeled Data as inputUses Unknown Data as input
Computational ComplexityLess Computational ComplexityMore Computational Complex
Real-TimeUses off-line analysisUses Real-Time Analysis of Data
Number of ClassesThe number of Classes is knownThe number of Classes is not known
Accuracy of ResultsAccurate and Reliable ResultsModerate Accurate and Reliable Results
Output data The desired output is given.The desired, output is not given.
Model In supervised learning it is not possible to learn larger and more complex models than in unsupervised learningIn unsupervised learning it is possible to learn larger and more complex models than in supervised learning
Training dataIn supervised learning training data is used to infer model In unsupervised learning training data is not used.
Another name Supervised learning is also called classification.Unsupervised learning is also called clustering.
Test of model We can test our model.We can not test our model.
Example  Optical Character RecognitionFind a face in an image.


C. Explain Hebbian Learning rule. [05]

Hebbian Learning Rule, also known as Hebb Learning Rule, was proposed by Donald O Hebb. It is one of the first and also easiest learning rules in the neural network. It is used for pattern classification. It is a single layer neural network, i.e. it has one input layer and one output layer. The input layer can have many units, say n. The output layer only has one unit. Hebbian rule works by updating the weights between neurons in the neural network for each training sample.

Hebbian Learning Rule Algorithm : 

  1. Set all weights to zero, wi = 0 for i=1 to n, and bias to zero.
  2. For each input vector, S(input vector) : t(target output pair), repeat steps 3-5.
  3. Set activations for input units with the input vector Xi = Si for i = 1 to n.
  4. Set the corresponding output value to the output neuron, i.e. y = t.
  5. Update weight and bias by applying Hebb rule for all i = 1 to n:


D. Explain the Perceptron model with Bias. [05]

Answer : /q/explain-perceptron-model-bias/


E. Differentiate between Ridge and Lasso Regression. [05]

Ridge RegressionLasso Regression
Shrinks the coefficients toward zeroEncourages some coefficients to be exactly zero
Adds a penalty term proportional to the sum of squared coefficientsAdds a penalty term proportional to the sum of absolute values of coefficients
Does not eliminate any featuresCan eliminate some features
Suitable when all features are importantlySuitable when some features are irrelevant or redundant
More computationally efficientLess computationally efficient
Requires setting a hyperparameterRequires setting a hyperparameter
Performs better when there are many small to medium-sized coefficientsPerforms better when there are a few large coefficients

Q.2 Solve the following

A. Draw a block diagram of the Error Back Propagation Algorithm and explain with a flowchart the Error Back Propagration concept . [10]

The Error Back Propagation Algorithm, commonly referred to as Backpropagation, is a fundamental algorithm used for training artificial neural networks. It is a supervised learning algorithm that aims to minimize the error between the predicted outputs and the actual outputs by adjusting the weights of the network.

  1. Neural Network Structure:
    • Input Layer: Receives input features.
    • Hidden Layers: Intermediate layers where computations are performed.
    • Output Layer: Produces the final output.
  2. Weights and Biases: Parameters that the algorithm adjusts to minimize the error.
  3. Activation Function: A non-linear function (like Sigmoid, ReLU, Tanh) applied to the neuron’s output to introduce non-linearity into the model.
  4. Loss Function: Measures the difference between the predicted output and the actual output (e.g., Mean Squared Error, Cross-Entropy Loss).

Steps in Backpropagation

  1. Initialization: Randomly initialize weights and biases.
  2. Forward Propagation:
    • Compute the output of each neuron from the input layer to the output layer.
    • For each layer, apply the activation function to the weighted sum of inputs.
  3. Compute Loss: Calculate the error using the loss function.
  4. Backward Propagation:
    • Compute the gradient of the loss function with respect to each weight by applying the chain rule of calculus.
    • This involves:
    • Output Layer: Calculate the gradient of the loss with respect to the output.
    • Hidden Layers: Backpropagate the gradient to previous layers, updating the weights and biases.
  5. Update Weights:
    • Adjust weights and biases using gradient descent or an optimization algorithm (like Adam, RMSprop).
    • Update rule:
      [
      w = w – \eta \frac{\partial L}{\partial w}
      ]
      where ( \eta ) is the learning rate, ( w ) is the weight, and ( \frac{\partial L}{\partial w} ) is the gradient of the loss with respect to the weight.
  6. Iterate: Repeat the forward and backward pass for a number of epochs or until convergence.

Practical Considerations

B. Find a linear regression equation for the following two sets of data:

XY
312
518
724
930

Answer : /q/find-linear-regression-equation-sets-data-10/

Q.3 Solve the following

A. Diagonalize the matrix A

Answer : /q/diagonalize-matrix-1-5-4-2/

B. Write a short note on overfitting and underfitting of a model. [05]

Underfitting in Machine Learning

Underfitting occurs when a statistical model or machine learning algorithm is too simple to capture the complexities of the data. This results in poor performance on both the training and testing data. An underfit model fails to learn the underlying patterns in the training data, leading to inaccurate predictions, especially on new, unseen examples. Underfitting is typically associated with high bias and low variance, often caused by overly simplified models with basic assumptions.

Reasons for Underfitting:

  1. Overly Simple Model: The model lacks the capacity to represent the data’s complexities.
  2. Inadequate Feature Representation: The input features do not sufficiently capture the factors influencing the target variable.
  3. Insufficient Training Data: The dataset is too small to allow the model to learn effectively.
  4. Excessive Regularization: Over-regularizing to avoid overfitting can overly constrain the model.
  5. Unscaled Features: Features not appropriately scaled can hinder the model’s ability to learn.

Techniques to Reduce Underfitting:

  1. Increase Model Complexity: Use models with more parameters or layers.
  2. Enhance Feature Engineering: Add more relevant features to better capture underlying patterns.
  3. Remove Noise: Clean the data to reduce irrelevant information.
  4. Increase Training Duration: Train the model for more epochs to allow better learning.

Overfitting in Machine Learning

Overfitting occurs when a model performs exceptionally well on the training data but fails to generalize to new, unseen data. This happens when the model learns not only the underlying patterns but also the noise and outliers in the training data, resulting in high variance and poor performance on test data. Overfitting is often seen with complex models that have too much flexibility.

Reasons for Overfitting:

  1. High Variance, Low Bias: The model captures too much detail from the training data.
  2. Excessive Model Complexity: The model has too many parameters relative to the amount of training data.
  3. Insufficient Training Data: The model learns noise due to a lack of diverse training examples.

Techniques to Reduce Overfitting:

  1. Improve Training Data Quality: Ensure the data is clean and representative of the problem.
  2. Increase Training Data Size: More data helps the model generalize better.
  3. Reduce Model Complexity: Simplify the model by reducing the number of parameters or layers.
  4. Early Stopping: Stop training once the performance on validation data starts to degrade.
  5. Regularization Techniques: Use Ridge (L2) or Lasso (L1) regularization to penalize large coefficients.
  6. Dropout in Neural Networks: Randomly drop units during training to prevent co-adaptation.


C. What are activation functions? Explain Binary, Bipolar, Continuous, and Ramp activation functions. [10]

Activation functions are mathematical functions applied to the output of each neuron in a neural network. They introduce non-linearity into the network, enabling it to learn complex patterns in the data. Here’s an explanation of the Binary, Bipolar, Continuous, and Ramp activation functions:

  1. Binary Activation Function:
  1. Bipolar Activation Function:
  1. Continuous Activation Function:
  1. Ramp Activation Function:

These activation functions serve different purposes and are chosen based on the requirements of the neural network architecture and the nature of the problem being solved.

Q.4 Solve the following

A. Explain Least-Squares Regression for classification. [10]

B. What is the curse of dimensionality? Explain the PCA dimensionality reduction technique in detail. [10]

Handling the high-dimensional data is very difficult in practice, commonly known as the curse of dimensionality. If the dimensionality of the input dataset increases, any machine learning algorithm and model becomes more complex. As the number of features increases, the number of samples also gets increased proportionally, and the chance of overfitting also increases. If the machine learning model is trained on high-dimensional data, it becomes overfitted and results in poor performance.

Hence, it is often required to reduce the number of features, which can be done with dimensionality reduction.

Principal Component Analysis is an unsupervised learning algorithm that is used for the dimensionality reduction in machine learning. It is a statistical process that converts the observations of correlated features into a set of linearly uncorrelated features with the help of orthogonal transformation. These new transformed features are called the Principal Components. It is one of the popular tools that is used for exploratory data analysis and predictive modeling. It is a technique to draw strong patterns from the given dataset by reducing the variances.

PCA generally tries to find the lower-dimensional surface to project the high-dimensional data.

PCA works by considering the variance of each attribute because the high attribute shows the good split between the classes, and hence it reduces the dimensionality. Some real-world applications of PCA are image processing, movie recommendation system, optimizing the power allocation in various communication channels. It is a feature extraction technique, so it contains the important variables and drops the least important variable.

The PCA algorithm is based on some mathematical concepts such as:

Steps for PCA algorithm

  1. Getting the dataset
    Firstly, we need to take the input dataset and divide it into two subparts X and Y, where X is the training set, and Y is the validation set.
  2. Representing data into a structure
    Now we will represent our dataset into a structure. Such as we will represent the two-dimensional matrix of independent variable X. Here each row corresponds to the data items, and the column corresponds to the Features. The number of columns is the dimensions of the dataset.
  3. Standardizing the data
    In this step, we will standardize our dataset. Such as in a particular column, the features with high variance are more important compared to the features with lower variance.
    If the importance of features is independent of the variance of the feature, then we will divide each data item in a column with the standard deviation of the column. Here we will name the matrix as Z.
  4. Calculating the Covariance of Z
    To calculate the covariance of Z, we will take the matrix Z, and will transpose it. After transpose, we will multiply it by Z. The output matrix will be the Covariance matrix of Z.
  5. Calculating the Eigen Values and Eigen Vectors
    Now we need to calculate the eigenvalues and eigenvectors for the resultant covariance matrix Z. Eigenvectors or the covariance matrix are the directions of the axes with high information. And the coefficients of these eigenvectors are defined as the eigenvalues.
  6. Sorting the Eigen Vectors
    In this step, we will take all the eigenvalues and will sort them in decreasing order, which means from largest to smallest. And simultaneously sort the eigenvectors accordingly in matrix P of eigenvalues. The resultant matrix will be named as P*.
  7. Calculating the new features Or Principal Components
    Here we will calculate the new features. To do this, we will multiply the P* matrix to the Z. In the resultant matrix Z*, each observation is the linear combination of original features. Each column of the Z* matrix is independent of each other.
  8. Remove less or unimportant features from the new dataset.
    The new feature set has occurred, so we will decide here what to keep and what to remove. It means, we will only keep the relevant or important features in the new dataset, and unimportant features will be removed out.

Applications of Principal Component Analysis

Q.5 Solve the following

A. How to calculate Performance Measures by Measuring the Quality of a model. [10]

Answer : /q/calculate-performance-measures-measuring-quality-model/


B. Explain the Perceptron Neural Network. [10]

Perceptron is one of the simplest Artificial neural network architectures. It was introduced by Frank Rosenblatt in 1957s. It is the simplest type of feedforward neural network, consisting of a single layer of input nodes that are fully connected to a layer of output nodes. It can learn the linearly separable patterns. it uses slightly different types of artificial neurons known as threshold logic units (TLU). it was first introduced by McCulloch and Walter Pitts in the 1940s.

Types of Perceptron

Basic Components of Perceptron

A perceptron, the basic unit of a neural network, comprises essential components that collaborate in information processing.

Q.6

A. Discuss the various steps of developing a Machine Learning Application. [10]

Machine Learning (ML) is a subset of artificial intelligence (AI) that focuses on enabling machines to learn from data without explicit programming. It allows computers to automatically learn and improve from experience, uncover patterns in data, and make predictions or decisions based on that data.

Steps in Developing a Machine Learning Application:

  1. Collect Data: Gather relevant data from various sources.
  2. Prepare Input Data: Clean, preprocess, and format the collected data for analysis.
  3. Analyze Input Data: Explore and understand the data to identify patterns, trends, and relationships.
  4. Train Algorithm: Select and train the machine learning algorithm(s) on the prepared data.
  5. Test Algorithm: Evaluate the performance of the trained algorithm(s) using test data to ensure accuracy and generalization.
  6. Use Algorithm: Deploy the trained algorithm(s) for making predictions or decisions in real-world applications.
  7. Predict and Revise: Utilize the algorithm(s) to generate predictions or outcomes, and revisit the model periodically for updates and improvements based on new data or feedback.

B. Write a short note on LMS-Widrow Hoff. [05]

Answer here : /q/draw-delta-learning-rule-lms-widrow-hoff-model/

C. Explain the Maximization algorithm for clustering. [05]

The Expectation-Maximization (EM) algorithm is defined as the combination of various unsupervised machine learning algorithms, which is used to determine the local maximum likelihood estimates (MLE) or maximum a posteriori estimates (MAP) for unobservable variables in statistical models. Further, it is a technique to find maximum likelihood estimation when the latent variables are present. It is also referred to as the latent variable model.

A latent variable model consists of both observable and unobservable variables where observable can be predicted while unobserved are inferred from the observed variable. These unobservable variables are known as latent variables.

Key Points:

EM Algorithm

The EM algorithm is the combination of various unsupervised ML algorithms, such as the k-means clustering algorithm. Being an iterative approach, it consists of two modes. In the first mode, we estimate the missing or latent variables. Hence it is referred to as the Expectation/estimation step (E-step). Further, the other mode is used to optimize the parameters of the models so that it can explain the data more clearly. The second mode is known as the maximization-step or M-step.

EM Algorithm in Machine Learning

The primary goal of the EM algorithm is to use the available observed data of the dataset to estimate the missing data of the latent variables and then use that data to update the values of the parameters in the M-step.


Practise this on Doubtly (free)