Back to the page

In the print window, choose “Save as PDF”.

doubtly doubtly.in

DAV Question Paper solution May 2023

Semester 6 AI-DS QP Solutions · Updated 30 May 2024 · Read online: www.doubtly.in/dav-question-paper-solution-may-2023/

download as an pdf :

https://drive.google.com/file/d/1ii57ewWx1IrbSarEgfZ9M6cWz7-md59p/view?usp=sharing

DAV Question Paper solution May 2023 , some of the answers have an link , if you know the answer please add an solution to it

Q1

a) What is an analytic sandbox, and why is it important ?

An analytical sandbox is a testing environment that is used by data analysts and data scientists to experiment with data and explore various analytical approaches without affecting the production environment. It is a separate, isolated environment that contains a copy of the production data, as well as the necessary tools and resources for data analysis and visualization.

Analytical sandboxes are typically used for a variety of purposes, including testing and validating new analytical approaches and algorithms, trying out different data sets, collaborating and sharing work with colleagues, and testing new data visualization techniques and dashboards.

Analytical Sandbox’s Essential Components Include:

  1. Business Analytics (Enterprise Analytics) – The self-service BI tools for situational analysis and discovery are part of business analytics.
  2. Analytical Sandbox Platform – The capabilities for processing, storing, and networking are provided by the analytical sandbox platform.
  3. Data Access and Delivery – Data collection and integration are made possible by data access and delivery from a number of data sources and data kinds.
  4. Data Sources – Big data (unstructured) and transactional data (structured) are two types of data sources that can come from both inside and outside of the company. Examples of these sources include extracts, feeds, messages, spreadsheets, and documents.

Graphical view of Analytical Sandbox Components

https://media.geeksforgeeks.org/wp-content/uploads/20221031183832/gfg1.png

Importance of an Analytical Sandbox

  1. Data from various sources, both internal and external, both unstructured and structured, can be combined and filtered using analytical sandboxes.
  2. Data scientists can carry out complex analytics with the help of analytical sandboxes.
  3. Analytical sandboxes enable working with data initially.
  4. Analytical sandboxes make it possible to use high-performance computing while processing databases because the analytics takes place inside the database itself.

b) Why use autocorrelation instead of autocovariance when examining stationary  time series?

Autocorrelation and autocovariance are one of the most critical metrics in financial time series econometrics. Both functions are based on covariance and correlation metrics.

What is Autocorrelation? Autocorrelation measures the degree of similarity between a given time series and the lagged version of that time series over successive time periods. It is similar to calculating the correlation between two different variables except in Autocorrelation we calculate the correlation between two different versions Xt and Xt-k of the same time series.

What is Autocovariance? Autocovariance is defined as the covariance between the present value (xt) with the previous value (xt-1) and the present value (xt) with (xt-2). And it is denoted as γ. In a stationary time series, data are challenging and are likely to change considering the assumptions of linear programming. In such a situation, the residual errors can be correlated in order to eliminate inconsistency. When the autocovariance is utilized, its variables can change over time which can lead to invalid results.

In stationary time series, the statistical properties of the data do not change considerably over time. Hence autocorrelation will be much applicable when compared to autocovariance.

c) Difference between Pandas and NumPy.

Answer : /q/difference-pandas-numpy/

d) What is regression? What is simple linear regression?

Answer : /q/regression-simple-linear-regression/

Q2 a) Explain in detail how dirty data can be detected in the data exploration phase  with visualizations. 

Answer will be added here : /q/explain-detail-dirty-data-detected/

Q2 b) List and explain methods that can be used for sentiment analysis.

Sentiment analysis is the process of identifying and extracting subjective information from text, such as opinions and emotions. 

There are several methods that can be used for sentiment analysis, including: 

  1. Lexicon-based methods: In this method, sentiment scores are assigned to words based on their dictionary meanings. The sentiment scores are then aggregated to produce an overall sentiment score for a piece of text. 
https://ars.els-cdn.com/content/image/3-s2.0-B9780128186992000019-f01-08-9780128186992.jpg

The different approaches to lexicon-based approach are:

Dictionary-based approach

In this approach, a dictionary is created by taking a few words initially. Then an online dictionary, thesaurus or WordNet can be used to expand that dictionary by incorporating synonyms and antonyms of those words. The dictionary is expanded till no new words can be added to that dictionary. The dictionary can be refined by manual inspection.

 Corpus-based approach

This finds sentiment orientation of context-specific words. The two methods of this approach are:

  1. Machine learning-based methods: In this method, a machine learning algorithm is trained on a labeled dataset to predict the sentiment of a given piece of text. The algorithm can then be used to classify new text as positive, negative, or neutral. 
  2. Hybrid methods: These methods combine lexicon-based and machine learning-based methods to improve the accuracy of sentiment analysis. 
  3. Rule-based methods: In this method, a set of rules are defined to identify sentiment in a piece of text. The rules may be based on linguistic rules or domain-specific rules. The choice of method depends on the specific requirements of the analysis, the quality and quantity of data, and the available resources.
https://media.geeksforgeeks.org/wp-content/uploads/20230405125355/Rule-based-nlp.jpg

Steps in Rule-based approach:

  1. Rule Creation: Based on the desired tasks, domain-specific linguistic rules are created such as grammar rules, syntax patterns, semantic rules or regular expressions.
  2. Rule Application: The predefined rules are applied to the inputted data to capture matched patterns.
  3. Rule Processing: The text data is processed in accordance with the results of the matched rules to extract information, make decisions or other tasks.
  4. Rule refinement: The created rules are iteratively refined by repetitive processing to improve accuracy and performance. Based on previous feedback, the rules are modified and updated when needed.

Q3 a) List and explain the main phases of the Data Analytics Lifecycle.

Data Analytics Lifecycle :
The Data analytic lifecycle is designed for Big Data problems and data science projects. The cycle is iterative to represent real project. To address the distinct requirements for performing analysis on Big Data, step – by – step methodology is needed to organize the activities and tasks involved with acquiring, processing, analyzing, and repurposing data.

Phase 1: Discovery –

Phase 2: Data Preparation –

Phase 3: Model Planning –

Phase 4: Model Building –

Phase 5: Communication Results –

Phase 6: Operationalize –

Q3 b) Describe how logistic regression can be used as a classifier.

Logistic Regression in Machine Learning

Why do we use Logistic Regression rather than Linear Regression?

Logistic Regression  greater than 1

Q4 a) Suppose everyone who visits a retail website gets one promotional offer or no promotion at all. We want to see if making a promotional offer makes a  difference. What statistical method would you recommend for this analysis? 

Q4 b) List and explain the steps in the Text Analysis.

Language Identification

Tokenization

Sentence Breaking

Part of Speech Tagging

Chunking

Syntax Parsing

Sentence Chaining

Q5 a) How does the ARMA model differ from the ARIMA model? In what situation is  the ARMA model appropriate? 

The ARMA (AutoRegressive Moving Average) model and the ARIMA (AutoRegressive Integrated Moving Average) model are both commonly used in time series analysis, but they have distinct differences in their structure and application.

  1. ARMA Model:
    • Definition: The ARMA model combines autoregressive (AR) and moving average (MA) components to model a time series.
    • Autoregressive (AR) Component: The AR part of the model captures the relationship between an observation and a number of lagged observations (i.e., its own past values). It essentially predicts the next value in the time series based on its previous values.
    • Moving Average (MA) Component: The MA part of the model represents the relationship between an observation and a residual error from a moving average model applied to lagged observations.
    • Formula: An ARMA(p, q) model has p autoregressive terms and q moving average terms.
    • Applicability: The ARMA model is appropriate when the time series exhibits both autoregressive and moving average properties, but does not have a trend or seasonality.
  2. ARIMA Model:
    • Definition: The ARIMA model extends the ARMA model by including differencing to handle non-stationary time series.
    • Integrated (I) Component: The “I” in ARIMA stands for integrated, indicating that the series has been differenced at least once to achieve stationarity. This differencing step removes trends or seasonality present in the series.
    • Formula: An ARIMA(p, d, q) model has parameters p, d, and q, where p represents autoregressive terms, d represents the degree of differencing, and q represents moving average terms.
    • Applicability: The ARIMA model is suitable when the time series exhibits trends and/or seasonality that can be removed through differencing, making the series stationary. It’s commonly used when the series shows a trend or seasonal patterns that need to be accounted for in the modeling process.

When to Use ARMA vs ARIMA:

Q5 b) Explain with suitable example how the Term Frequency and Inverse Document  Frequency are used in information retrieval. 

Answer : /q/explain-suitable-term-frequency/

Q6 Write short notes on:  

a) Evaluating the Residuals in Linear regression.

Answer : /q/evaluating-residuals-linear-regression/

b) Box-Jenkins Methodology

Box-Jenkins Method

Box-Jenkins method is a type of forecasting and analyzing methodology for time series data. Box-Jenkins method comprises of three stages through which time series analysis could be performed. It comprises of different steps including identification, estimation, diagnostic checking, model refinement and forecasting. The Box-Jenkins method is an iterative process, and steps 1 to 4 from identification to model refinement are often repeated until a suitable and well-diagnosed model is obtained. It is important to note that the method assumes that the underlying time series data is generated by a stationary and linear process. The different stages of the Box-Jenkins model could be identified as:

Identification:

Identification is the first step of Box-Jenkins method it helps in determining the orders of autoregressive (AR), differencing (I), and moving average (MA) components that are appropriate for a given time series. This step helps in identifying the values of p, d and q for the given time series. Let’s see the key stages involved in this phase:

Estimation:

Estimation is the second stage in the Box-Jenkins methodology for ARIMA modeling. In this stage, the identified ARIMA model parameters, including the autoregressive (AR), differencing (I), and moving average (MA) components, are estimated based on historical time series data. The primary goal is to fit the chosen ARIMA model to the observed data. Let’s see the key stages involved in this phase:

Diagnostic Checking:

Diagnostic checking is an important step in the Box-Jenkins methodology for ARIMA modeling. It involves evaluating the acceptance of the fitted ARIMA model by examining the residuals, which are the differences between the observed and predicted values. The goal is to ensure that the residuals are random and do not contain any patterns or structure. Now, let’s discuss the key aspects of diagnostic checking in Box-Jenkins:

Model Refinement:

c) Seaborn Library.

Seaborn is an amazing visualization library for statistical graphics plotting in Python. It provides beautiful default styles and color palettes to make statistical plots more attractive. It is built on top matplotlib library and is also closely integrated with the data structures from pandas.
Seaborn aims to make visualization the central part of exploring and understanding data. It provides dataset-oriented APIs so that we can switch between different visual representations for the same variables for a better understanding of the dataset.

Different categories of plot in Seaborn 

Plots are basically used for visualizing the relationship between variables. Those variables can be either completely numerical or a category like a group, class, or division. Seaborn divides the plot into the below categories – 
 

d) Data import and Export in R

In R, you can import and export data using various functions and packages. Here’s an overview of some common methods:

Importing Data

  1. From a CSV file using read.csv() or read.csv2():
   data <- read.csv("filename.csv")
  1. From an Excel file using readxl package:
   library(readxl)
   data <- read_excel("filename.xlsx")
  1. From a text file using read.table():
   data <- read.table("filename.txt", header = TRUE) # Assuming the first row contains column names
  1. From a JSON file using jsonlite package:
   library(jsonlite)
   data <- fromJSON("filename.json")
  1. From a database using DBI and a specific DBMS package like RMySQL, RPostgreSQL, etc.:
   library(RMySQL)
   con <- dbConnect(MySQL(), user = "user", password = "password", dbname = "database")
   data <- dbGetQuery(con, "SELECT * FROM table")

Exporting Data

  1. To a CSV file using write.csv() or write.csv2():
   write.csv(data, "filename.csv", row.names = FALSE)
  1. To an Excel file using writexl package:
   library(writexl)
   write_xlsx(data, "filename.xlsx")
  1. To a text file using write.table():
   write.table(data, "filename.txt", sep = "\t", row.names = FALSE)
  1. To a JSON file using jsonlite package:
   library(jsonlite)
   toJSON(data, pretty = TRUE) %>% writeLines("filename.json")
  1. To a database using the appropriate function from DBI and the DBMS package:
   library(RMySQL)
   con <- dbConnect(MySQL(), user = "user", password = "password", dbname = "database")
   dbWriteTable(con, "table", data, overwrite = TRUE) # Write data to a MySQL table

These are just some of the common ways to import and export data in R. Depending on your specific requirements and the format of your data, you might need to explore other packages and functions.

Practise this on Doubtly (free)