INTRODUCTION – Multiple Linear Regression
From the simple basics on simple regression, participants will now advance into the more complicated world of multiple-linear regression, where there will be many very important variables combined together to make a good prediction model. The simple principles of linear regression have been the starting point for this section, and it is a step-by-step guide into modeling in a way that demonstrates the advancement and the continuum between these approaches.
Participants shall now have adequately using simulation, a systematic interdisciplinary exercise that will develop the conceptual understanding of multiple regression in the model development process. This main foundation also introduces learners to the key fundamental advanced machine learning concepts including variable selection, overfitting, and the bias-variance tradeoff. This base-formative approach not only emphasizes the whole-aided learning experience but also prepares learners for the much broader machine learning applications. This covers the practical meetings and theory that will hold these participants in their stride against multiple linear regression and some core learning principles around machine learning.
Learning Objectives
- Define Ridge and Lasso Regression.
- Apply variable selection techniques.
- Statistical power and its relation to variable selection are defined and explained.
- Identify and define interaction terms in multiple regression.
- Explore the possibility of interaction terms enhancing model performance.
- Use exploratory data analyses (EDA) to:
Determine if a certain scenario applies to multiple regression based on model assumptions. - Identify and protect from multicollinearity.
- Define homoscedasticity and heteroscedasticity in a regression analysis.
- Extend assumptions of simple linear regression to multiple regression models.
- Define one-hot encoding for dealing with categorical independent variables.
- Distinguish between simple and multiple linear regressions.
PRACTICE QUIZ: TEST YOUR KNOWLEDGE: UNDERSTAND MULTIPLE LINEAR REGRESSION
1. Fill in the blank: _____ is a technique that estimates the linear relationship between one continuous dependent variable and two or more independent variables.
- Singular linear regression
- Multiple curved regression
- Singular curved regression
- Multiple linear regression (CORRECT)
Correct: A statistical tool for multiple linear regression determines the relation of a continuous dependant variable and many other independent variables. With more than two co-variables, a complete study into the co-factors influencing the likelihood can be carried out. Furthermore, the applicability of multiple linear regression is in its capacity to yield interpretable and communicable results; thus, it is considered a great data analysis and decision-making tool.
2. What concept refers to how two independent variables together affect the y dependent variable?
- One hot encoding
- Interaction terms (CORRECT)
- Ordinary least squares
- Confidence band
PRACTICE QUIZ: TEST YOUR KNOWLEDGE: MODEL ASSUMPTIONS REVISITED
1. Which of the following statements is true? Select all that apply.
- One hot encoding is for ordinal variables.
- One hot encoding allows data professionals to turn several categorical variables into one binary variable.
- One hot encoding is a data transformation technique. (CORRECT)
- One hot encoding allows data professionals to turn one categorical variable into several binary variables. (CORRECT)
Correct: A method of data transformation called one-hot encoding can be applied for converting categorical variables into multiple binary variables. This method creates a corresponding binary column for each category and assigns a value 1 at the category and 0 elsewhere. It can be said that such coding is a requirement for using categorical data efficiently within machine learning models and statistical analyses.
2. What is the definition of the no multicollinearity assumption?
- No predictor variable can be linearly related to the outcome variable.
- No two independent variables can be highly correlated with each other. (CORRECT)
- No observation in the dataset can be independent.
- Variation of the residual must be constant or similar across the model.
Correct: Multicollinearity occurs when there are two or more independent variables, which are highly correlated among themselves. This means that there will be one independent variable:
3. In what ways might a data professional handle data with multicollinearity? Select all that apply.
- Square the variables that have high multicollinearity.
- Turn one categorical variable into several binary variables.
- Create new variables using existing data. (CORRECT)
- Drop one or more variables that have high multicollinearity. (CORRECT)
Correct: In order to address the multicollinearity problem, either a data professional would do one of the binary operations or they would create new variables with derived values formed from the existing data.
PRACTICE QUIZ: TEST YOUR KNOWLEDGE: MODEL INTERPRETATION
1. Fill in the blank: An interaction term represents how the relationship between two independent variables is associated with the changes in the _____ of the dependent variable.
- category
- multicollinearity
- mean (CORRECT)
- rate of change
Correct: An interaction term refers to a condition for an effect on the mean of the dependent variable by two independent variables. Generally, data experts denote such an interaction process by the product of two independent variables involved.
2. Which of the following relevant statistics can be found by using statsmodel’s OLS function? Select all that apply.
- Variance inflation factors
- Standard errors (CORRECT)
- Coefficients (CORRECT)
- P-values (CORRECT)
Correct: Using the OLS function in Statsmodels, one can find coefficients, standard errors, p-values, and t-statistics.
PRACTICE QUIZ: TEST YOUR KNOWLEDGE: VARIABLE SELECTION AND MODEL EVALUATION
1. Fill in the blank: Adjusted R squared is a variation of the R squared regression evaluation metric that _____ unnecessary explanatory variables.
- Adds
- Eliminates
- rewards
- penalizes (CORRECT)
Correct: Another version of R-squared is called the adjusted R-squared. It modifies the R-squared metric to compensate for the number of explanatory variables used in a model, thus penalizing unnecessary variables. As in the case of R-squared, it obtains a range of values from less than 0 to 1.
2. Which of the following statements accurately describe the differences between adjusted R squared and R squared? Select all that apply.
- Adjusted R squared is more easily interpretable.
- R squared is used to compare models of varying complexity.
- Adjusted R squared is used to compare models of varying complexity. (CORRECT)
- R squared is more easily interpretable. (CORRECT)
Correct: Unlike R-squared, which is more simple to interpret, that it says in a very straight form the percentage of total variance in the dependent variable that is explained by the model, Adjusted R-squared is much more used for comparing models of different complexities since it is adjusted to the number of predictors involved. The adjusted R-squared value tells whether adding more variables increases or decreases the quality of the model.
Correct: It measures the proportion of variation in the dependent variable explained by the model. In contrast, adjusted R-squared measures are used to compare different complexity models by adjusting the number of predictors in order to avoid overfitting.
3. What variable selection process begins with the full model that has all possible independent variables?
- Forward selection
- Backward elimination (CORRECT)
- Extra-sum-of Squares
- F-test
Correct: Full Model All Candidate Variables Backward Elimination Process for Variable Selection Then Starts Systematically, the Least Significant Variable-at-a-time Is Removed on a Basis of Such Criteria as P-value Comparison, Until Only the Most Significant Variables Are Left.
4. Which of the following are regularized regression techniques? Select all that apply.
- F-test regression
- Elastic-net regression (CORRECT)
- Lasso regression (CORRECT)
- Ridge regression (CORRECT)
Correct: Lasso regression, ridge regression, and elastic-net regression are all regularised regression techniques that apply penalty terms to the regression model.
QUIZ: MODULE 3 CHALLENGE
1. A data team working for an online magazine uses a regression technique to learn about advertising sales in different sections of the publication. They estimate the linear relationship between one continuous dependent variable and four independent variables. What technique are they using?
- Multiple linear regression (CORRECT)
- Simple linear regression
- Interaction regression
- Coefficient regression
2.What technique turns one categorical variable into several binary variables?
- Multiple linear regression
- Overfitting
- One hot encoding (CORRECT)
- Adjusted R squared
3. Which of the following is true regarding variance inflation factors? Select all that apply.
- The larger the variable inflation factor, the less multicollinearity in the model.
- The minimum value is 0.
- The larger the variable inflation factor, the more multicollinearity in the model. (CORRECT)
- The minimum value is 1. (CORRECT)
4. What term represents how the relationship between two independent variables is associated with changes in the mean of the dependent variable?
- Normality term
- Selection term
- Interaction term (CORRECT)
- Coefficient term
5. Which of the following statements accurately describe adjusted R squared? Select all that apply.
- It is greater than 1.
- It is a regression evaluation metric. (CORRECT)
- It can vary from 0 to 1. (CORRECT)
- It penalizes unnecessary explanatory variables. (CORRECT)
6. Which of the following statements accurately describe forward selection and backward elimination? Select all that apply.
- Forward selection begins with the full model with all possible independent variables.
- Forward selection begins with the full model with all possible dependent variables.
- Forward selection begins with the null model and zero independent variables. (CORRECT)
- Backward elimination begins with the full model with all possible independent variables. (CORRECT)
7. A data professional reviews model predictions for a human resources project. They discover that the model performs poorly on both the training data and the test holdout data, consistently predicting figures that are too low. This leads to inaccurate estimates about employee retention. What quality does this model have too much of?
- Bias (CORRECT)
- Entropy
- Variance
- Leakage
8. What regularization technique completely removes variables that are less important to predicting the y variable of interest?
- Elastic net regression
- Independent regression
- Lasso regression (CORRECT)
- Ridge regression
9. A data team with a restaurant group uses a regression technique to learn about customer loyalty and ratings. They estimate the linear relationship between one continuous dependent variable and two independent variables. What technique are they using?
- Coefficient regression
- Simple linear regression
- Interaction regression
- Multiple linear regression (CORRECT)
10. A data professional confirms that no two independent variables are highly correlated with each other. Which assumption are they testing for?
- No multicollinearity assumption (CORRECT)
- No linearity assumption
- No normality assumption
- No homoscedasticity assumption
11. What term represents the relationship for how two variables’ values affect each other?
- Underfitting term
- Linearity term
- Interaction term (CORRECT)
- Feature selection term
12. Which regression evaluation metric penalizes unnecessary explanatory variables?
- Holdout sampling
- Adjusted R squared (CORRECT)
- Overfitting
- Regression sampling
13. A data professional tells you that their model fails to adequately capture the relationship between the target variable and independent variables because it has too much bias. What is the most likely cause of the bias?
- Underfitting (CORRECT)
- Overfitting
- Leakage
- Entropy
14. What regularization technique minimizes the impact of less relevant variables, but drops none of the variables from the equation?
- Lasso regression
- Forward regression
- Elastic net regression
- Ridge regression (CORRECT)
15. Fill in the blank: The no multicollinearity assumption states that no two _____ variables can be highly correlated with each other.
- Dependent
- categorical
- independent (CORRECT)
- continuous
16. Fill in the blank: An interaction term represents how the relationship between two independent variables is associated with changes in the _____ of the dependent variable.
- category
- multicollinearity
- assumption
- mean (CORRECT)
17. A data professional uses an evaluation metric that penalizes unnecessary explanatory variables. Which metric are they using?
- Link function
- Adjusted R squared (CORRECT)
- Ordinary least squares
- Holdout sampling
18. What stepwise variable selection process begins with the full model with all possible independent variables?
- Forward selection
- Backward elimination (CORRECT)
- Extra-sum-of-squares F-test
- Overfit selection
19. A data analytics team creates a model for a project supporting their company’s sales department. The model performs very well on the training data, but it scores much worse when used to predict on new, unseen data. What does this model have too much of?
- Entropy
- Bias
- Leakage
- Variance (CORRECT)
20. A data professional at a car rental agency uses a regression technique to learn about how customers engage with various sections of the company website. They estimate the linear relationship between one continuous dependent variable and three independent variables. What technique are they using?
- One hot encoding
- Multiple linear regression (CORRECT)
- Simple linear regression
- Interaction terms
21. Which of the following are examples of categorical variables? Select all that apply.
- Shirt inventory
- Shirt country of manufacture (CORRECT)
- Shirt type (CORRECT)
- Shirt size (CORRECT)
22. Fill in the blank: One hot encoding is a data transformation technique that turns one categorical variable into several _____ variables.
- independent
- dependent
- overfit
- binary (CORRECT)
23. What stepwise variable selection process begins with the null model and zero independent variables?
- Backward elimination
- Holdout elimination
- Forward selection (CORRECT)
- Extra-sum-of-squares F-test
24. What data transformation technique turns one categorical variable into several binary variables?
- Label encoding
- Multiple regression
- One hot encoding (CORRECT)
- Adjusted R squared
Correct: One-hot encoding is a term used in data transformation as it divides one categorical variable into several binary variables- and thus converts the categorical variable into a numerical one. Such one-hot encoding is used by data professionals dealing with a categorical independent variable, converting each category as a separate binary feature.
25. Fill in the blank: The _____ states that no two independent variables (X−i and X−j) can be highly correlated with each other.
- no linearity assumption
- no homoscedasticity assumption
- no normality assumption
- no multicollinearity assumption (CORRECT)
Correct: The no multicollinearity assumption states that two independent variables (X₋ᵢ and X₋ⱼ) should not be significantly correlated with each other. Or simply put that X₋ᵢ and X₋ⱼ must not show strong linear relationship as it may distort the model’s estimate.
26. Fill in the blank: An interaction term represents the relationship between two independent variables and the change in the mean of the _____ variable.
- global
- instance
- dependent (CORRECT)
- independent
Correct: In other words, an interaction term measures to what extent the change in the mean dependent variable by one independent variable is affected by the other independent variable. Thus, in most cases, the form of an interaction term involves multiplying the two independent variables.
27. What is the process of determining which variables or features to include in a given model?
- Backward elimination
- Extra-sum-of-squares F-test
- Forward selection
- Variable selection (CORRECT)
Correct: It presents one of the procedures called variable selection. It indicates in its own sense as to which variables or features to include in the model. The process is repetitive, going into iterations in order to determine the most important variables applicable in the model’s performance.
CONCLUSION – Multiple Linear Regression
Basically, this section is an important part of the Google Advanced Data Analytics Certificate program. Here, learners would have progressed from simple linear regression toward the more complex area of multiple linear regression. This section would help learners learn how to bring in multiple variables into their models for a complete understanding of regression analysis. The learners would also be equipped with an introductory thrust towards concepts of machine learning, equipping them with the fundamental skills needed to traverse the vast domain of advanced analytics.
The theory becomes solidified in practice with an opportunity for participants to get hands-on experience in conducting and modeling regression on real-life data problems. Obviously, this is going to show quite a good reflection of the importance of this course section as a part of the wider course: it enables the learners to use their regression knowledge and experience with machine learning to grab insights from diverse datasets.