Suppose you wish to perform model selection in multiple regr

Suppose you wish to perform model selection in multiple regression but your statistical program can only fit individual multiple regression models and does not do best subset selection, stepwise selection, or backward selection. Which of these three model selection procedures will be easiest to implement by fitting a series of individual multiple regression models? Briefly explain how you would implement one of these three model selection procedures by fitting individual multiple regression models.

Solution

In case of Forward Stepwise Regression model, the alogorithm adds one independent variable(x) at a time, starting with the one which explains most of the variation in Dependent variable(y) and adding one more independent variable (x) to it , rechecking the model to see that both variables form a good model, then adding a third variable if it still adds to the explanation of dependent variable (y) and so on.

In case of Backward Stepwise Regression model , the algorithm gradually eliminates those one after another which donot explain much of the variation in dependent variable (y), untill it ends with an optimal mix of independent variables according to pre-set criteria for the exit of variables.

Stepwise regression is a modification of the forward selection so that after each step in which a variable was added, all candidate variables in the model are checked to see if their significance has been reduced below the specified tolerance level. If a nonsignificant variable is found, it is removed from the model.

Stepwise regression requires two significance levels: one for adding variables and one for removing variables. The cutoff probability for adding variables should be less than the cutoff probability for removing variables so that the procedure does not get into an infinite loop.

The value of coefficient of determination is highest in Stepwise Regression if compared with Forward and Stepwise Regression for any given set of data. So, the selection prosedure has to be stepwise regression.