Backward Elimination Explained

Backward Elimination :

Backward eliminations is one of the methods used to eliminate those features or independent variables which have no effect on the dependent variable(Y). It is done to prevent from doing extra processing on useless features which depreciates the learning of the model.

Backwards elimination has an advantage over forward selection and stepwise regression because it is possible for a set of variables to have considerable predictive capability even though any subset of them does not. Forward selection and stepwise regression will fail to identify them. Because the variables don't predict well individually, they will never get to enter the model to have their joint behavior noticed. Backwards elimination starts with everything in the model, so their joint predictive capability will be seen.

The process we will follow is as explained below :

1. Put all the columns of independent variable to a linear model separately.
2. Consider the Independent variable column With Pvalue(probability value) > SL (significance level[.5]), if Pval > SL the independent variable supports the null hypothesis which states that the specific feature or variable has no influence on the output or dependent variable.
3. Remove the independent variable with highest Pvalue > .5, if no such independent variable column left then stop and exit() as you will be left with the feature or independent variables which tend to effect the output , or go to step 2.

At last you will be left with columns of independent variables whic truly influence the output dependent variable.

For Pvalue and null hypothesis watch Pvalue and null hypothesis explained

The above videos will clear your concepts of Pvalue and null hypothesis
some articles on pvalue and null hypothesis :

https://statistics.laerd.com/statistical-guides/hypothesis-testing-3.php

Forward elimination coming soon ...