41) What are the feature selection methods used to select the proper variables?
Feature selection is an important concept of ML and involves removal of irrelevant or partially relevant features which may negatively impact the performance of our model.
There are three feature selection methods –
1.Filter Methods- This method of selection of features is independent of any ML algorithm and is usually used as a pre-processing step. The feature is chosen on the basis of their statistical tests.
This process involves:
• Pearson’s correlation
• Linear discrimination analysis
Filter methods don’t remove multicollinearity therefore this must be considered before training our data models.
In this method we use a subset of features and train a model using them and supported the inferences that we draw from the previous model, we decide to add or remove features from our subset. Wrapper methods are very labour-intensive, and high-end computers are needed if tons of data analysis are performed with the wrapper method.
• Forward Selection: We test one feature at a time and keep adding them until we get an honest fit.
• Backward Selection: We test all the features and begin removing them to ascertain what works better.
• Recursive Feature Elimination: Recursively looks through all the various features and the way they pair together.
They are a mixture of both filter and wrapper methods and comprise of qualities of both. It’s implemented with algorithms that have their own built-in feature selection methods.
They include LASSO and RIDGE regression methods which have inbuilt penalization functions to scale back overfitting.