Wrapper Based Feature Selection Techniques
There are many ways in which features can be selected from a set of predictors to improve the performance of the machine learning models. Among these techniques, a common practice is used to use another machine learning model as a wrapper, trained on the set of predictor combinations to determine which set of predictors performs the best on the given model.
In this article we shall look into some of the popular machine learning algorithms such as Recursive Feature Elimination (RFE), Forward Feature Selection, and Backward Feature Elimination. Since this article is a continuation from our previous article on Filter based Feature Selection techniques, we will use the same running example of the Breast Cancer Wisconsin (Diagnostic) dataset available at the UCI Machine Learning Repository for testing our models.
Wrapper Methods
Wrapper methods determine the importance of each feature based on the performance of a machine learning algorithm. The features are chosen based on the performance of the model trained on those features. The method is repeated for various feature combinations until the best collection of features is discovered.
