In this post, let us explore:
- What is the difference between Feature Selection, Feature Extraction, Feature Engineering and Feature Learning
- Process of Feature Engineering
- And examples of Feature Engineering
Both Feature engineering and feature extraction are similar: both refer to creating new features from the existing features. Feature engineering refers to creating a new feature when we could have used the raw feature as well whereas feature extraction is creating new features when we can’t use raw data in the analysis such as converting image to RGB values.
Comparing Feature Selection, Feature Extraction, Feature Engineering and Feature Learning
Comparison
Feature Engineering requires domain knowledge. Given that you are using the proper model, when correct feature engineering is used, model performance increases tremendously. Prof. Andrew Ng calls machine learning as largely feature engineering.
Feature engineering is a trial and error process. Hence it may take longer time, but the advantage is that more we try, higher is the chance of building a model with higher accuracy. Feature engineering requires understanding of relationship between predictor variables and target variables.
Proper feature engineering is a critical step in winning data science competitions and it separates winners from others.
Process
Process
This process continues till we attain satisfactory performance of the model.
Examples
- Creating ‘working days’ feature from the date column
- Using log transformation in financial analysis
- Binarization of ‘age’ variable to underage and adult
- Binning ‘age’ variable to underage, adult and senior citizen
Feature engineering requires critical thinking to understand in which way the predictor variables influence dependent variable.
Summary
In this post, we have explored:
- Compared Feature Selection, Feature Extraction, Feature Engineering and Feature Learning
- Process of Feature Engineering
- Examples of Feature Engineering
References
https://towardsdatascience.com/unsupervised-feature-learning-46a2fe399929
https://towardsdatascience.com/feature-engineering-for-machine-learning-3a5e293a5114
https://en.wikipedia.org/wiki/Feature_learning
https://www.kaggle.com/kashnitsky/topic-6-feature-engineering-and-feature-selection