Project 5
1. Adaboost:
Advantage:
Adaboost is less prone to overfitting as the input parameters are not jointly optimized. The accuracy of weak classifiers can be improved by using Adaboost. Nowadays, Adaboost is being used to classify text and images rather than binary classification problems.
Disadvantage:
The main disadvantage of Adaboost is that it needs a quality dataset. Noisy data and outliers have to be avoided before adopting an Adaboost algorithm.
2.Gradient Boost:
Advantage:
- Lots of flexibility - can optimize on different loss functions and provides several hyper parameter tuning options that make the function fit very flexible.
- No data pre-processing required - often works great with categorical and numerical values as is.
- Handles missing data - imputation not required.
Disadvantage:
- Gradient Boosting Models will continue improving to minimize all errors. This can overemphasize outliers and cause overfitting.
- Computationally expensive - often require many trees (>1000) which can be time and memory exhaustive.
- Less interpretative in nature, although this is easily addressed with various tools.
3.XGBoost:
Advantage:
It works well on small data, data with subgroups, big data, and complicated data. It doesn’t work so well on sparse data, though, and very dispersed data can create some issues, as well. However, it tends to do better than most supervised learning algorithms on those types of data problems.
Disadvantage:
The biggest limitation is probably the black box nature. If you need effect sizes, XGBoost won’t give them to you (though some adaboost-type algorithms can give that to you). You’d have to derive and program that part yourself. Given the models that exist (like penalized GLMs), XGBoost wouldn’t be your go-to algorithm for those use cases.
4.Light GBM:
Advantage:
Light GBM has the advantages of training efficiency, low memory usage, high accuracy, parallel learning, corporate support, and scale-ability.
Disadvantage:
its’ newness is its main disadvantage because there is little community support.
5.Cat Boost:
Advantage:
CatBoost provides state of the art results and it is competitive with any leading machine learning algorithm on the performance front. Handling Categorical features automatically: We can use CatBoost without any explicit pre-processing to convert categories into numbers.
Disadvantage:
The slight difficulty in tuning the parameters to optimize the model for categorical features.