Suppose you apply bagging and boosting to a hypothesis space

Suppose you apply bagging and boosting to a hypothesis space of linear separators. Will the hypothesis space of the ensemble still be linear for boosting? For bagging?

Solution

Both bagging and boosting are classified to be voting methods.This is because ,they give learners a weighted vote on what the output should be.

Bagging which is short for bootstrap aggregating, takes multiple learners.It trains each of them on a sample of the data. These samples of data should be taken with replacement and is must also be roughly the same size as the original data set.Due to this averaging, bagging improves unstable learners.

Boosting also like bagging combines multiple learners.This works in a more sophisticated manner. Boosting is less constrained than bagging. It is also defined to be bagging while changing the distribution of the training set. The idea behind it is to combine multiple weak rules into one strong rule.

Thus , voting methods such as bagging and boosting increase the representational capacity of the hypothesis space, by introducing the hypotheses of multiple machine learners.