> For the complete documentation index, see [llms.txt](https://vikram-bajaj.gitbook.io/cs-gy-6923-machine-learning/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://vikram-bajaj.gitbook.io/cs-gy-6923-machine-learning/main-4/types-of-machine-learning/supervised-learning/decision-trees/bias-and-variance.md).

# Bias and Variance

In general, decision trees have low bias and high variance.

To reduce variance, perform **bagging**:

* Take multiple versions of the training set (random subsets)
* Build a separate tree using each subset
* Given a new example to predict, compute the prediction from each tree. Then for classification, predict the majority class. For regression, compute the average of the predictions.

To get a random subset of approx. 2/3 of the total examples (N), sample N times with replacement. Then, eliminate duplicates and the remaining elements form the random subset.

(Also, at each node, consider only a random subsets of attributes to be used for the decision).

This model is known as a **Random Forest**.