How the root split is chosen
Every candidate threshold on each feature, scored by weighted Gini impurity. The lowest point wins.
feature xfeature y
—
01204563 Advanced Machine Learning · Topic 8
Random Forest Voting Lab
Many slightly different decision trees make one more stable classification together.
RFvote
Depth vs. overfitting
Training accuracy always climbs with depth. Leave-one-out accuracy — refit with each point held out in turn — shows where that stops helping. The 2-class dataset is separable by one rule, so try the 3-class cat/dog/bird scenario above for a curve that actually climbs.
Train: —Leave-one-out: —
Inside this forest
Tinted regions show each leaf’s predicted class; green/red highlight whether that tree predicts the selected point correctly.
Training data
Weekly practice time and quiz score. Click a point to inspect the forest’s vote.
Click empty plot space to add a ◇ test data point; its colour shows the predicted class.
Majority vote
Selected example
Why do the trees differ?Each tree sees a bootstrap sample of rows and randomly chooses one candidate feature at every split.
Read the mechanism
Bootstrap rows make the trees different. Random feature choices reduce the chance that every tree repeats the same split. Voting then averages out some of an individual tree’s instability.
This small teaching dataset uses only two features. A production random forest usually has many more rows, features, and trees.