Decision trees and ensembles quiz

Question 1

What is a decision tree in machine learning?

Accepted Answer

A model that splits data into branches based on feature tests, ending in leaf predictions

Answer

A neural network with many hidden layers

Answer

An algorithm that only works on images

Answer

A method that requires no training data

Question 2

What do the internal nodes and leaf nodes of a decision tree represent?

Accepted Answer

Internal nodes test features; leaf nodes give predictions

Answer

Internal nodes give predictions; leaf nodes store raw data

Answer

Both store the dataset

Answer

Leaf nodes test features; internal nodes give predictions

Question 3

What is Gini impurity and what does a value of 0 mean?

Accepted Answer

A node where all samples belong to one class (pure node)

Answer

A node with maximum class mixing

Answer

A node containing no samples

Answer

A node that must be split further

Question 4

How is Gini impurity calculated for a node with class probabilities p1, p2, ..., pk?

Accepted Answer

1 - Σ p_i²

Answer

Σ p_i log p_i

Answer

Σ p_i²

Answer

1 - Σ p_i

Question 5

What is entropy in the context of decision trees, and what is its formula?

Accepted Answer

-Σ p_i log2(p_i)

Answer

1 - Σ p_i²

Answer

Σ p_i²

Answer

Σ (p_i - mean)²

Question 6

What is information gain and how is it used when building a decision tree?

Accepted Answer

Parent impurity minus weighted child impurity, maximized to choose splits

Answer

The total number of leaves in the tree

Answer

The depth of the deepest branch

Answer

The number of features in the dataset

Question 7

Why does a fully grown, unpruned decision tree tend to overfit the training data?

Accepted Answer

It memorizes noise by creating overly specific rules, causing high variance

Answer

It always underfits because it is too simple

Answer

It cannot represent nonlinear boundaries

Answer

It ignores most of the features

Question 8

What is pruning in decision trees and why is it used?

Accepted Answer

Removing weak branches to reduce overfitting and simplify the tree

Answer

Adding more branches to increase accuracy

Answer

Randomly deleting training rows

Answer

Converting the tree into a neural network

Question 9

What hyperparameters can control the complexity of a decision tree to prevent overfitting?

Accepted Answer

max_depth, min_samples_leaf, min_samples_split

Answer

learning_rate and momentum only

Answer

number of epochs and batch size

Answer

activation function and dropout

Question 10

How does a decision tree handle a regression problem?

Accepted Answer

Each leaf predicts the mean target value; splits minimize variance/MSE

Answer

It outputs class probabilities

Answer

It fits a straight line through all points

Answer

It cannot be used for regression

Question 11

What is ensemble learning and why is it often more accurate than a single model?

Accepted Answer

Combining multiple models so their errors cancel, improving accuracy

Answer

Using a single very deep tree

Answer

Training one model many times on the same data

Answer

Reducing the dataset to one feature

Question 12

What is bagging (bootstrap aggregating) and what problem does it primarily address?

Accepted Answer

Training models on bootstrap samples and aggregating; reduces variance

Answer

Sequentially correcting errors; reduces bias

Answer

Training a single model on all data

Answer

Removing features one at a time

Question 13

What is a bootstrap sample in bagging?

Accepted Answer

A same-size sample drawn with replacement from the training data

Answer

A sample of held-out test data

Answer

The full dataset used without changes

Answer

A sample of the features only, not rows

Question 14

What is a random forest and how does it differ from plain bagging of decision trees?

Accepted Answer

It adds random feature selection at each split to decorrelate trees

Answer

It uses only one tree

Answer

It boosts trees sequentially

Answer

It uses no randomness at all

Question 15

Why does considering only a random subset of features at each split improve a random forest?

Accepted Answer

It decorrelates the trees, so averaging reduces variance more

Answer

It speeds up prediction only

Answer

It increases the bias of each tree on purpose

Answer

It guarantees each tree is identical

Question 16

What is the out-of-bag (OOB) error in a random forest?

Accepted Answer

An error estimate using samples each tree did not train on

Answer

The training error of the deepest tree

Answer

The error on a manually held-out test set

Answer

The error before any trees are built

Question 17

What is boosting and how does it differ fundamentally from bagging?

Accepted Answer

Boosting trains models sequentially to fix prior errors; bagging trains in parallel

Answer

Boosting and bagging are identical

Answer

Boosting uses only one model

Answer

Bagging is always sequential

Question 18

How does AdaBoost adjust the training data between rounds?

Accepted Answer

It increases weights of misclassified samples so later learners focus on them

Answer

It removes misclassified samples

Answer

It gives all samples equal weight forever

Answer

It only changes the features, not weights

Question 19

What is the core idea of gradient boosting?

Accepted Answer

Each new tree fits the residual errors of the current ensemble

Answer

Each tree is trained on a random feature only

Answer

All trees are averaged with equal weight

Answer

It trains all trees fully independently

Question 20

What role does the learning rate play in gradient boosting?

Accepted Answer

It scales each tree's contribution; smaller values need more trees but generalize better

Answer

It sets the maximum tree depth

Answer

It controls the number of features per split

Answer

It has no effect on the final model

Question 21

What are 'weak learners' in boosting and why are they used?

Accepted Answer

Simple models slightly better than chance, combined into a strong model

Answer

Very deep, fully grown trees

Answer

Models that always overfit

Answer

Models with zero training error

Question 22

What is a decision stump?

Accepted Answer

A depth-1 decision tree with a single split

Answer

A tree with no splits

Answer

A fully grown deep tree

Answer

A random forest with one feature

Question 23

How does a random forest estimate feature importance?

Accepted Answer

By summing each feature's impurity reduction across all trees

Answer

By counting how many leaves exist

Answer

By the alphabetical order of features

Answer

By the size of the dataset

Question 24

Why are decision trees considered highly interpretable models?

Accepted Answer

They form readable if-then rules that explain each prediction

Answer

They use hidden weights that cannot be inspected

Answer

They require advanced math to read

Answer

They give no explanation for predictions

Question 25

Do decision trees require feature scaling (normalization) like many other algorithms? Why or why not?

Accepted Answer

No, because splits use thresholds on individual features, unaffected by scale

Answer

Yes, they require standardization always

Answer

Yes, otherwise they cannot split

Answer

No, but only if data is already normalized

Question 26

What type of decision boundary does a decision tree produce?

Accepted Answer

Axis-aligned, rectangular (staircase) boundaries

Answer

Perfectly smooth circles

Answer

A single straight diagonal line

Answer

Curved exponential boundaries

Question 27

In a classification decision tree, how is the prediction made at a leaf node?

Accepted Answer

The majority class of training samples in that leaf

Answer

The average of all features

Answer

A random class

Answer

The first class seen during training

Question 28

How do random forests combine the predictions of their individual trees?

Accepted Answer

Majority vote for classification, averaging for regression

Answer

It always uses only the first tree

Answer

It multiplies all predictions together

Answer

It picks a random tree's output

Question 29

What is the bias-variance tradeoff, and how do bagging and boosting each relate to it?

Accepted Answer

Bagging reduces variance; boosting reduces bias

Answer

Bagging reduces bias; boosting reduces variance

Answer

Both only reduce bias

Answer

Neither affects bias or variance

Question 30

Why can boosting be more prone to overfitting noisy data than bagging?

Accepted Answer

Boosting keeps focusing on hard/noisy points, which it may end up fitting

Answer

Boosting ignores all difficult points

Answer

Bagging always overfits more than boosting

Answer

Neither method can ever overfit

Test yourself