Gradient boosting and XGBoost quiz

Question 1

What does the term "boosting" mean in the context of ensemble learning?

Accepted Answer

Sequentially combining weak learners where each corrects the previous errors

Answer

Training many strong models independently and averaging them

Answer

Removing weak features from a dataset before training

Answer

Splitting data into many subsets and training one model per subset

Question 2

What is a "weak learner" in boosting, and what is commonly used as one?

Accepted Answer

A model only slightly better than random guessing, usually a shallow decision tree

Answer

A deep neural network that overfits the data

Answer

A model that always predicts the mean

Answer

A model trained on a single feature only

Question 3

What is the key difference between bagging (like Random Forest) and boosting?

Accepted Answer

Bagging trains models independently in parallel; boosting trains them sequentially to fix errors

Answer

Bagging is sequential while boosting is parallel

Answer

Both train identical models on identical data

Answer

Boosting only works for regression, bagging only for classification

Question 4

What is the core idea behind gradient boosting?

Accepted Answer

Each tree predicts the residual errors (negative gradient) of the current ensemble

Answer

Each tree is trained on a completely random subset and averaged

Answer

All trees are trained simultaneously to minimize variance

Answer

Trees vote and the majority class wins

Question 5

In gradient boosting for regression with squared-error loss, what exactly does each new tree try to fit?

Accepted Answer

The residuals (actual minus current prediction)

Answer

The original target values directly

Answer

A random noise vector

Answer

The squared values of the features

Question 6

Why are the values fit by gradient boosting trees called "pseudo-residuals"?

Accepted Answer

Because they are the negative gradient of the loss, which only equals true residuals for squared error

Answer

Because they are estimated using a random sample

Answer

Because they are always slightly larger than true residuals

Answer

Because they ignore the loss function entirely

Question 7

What is the role of the learning rate (shrinkage) in gradient boosting?

Accepted Answer

It scales each tree's contribution; smaller values need more trees but generalize better

Answer

It controls how many features each tree uses

Answer

It determines the depth of each tree

Answer

It sets the random seed for sampling

Question 8

In gradient boosting, what typically happens to test accuracy if you keep adding more and more trees with a fixed learning rate?

Accepted Answer

Test error improves at first, then plateaus and can rise as the model overfits

Answer

Test error keeps decreasing forever

Answer

Test error immediately increases from the first tree

Answer

The number of trees has no effect on test error

Question 9

What does the abbreviation XGBoost stand for?

Accepted Answer

eXtreme Gradient Boosting

Answer

eXtra Gradient Boosting

Answer

eXtended General Boosting

Answer

eXact Gradient Bagging

Question 10

What is one major improvement XGBoost adds over traditional gradient boosting to reduce overfitting?

Accepted Answer

It adds L1/L2 regularization terms to the objective function

Answer

It removes decision trees entirely

Answer

It trains all trees in parallel instead of sequentially

Answer

It uses neural networks as base learners

Question 11

XGBoost's objective function has two main parts. What are they?

Accepted Answer

A loss term and a regularization (complexity) term

Answer

A bias term and a variance term

Answer

A training term and a testing term

Answer

A gradient term and a momentum term

Question 12

How does XGBoost use second-order information that classic gradient boosting does not?

Accepted Answer

It uses both gradient and Hessian via a second-order Taylor approximation

Answer

It uses only the gradient like classic boosting

Answer

It ignores derivatives and uses random search

Answer

It uses the third derivative of the loss

Question 13

What does the gamma parameter (min_split_loss) control in XGBoost?

Accepted Answer

The minimum loss reduction needed to make a split

Answer

The learning rate of each tree

Answer

The number of trees in the ensemble

Answer

The fraction of rows sampled per tree

Question 14

What does the max_depth hyperparameter do in XGBoost, and why keep it modest?

Accepted Answer

It limits each tree's depth; large values overfit, so keep it modest

Answer

It sets the number of trees; deeper means more trees

Answer

It controls the learning rate decay

Answer

It chooses which loss function to use

Question 15

How does XGBoost handle missing values in the input data?

Accepted Answer

It learns a default split direction and routes missing values there automatically

Answer

It throws an error and stops training

Answer

It replaces all missing values with zero

Answer

It deletes any row containing a missing value

Question 16

What is early stopping in XGBoost and why is it useful?

Accepted Answer

Halting training when the validation metric stops improving for several rounds

Answer

Stopping training after a fixed time limit regardless of performance

Answer

Removing the first few trees after training

Answer

Skipping the slowest features during training

Question 17

What do the parameters subsample and colsample_bytree control in XGBoost?

Accepted Answer

The fraction of rows and the fraction of columns sampled per tree

Answer

The depth and number of trees

Answer

The learning rate and gamma

Answer

The L1 and L2 penalties

Question 18

In XGBoost, what do the reg_lambda and reg_alpha parameters represent?

Accepted Answer

L2 (lambda) and L1 (alpha) regularization on leaf weights

Answer

Learning rates for odd and even trees

Answer

The number of leaves and depth

Answer

Row and column sampling fractions

Question 19

How does AdaBoost differ from gradient boosting?

Accepted Answer

AdaBoost reweights examples; gradient boosting fits the gradient of a loss function

Answer

They are identical algorithms with different names

Answer

AdaBoost is unsupervised; gradient boosting is supervised

Answer

Gradient boosting only uses one tree, AdaBoost uses many

Question 20

What does "feature importance" tell you in a trained gradient boosting model?

Accepted Answer

How much each feature contributed to reducing loss across the trees

Answer

The order in which features appear in the dataset

Answer

The number of missing values in each feature

Answer

The correlation between two features

Question 21

In XGBoost, what does the "gain" of a split represent?

Accepted Answer

The improvement in the objective (loss reduction) from making the split

Answer

The total number of samples in a leaf

Answer

The learning rate applied to the split

Answer

The depth at which the split occurs

Question 22

Why is gradient boosting described as building an "additive" model?

Accepted Answer

Because the final prediction is a sum of all the trees' contributions

Answer

Because it multiplies the outputs of all trees

Answer

Because it adds new features at each step

Answer

Because it averages the predictions equally

Question 23

For binary classification, which loss function does gradient boosting commonly minimize?

Accepted Answer

Logistic / log loss (binary cross-entropy)

Answer

Mean squared error

Answer

Hinge loss only

Answer

Mean absolute error

Question 24

What is the min_child_weight parameter in XGBoost?

Accepted Answer

The minimum sum of instance weights needed in a child for a split

Answer

The minimum learning rate per child node

Answer

The minimum number of trees per class

Answer

The minimum depth of a child node

Question 25

Name three commonly tuned hyperparameters in XGBoost and what they broadly affect.

Accepted Answer

learning_rate, n_estimators, max_depth

Answer

batch_size, epochs, dropout

Answer

kernel, gamma, degree

Answer

window_size, stride, padding

Question 26

Why does a smaller learning rate usually require more trees (estimators)?

Accepted Answer

Each tree adds only a small correction, so more trees are needed to converge

Answer

Smaller learning rate makes each tree useless, so trees are random

Answer

Smaller learning rate increases tree depth automatically

Answer

It does not; learning rate and tree count are unrelated

Question 27

What is one key reason XGBoost is faster than many older gradient boosting implementations?

Accepted Answer

It parallelizes split-finding and uses cache-aware, out-of-core optimizations

Answer

It trains all trees fully in parallel, ignoring order

Answer

It uses fewer data points than other methods

Answer

It skips computing gradients to save time

Question 28

In gradient boosting, is the addition of trees a sequential or parallel process, and why?

Accepted Answer

Sequential, because each tree depends on the residuals of the previous ones

Answer

Parallel, because all trees are independent

Answer

Random, because the order does not matter

Answer

Parallel, because trees vote at the end

Question 29

What is the purpose of the initial prediction (base score) F0 in gradient boosting?

Accepted Answer

A constant baseline (e.g., the mean) that trees improve upon

Answer

A random value chosen each run

Answer

The deepest tree in the ensemble

Answer

The validation accuracy

Question 30

How do LightGBM and CatBoost relate to XGBoost?

Accepted Answer

They are alternative gradient boosting libraries with different optimizations

Answer

They are unrelated deep learning frameworks

Answer

They are preprocessing tools for XGBoost only

Answer

They are clustering algorithms

Test yourself