Ensemble methods (bagging, boosting, stacking) quiz

Question 1

What is an 'ensemble method' in machine learning?

Accepted Answer

Combining multiple models to make one stronger prediction

Answer

Training a single very deep neural network

Answer

Cleaning and normalizing the dataset before training

Answer

Reducing the number of features using PCA

Question 2

What does 'bagging' stand for, and what is its core idea?

Accepted Answer

Bootstrap Aggregating

Answer

Backward Gradient Generation

Answer

Balanced Augmented Grouping

Answer

Batch Averaged Gain

Question 3

In bagging, what is a 'bootstrap sample'?

Accepted Answer

A random sample drawn with replacement from the training data

Answer

The first 80% of the dataset used for training

Answer

A sample containing only the misclassified points

Answer

A small held-out validation set

Question 4

Bagging primarily reduces which component of a model's error: bias or variance?

Accepted Answer

Variance

Answer

Bias

Answer

Irreducible noise

Answer

Both bias and noise equally

Question 5

What is a Random Forest and how does it extend bagging?

Accepted Answer

Bagged decision trees plus random feature selection at each split

Answer

A single very deep decision tree

Answer

A boosting method that weights misclassified samples

Answer

A neural network with many hidden layers

Question 6

What is the core idea behind 'boosting'?

Accepted Answer

Training models sequentially, each correcting the previous one's errors

Answer

Training all models independently in parallel and averaging

Answer

Combining models of different types using a meta-model

Answer

Selecting the single best model via cross-validation

Question 7

What is the key difference between bagging and boosting in how base models are trained?

Accepted Answer

Bagging trains models in parallel; boosting trains them sequentially

Answer

Bagging is sequential; boosting is parallel

Answer

Both train models sequentially with equal weights

Answer

Both train one model on the full dataset

Question 8

What does 'stacking' (stacked generalization) mean?

Accepted Answer

Using a meta-model to combine predictions of several base models

Answer

Stacking many copies of the same tree in a row

Answer

Repeatedly reweighting misclassified points

Answer

Stacking data rows from multiple datasets

Question 9

In stacking, what role does the 'meta-learner' play?

Accepted Answer

It learns how to combine the base models' predictions

Answer

It generates the bootstrap samples

Answer

It selects which features each base model sees

Answer

It computes the gradients for boosting

Question 10

Why does combining diverse models in an ensemble usually improve performance?

Accepted Answer

Their uncorrelated errors tend to cancel out when combined

Answer

Combining always increases each model's individual accuracy

Answer

It reduces the dataset size needed for training

Answer

It eliminates the need for any feature engineering

Question 11

What is a 'weak learner' in the context of boosting?

Accepted Answer

A model only slightly better than random guessing

Answer

A deep neural network that overfits

Answer

The final combined ensemble model

Answer

A model trained on the full dataset with all features

Question 12

How does AdaBoost decide which training examples to focus on next?

Accepted Answer

It increases the weights of misclassified examples

Answer

It removes misclassified examples from the data

Answer

It gives equal weight to all examples every round

Answer

It only trains on correctly classified examples

Question 13

What is the central idea of Gradient Boosting?

Accepted Answer

Each new model fits the residual errors of the current ensemble

Answer

Each model is trained on an independent bootstrap sample

Answer

All models are averaged with equal weight

Answer

A meta-model blends predictions of unrelated models

Question 14

What is the role of the 'learning rate' in gradient boosting?

Accepted Answer

It scales how much each new tree contributes to the ensemble

Answer

It sets the number of trees in the forest

Answer

It controls the bootstrap sample size

Answer

It chooses the meta-learner type

Question 15

What is 'majority voting' (hard voting) in an ensemble classifier?

Accepted Answer

The ensemble predicts the class chosen by most base models

Answer

The ensemble averages predicted probabilities

Answer

The ensemble picks the most confident single model

Answer

The ensemble multiplies all predictions together

Question 16

What is the difference between 'hard voting' and 'soft voting'?

Accepted Answer

Hard voting uses class labels; soft voting averages probabilities

Answer

Hard voting averages probabilities; soft voting uses labels

Answer

They are two names for the same method

Answer

Soft voting only works for regression

Question 17

What are 'out-of-bag' (OOB) samples in bagging?

Accepted Answer

Training points left out of a model's bootstrap sample

Answer

Outliers removed before training

Answer

Predictions outside the valid probability range

Answer

Extra synthetic samples added to the data

Question 18

Roughly what fraction of unique original samples appears in a bootstrap sample of the same size?

Accepted Answer

About 63%

Answer

About 100%

Answer

About 50%

Answer

About 10%

Question 19

Which type of base model is bagging most effective with, and why?

Accepted Answer

High-variance models like deep decision trees

Answer

Very high-bias linear models

Answer

Models that already have near-zero variance

Answer

Constant predictors that always output the mean

Question 20

Why is boosting generally more prone to overfitting noisy data than bagging?

Accepted Answer

It keeps focusing on hard/noisy points, fitting the noise

Answer

It uses too few trees by default

Answer

It always averages predictions equally

Answer

It never reweights any examples

Question 21

In stacking, why is it important to generate base-model predictions using cross-validation (out-of-fold predictions)?

Accepted Answer

To avoid data leakage and overfitting in the meta-learner

Answer

To make the base models train faster

Answer

To reduce the number of base models needed

Answer

To convert regression into classification

Question 22

What is XGBoost and why is it popular?

Accepted Answer

An optimized, regularized gradient-boosting tree library

Answer

A bagging-only random forest variant

Answer

A deep learning framework for images

Answer

A clustering algorithm

Question 23

In a decision tree ensemble, what does 'feature importance' measure?

Accepted Answer

How much each feature contributes to reducing prediction error

Answer

The number of features in the dataset

Answer

The correlation between two features

Answer

The amount of missing data per feature

Question 24

What does 'n_estimators' control in a Random Forest or gradient boosting model?

Accepted Answer

The number of base trees in the ensemble

Answer

The maximum depth of each tree

Answer

The fraction of features used per split

Answer

The learning rate of the model

Question 25

Why might a single decision tree overfit while a Random Forest of the same trees does not?

Accepted Answer

Averaging many diverse trees cancels out their noise-fitting

Answer

The forest uses far fewer features overall

Answer

Forest trees are always shallower by definition

Answer

The forest ignores the training labels

Question 26

What is 'pasting' and how does it differ from bagging?

Accepted Answer

Pasting samples without replacement; bagging samples with replacement

Answer

Pasting trains models sequentially like boosting

Answer

Pasting uses a meta-learner like stacking

Answer

Pasting uses only one model

Question 27

How do bagging and boosting differ in their main effect on the bias-variance tradeoff?

Accepted Answer

Bagging reduces variance; boosting reduces bias

Answer

Bagging reduces bias; boosting reduces variance

Answer

Both only reduce irreducible error

Answer

Neither affects bias or variance

Question 28

In AdaBoost, how is the influence (weight) of each weak learner in the final prediction determined?

Accepted Answer

By the learner's accuracy: more accurate learners get higher weight

Answer

All learners always get equal weight

Answer

By the order in which they were trained only

Answer

By the size of their bootstrap sample

Question 29

Which ensemble approach typically allows base models to be trained fully in parallel?

Accepted Answer

Bagging

Answer

Boosting

Answer

Sequential stacking

Answer

Gradient boosting

Question 30

What is the difference between a 'level-0' model and a 'level-1' model in stacking?

Accepted Answer

Level-0 are base models; level-1 is the meta-learner combining them

Answer

Level-0 is the meta-learner; level-1 are the base models

Answer

Both levels train on the original features only

Answer

They are two identical copies of one model

Test yourself