Supervised learning algorithms quiz

Question 1

What fundamentally distinguishes supervised learning from unsupervised learning?

Accepted Answer

Supervised learning uses labeled input-output pairs; unsupervised does not

Answer

Supervised learning only works on images

Answer

Unsupervised learning always needs more data

Answer

Supervised learning never uses a loss function

Question 2

What is the difference between a classification task and a regression task in supervised learning?

Accepted Answer

Classification predicts categories; regression predicts continuous values

Answer

Classification is unsupervised; regression is supervised

Answer

Regression only works with two classes

Answer

Classification always outputs probabilities between 0 and 100

Question 3

In linear regression, what does the model assume about the relationship between features and the target?

Accepted Answer

The target is a linear combination of features plus an intercept

Answer

The features are always independent of each other

Answer

The target must be a probability

Answer

The relationship is always exponential

Question 4

Why is Mean Squared Error (MSE) a common loss function for regression, and how is it computed?

Accepted Answer

It averages squared prediction errors and penalizes large errors strongly

Answer

It counts how many predictions are exactly correct

Answer

It measures only the maximum error

Answer

It is used only for classification

Question 5

Despite its name, why is logistic regression considered a classification algorithm?

Accepted Answer

It models class probabilities via the sigmoid and assigns labels by threshold

Answer

It predicts continuous house prices

Answer

It clusters data without labels

Answer

It cannot output probabilities

Question 6

What is the role of the sigmoid function in logistic regression?

Accepted Answer

It maps any real number into the range (0,1) to represent a probability

Answer

It makes the output exactly 0 or 1 only

Answer

It removes the need for weights

Answer

It converts probabilities into class counts

Question 7

What is overfitting in supervised learning, and how can you recognize it?

Accepted Answer

Low training error but high test error due to memorizing noise

Answer

Equal error on training and test sets

Answer

High error on both training and test sets

Answer

Perfect performance on all future data

Question 8

What is underfitting, and what typically causes it?

Accepted Answer

A too-simple model with high error on both training and test data

Answer

A model that memorizes the training set

Answer

A model that only fails on the test set

Answer

A model with zero training error

Question 9

Explain the bias-variance tradeoff in supervised learning.

Accepted Answer

A balance where reducing bias often increases variance and vice versa

Answer

A rule that bias and variance always rise together

Answer

A method to remove all error

Answer

A way to label data faster

Question 10

Why must data be split into training and test sets, and what is the test set used for?

Accepted Answer

To estimate generalization on data the model never saw during training

Answer

To give the model more examples to memorize

Answer

To increase the training accuracy

Answer

To replace the loss function

Question 11

How does a decision tree make a prediction for a new example?

Accepted Answer

It follows feature-based splits from root to a leaf and uses the leaf's value

Answer

It averages all training labels

Answer

It computes a sigmoid of the inputs

Answer

It finds the nearest cluster center

Question 12

What criterion do classification decision trees commonly use to choose splits, and what does it measure?

Accepted Answer

Gini impurity or entropy, measuring class mixing within a node

Answer

Mean squared error of probabilities only

Answer

The number of features used

Answer

The depth of the tree only

Question 13

Why are individual decision trees prone to overfitting, and how is this often controlled?

Accepted Answer

Deep trees memorize noise; limit depth, set min samples, or prune

Answer

Trees never overfit

Answer

Adding more leaves always helps generalization

Answer

Pruning increases training error to zero

Question 14

How does a Random Forest improve upon a single decision tree?

Accepted Answer

It averages many diverse trees to reduce variance and overfitting

Answer

It builds one extremely deep tree

Answer

It removes the need for labels

Answer

It always uses all features in every split

Question 15

What is bagging (bootstrap aggregating) and why does it help?

Accepted Answer

Training models on bootstrap samples and combining them to reduce variance

Answer

Training one model on all data repeatedly

Answer

Reducing bias by simplifying each model

Answer

Removing duplicate rows before training

Question 16

How does the k-Nearest Neighbors (k-NN) algorithm classify a new point?

Accepted Answer

By majority vote among the k closest stored training examples

Answer

By fitting a line through the data

Answer

By computing entropy at each node

Answer

By averaging all training labels

Question 17

Why is feature scaling important for distance-based algorithms like k-NN?

Accepted Answer

Unscaled large-range features dominate distance calculations

Answer

Scaling adds more training data

Answer

Scaling converts regression into classification

Answer

Distance methods ignore feature values entirely

Question 18

What does the parameter k control in k-NN, and what happens at extreme values?

Accepted Answer

k sets how many neighbors vote; small k overfits, large k underfits

Answer

k is the number of features used

Answer

k controls the learning rate

Answer

Larger k always improves accuracy

Question 19

What is the core idea behind a Support Vector Machine (SVM) for classification?

Accepted Answer

Find the separating hyperplane with the maximum margin between classes

Answer

Cluster points without using labels

Answer

Average the nearest neighbors

Answer

Minimize tree depth

Question 20

What is the 'kernel trick' in SVMs and why is it useful?

Accepted Answer

It enables nonlinear boundaries via implicit high-dimensional mapping

Answer

It removes the need for support vectors

Answer

It always uses a linear boundary

Answer

It reduces the number of features to one

Question 21

Why is accuracy a misleading metric for highly imbalanced classification problems?

Accepted Answer

A trivial majority-class predictor can score high accuracy yet miss rare cases

Answer

Accuracy is undefined for imbalanced data

Answer

Accuracy always equals recall

Answer

Imbalance makes accuracy lower than F1 always

Question 22

Define precision and recall in binary classification.

Accepted Answer

Precision = TP/(TP+FP); Recall = TP/(TP+FN)

Answer

Precision = TP/(TP+FN); Recall = TP/(TP+FP)

Answer

Both equal accuracy

Answer

Precision counts only negatives

Question 23

What is the F1-score and when is it preferred over accuracy?

Accepted Answer

The harmonic mean of precision and recall, useful for imbalanced data

Answer

The average of accuracy and recall

Answer

The square of precision

Answer

The number of correct predictions

Question 24

What information does a confusion matrix provide in classification?

Accepted Answer

Counts of TP, TN, FP, and FN showing the types of errors made

Answer

Only the overall accuracy number

Answer

The model's learning rate

Answer

The number of features used

Question 25

What is gradient descent and how does it train models like linear/logistic regression?

Accepted Answer

It iteratively updates parameters opposite the loss gradient to minimize loss

Answer

It randomly guesses parameters until the loss is zero

Answer

It sorts the training data by label

Answer

It selects the best features automatically

Question 26

What is the effect of the learning rate in gradient descent if it is too large or too small?

Accepted Answer

Too large overshoots/diverges; too small converges very slowly

Answer

Too large always converges fastest with no risk

Answer

Learning rate has no effect on convergence

Answer

Too small causes divergence

Question 27

Why is regularization (e.g., L1 or L2) used in supervised learning?

Accepted Answer

It penalizes large weights to reduce overfitting and complexity

Answer

It increases model capacity to fit noise

Answer

It removes the loss function

Answer

It always improves training accuracy

Question 28

What is the key practical difference between L1 (Lasso) and L2 (Ridge) regularization?

Accepted Answer

L1 can zero out coefficients (feature selection); L2 only shrinks them

Answer

L2 produces sparse models; L1 never does

Answer

Both always zero out coefficients

Answer

Neither affects coefficient size

Question 29

What is k-fold cross-validation and why is it useful?

Accepted Answer

Rotating train/validation folds k times and averaging for a reliable estimate

Answer

Training the model once on all data

Answer

Splitting features into k groups

Answer

Removing k percent of the data permanently

Question 30

What is the difference between a model parameter and a hyperparameter?

Accepted Answer

Parameters are learned from data; hyperparameters are set before training

Answer

Both are learned automatically during training

Answer

Hyperparameters are the model's weights

Answer

Parameters are chosen manually before training

Test yourself