Decision boundaries quiz

Question 1

What is a decision boundary in a classification problem?

Accepted Answer

The surface in feature space separating regions predicted as different classes

Answer

The line connecting all training points of one class

Answer

A measure of how accurate the model is on test data

Answer

The slope of the loss function during training

Question 2

For a 2D feature space, what geometric shape is the decision boundary of a basic linear classifier like logistic regression?

Accepted Answer

A straight line

Answer

A circle

Answer

A parabola

Answer

A zig-zag staircase

Question 3

In logistic regression, the decision boundary corresponds to which probability value by default?

Accepted Answer

0.5

Answer

0.0

Answer

1.0

Answer

0.25

Question 4

What does it mean for two classes to be 'linearly separable'?

Accepted Answer

A single straight line can perfectly separate the two classes

Answer

The two classes have the same number of points

Answer

The features are all positive numbers

Answer

The classes overlap completely

Question 5

Why can a single perceptron not solve the XOR problem?

Accepted Answer

XOR is not linearly separable, so a single linear boundary cannot solve it

Answer

XOR has too many features for a perceptron

Answer

Perceptrons cannot handle binary inputs

Answer

XOR requires probability outputs

Question 6

How does a k-Nearest Neighbors (kNN) classifier form its decision boundary?

Accepted Answer

By majority vote of the k nearest neighbors, giving a jagged piecewise boundary

Answer

By fitting a single straight line through the data

Answer

By minimizing a sigmoid loss function

Answer

By computing the average of all training labels

Question 7

In kNN, how does increasing the value of k generally affect the decision boundary?

Accepted Answer

It makes the boundary smoother and less sensitive to noise

Answer

It makes the boundary more jagged and complex

Answer

It has no effect on the boundary

Answer

It always improves test accuracy

Question 8

What kind of decision boundaries do decision trees produce?

Accepted Answer

Axis-aligned rectangular (piecewise) boundaries

Answer

Smooth curved boundaries

Answer

A single diagonal line

Answer

Circular boundaries

Question 9

What is the 'margin' in a Support Vector Machine, and how does it relate to the decision boundary?

Accepted Answer

The distance from the boundary to the nearest points, which SVM maximizes

Answer

The number of misclassified points

Answer

The slope of the boundary line

Answer

The total number of support vectors

Question 10

What are 'support vectors' in an SVM?

Accepted Answer

The points closest to the boundary that determine its position

Answer

The points farthest from the boundary

Answer

All correctly classified points

Answer

The center points of each class

Question 11

How does the kernel trick allow an SVM to create nonlinear decision boundaries?

Accepted Answer

It maps data to a higher dimension where a linear boundary becomes nonlinear in the original space

Answer

It adds more training data automatically

Answer

It removes outliers from the dataset

Answer

It converts classification into regression

Question 12

What effect does an RBF (Gaussian) kernel with a very large gamma have on an SVM's decision boundary?

Accepted Answer

A very wiggly boundary that wraps tightly around points, risking overfitting

Answer

A perfectly straight line

Answer

A boundary that ignores all data

Answer

A smoother, simpler boundary

Question 13

How does adding polynomial features (e.g., x1^2, x2^2, x1*x2) let a linear classifier model curved boundaries?

Accepted Answer

A linear boundary in expanded feature space maps to a curve in the original space

Answer

Polynomial features delete noisy data

Answer

It forces the boundary to stay straight

Answer

It reduces the number of features needed

Question 14

A circular decision boundary x1^2 + x2^2 = r^2 separates inside from outside. Can a linear model in the original features (x1, x2) represent this directly?

Accepted Answer

No, unless squared features are added; raw x1,x2 only give straight lines

Answer

Yes, logistic regression naturally draws circles

Answer

Yes, any linear model handles circles

Answer

No, circles are impossible for any model

Question 15

What is the relationship between the weight vector w of a linear classifier and its decision boundary?

Accepted Answer

w is perpendicular (normal) to the boundary, pointing toward the positive class

Answer

w lies along the boundary line

Answer

w determines the number of classes

Answer

w has no geometric meaning

Question 16

How does lowering the classification threshold (below 0.5) move the decision boundary's effect for logistic regression?

Accepted Answer

More inputs are labeled positive, raising recall but lowering precision

Answer

Fewer inputs are labeled positive

Answer

It retrains the weight vector

Answer

It has no effect on predictions

Question 17

Why do more flexible models tend to produce more complex decision boundaries, and what is the risk?

Accepted Answer

They can fit noise, causing overfitting and poor generalization

Answer

They always generalize better

Answer

They cannot make errors on training data

Answer

Complex boundaries reduce variance

Question 18

How does L2 regularization typically affect a logistic regression decision boundary?

Accepted Answer

It shrinks weights, giving a smoother, less overfit boundary

Answer

It makes the boundary more jagged

Answer

It removes features entirely like a switch

Answer

It has no effect on the boundary

Question 19

In a multiclass problem with three classes, what do the decision boundaries look like for a linear classifier using one-vs-rest?

Accepted Answer

Piecewise-linear boundaries between regions where the top class changes

Answer

A single curved boundary

Answer

Three parallel lines

Answer

No boundaries are formed

Question 20

How can a neural network with a hidden layer and nonlinear activations create complex decision boundaries?

Accepted Answer

Nonlinear activations let layers combine into flexible, complex boundaries

Answer

Each neuron draws one circle

Answer

They can only draw straight lines

Answer

Hidden layers remove the boundary entirely

Question 21

What happens to the decision boundary if you scale one feature to a much larger range than the others (without normalization) in a distance-based model like kNN?

Accepted Answer

The large-range feature dominates distances, distorting the boundary

Answer

Scaling never affects kNN

Answer

It improves the boundary automatically

Answer

All features are weighted equally regardless of scale

Question 22

What is a 'soft margin' in an SVM and why is it useful for decision boundaries?

Accepted Answer

It allows some misclassifications for a more robust boundary on noisy data

Answer

It forbids any errors

Answer

It removes the margin concept

Answer

It only works for regression

Question 23

In an SVM, how does a smaller value of the penalty parameter C affect the decision boundary?

Accepted Answer

A wider margin that tolerates more errors (more general boundary)

Answer

A narrower margin with zero errors

Answer

No change to the margin

Answer

It removes all support vectors

Question 24

Why might two different classifiers achieve the same training accuracy but have very different decision boundaries?

Accepted Answer

Many boundaries fit the same points, but they generalize differently

Answer

Training accuracy uniquely determines the boundary

Answer

They must have identical boundaries

Answer

Boundaries are random and unrelated to accuracy

Question 25

What does the bias term b control in the linear decision boundary w·x + b = 0?

Accepted Answer

It shifts the boundary's position (offset from the origin)

Answer

It rotates the boundary

Answer

It sets the number of classes

Answer

It scales the features

Question 26

How does class imbalance affect where a learned decision boundary ends up?

Accepted Answer

The boundary tends to favor the majority class, neglecting the minority

Answer

Imbalance has no effect on the boundary

Answer

The boundary always favors the minority class

Answer

It makes the boundary perfectly centered

Question 27

What is the geometric meaning of the distance from a point x to the linear decision boundary w·x + b = 0?

Accepted Answer

(w·x + b) / ||w||, whose sign gives the class and magnitude gives confidence

Answer

w·x + b multiplied by ||w||

Answer

Always exactly 1 for every point

Answer

The number of features times b

Question 28

Why is Naive Bayes with Gaussian features and equal variances said to produce a linear decision boundary?

Accepted Answer

Equal variances cancel the quadratic terms, leaving a linear boundary

Answer

Naive Bayes always uses a circle

Answer

It ignores the features entirely

Answer

Probabilities cannot form boundaries

Question 29

How does an ensemble like a random forest's decision boundary differ from that of a single decision tree?

Accepted Answer

The forest's boundary is smoother and more robust due to averaging many trees

Answer

The forest boundary is always a straight line

Answer

A single tree is always smoother

Answer

They produce identical boundaries

Question 30

In scikit-learn, what is a common technique to visualize a 2D decision boundary?

Accepted Answer

Predict over a meshgrid and plot regions with contourf

Answer

Print the model's accuracy score

Answer

Plot a histogram of one feature

Answer

List the model's hyperparameters

Test yourself