SVM kernels explained quiz

Question 1

What is a Support Vector Machine (SVM)?

Accepted Answer

A supervised algorithm that maximizes the margin between classes using an optimal hyperplane

Answer

An unsupervised algorithm that clusters data points by their pairwise distances

Answer

A neural network that uses support layers to boost prediction accuracy

Answer

A tree-based method that splits features at learned support thresholds

Question 2

What is the 'kernel trick' in SVM, and why is it useful?

Accepted Answer

Computing dot products in a high-dimensional transformed space implicitly using K(x, z)

Answer

A method to speed up training by removing non-support vectors before fitting

Answer

A technique that adds controlled noise to features to prevent overfitting

Answer

Splitting the dataset into smaller kernel-sized mini-batches for faster iteration

Question 3

What is the formula for the linear kernel, and when is it the right choice?

Accepted Answer

K(x, z) = x · z

Answer

K(x, z) = exp(−γ||x − z||²)

Answer

K(x, z) = (x · z + c)^d

Answer

K(x, z) = tanh(γ · x · z + c)

Question 4

What is the formula for the RBF (Radial Basis Function) kernel, and what does it measure?

Accepted Answer

K(x, z) = exp(−γ||x − z||²)

Answer

K(x, z) = (x · z)^γ

Answer

K(x, z) = x · z + γ

Answer

K(x, z) = 1 / (1 + γ||x − z||)

Question 5

What does the hyperparameter gamma (γ) control in the RBF kernel?

Accepted Answer

The radius of influence of individual training points in the RBF kernel

Answer

The maximum number of support vectors the model is allowed to use

Answer

The polynomial degree of the decision boundary in feature space

Answer

The strength of the L2 regularization penalty on the weight vector

Question 6

What is the formula for the polynomial kernel, and what does the degree parameter do?

Accepted Answer

K(x, z) = (γ · x · z + c)^d

Answer

K(x, z) = exp(−γ||x − z||²)

Answer

K(x, z) = tanh(γ · x · z + c)

Answer

K(x, z) = x · z / (||x|| · ||z||)

Question 7

What does the C hyperparameter in SVM control?

Accepted Answer

The trade-off between margin size and misclassification penalty

Answer

The number of support vectors to retain from the training set

Answer

The width of the Gaussian kernel in RBF mode

Answer

The degree of the polynomial decision boundary

Question 8

What are support vectors, and why are they important?

Accepted Answer

The data points closest to the decision boundary that define the margin

Answer

All data points used during the SVM training process

Answer

The latent feature vectors extracted from the kernel matrix diagonal

Answer

The data points lying farthest from the decision boundary in each class

Question 9

Why does a linear SVM fail to classify XOR data, and how does a kernel fix this?

Accepted Answer

XOR data is not linearly separable, so no hyperplane can divide the classes

Answer

XOR data has too many redundant features for SVM to converge

Answer

Linear SVM requires normalized data, and XOR labels are not normalized

Answer

Linear SVM cannot handle binary labels encoded as 0 and 1

Question 10

What does Mercer's theorem say about what makes a function a valid kernel?

Accepted Answer

K(x, z) is a valid kernel only if the resulting Gram matrix is positive semi-definite

Answer

Every continuous symmetric function can be used as a valid SVM kernel

Answer

A kernel is valid if it always produces values strictly between 0 and 1

Answer

Kernel functions must be differentiable everywhere to guarantee convergence

Question 11

What is the Gram (kernel) matrix, and what role does it play in SVM training?

Accepted Answer

An n×n matrix where K[i,j] = K(xᵢ, xⱼ) for all pairs of training examples

Answer

A diagonal matrix of kernel weights assigned to each input feature

Answer

The matrix of Euclidean distances between test and training points

Answer

The covariance matrix computed over the input feature dimensions

Question 12

What is the difference between hard-margin and soft-margin SVM?

Accepted Answer

Hard-margin allows no misclassifications; soft-margin permits some violations via slack variables

Answer

Hard-margin uses the RBF kernel; soft-margin uses the linear kernel

Answer

Hard-margin SVM always trains faster than soft-margin SVM

Answer

Hard-margin minimizes the margin width; soft-margin maximizes it

Question 13

How does the RBF kernel implicitly map data into an infinite-dimensional space?

Accepted Answer

Taylor expansion of exp(−γ||x−z||²) yields an infinite sum equivalent to an inner product in infinite dimensions

Answer

RBF creates new features by drawing random projections of the input vectors

Answer

RBF raises the input vector to successively higher powers during each training iteration

Answer

RBF duplicates and rotates each feature dimension until the data becomes linearly separable

Question 14

Which Python code snippet correctly trains a kernel SVM with the RBF kernel using best practices?

Accepted Answer

SVC(kernel='rbf', C=1.0, gamma='scale') inside a StandardScaler pipeline

Answer

SVC(kernel='gaussian') — sklearn uses 'gaussian' as the name for the RBF kernel

Answer

SVC(kernel='rbf') without scaling — scaling distorts kernel distance calculations

Answer

SVC(kernel='radial') — the correct sklearn identifier for the Gaussian kernel

Question 15

Why is feature scaling essential before training an RBF kernel SVM?

Accepted Answer

RBF uses Euclidean distance, so unscaled features with large ranges dominate the kernel

Answer

Scaling ensures all kernel values fall between 0 and 1 as required by SVC

Answer

sklearn's SVC solver requires zero-mean inputs to converge correctly

Answer

Scaling is only required for polynomial kernels, not for RBF

Question 16

What does `gamma='scale'` mean in sklearn's SVC, and why is it preferred over `gamma='auto'`?

Accepted Answer

gamma='scale' sets γ = 1 / (n_features × Var(X)), adapting to features and variance

Answer

gamma='scale' sets γ = 1 / n_samples, scaling by dataset size

Answer

gamma='scale' clips gamma to the range [0, 1] after each training epoch

Answer

gamma='scale' uses gradient descent to automatically learn the optimal gamma

Question 17

For high-dimensional sparse text data (e.g., TF-IDF vectors), which SVM kernel typically performs best and why?

Accepted Answer

Linear kernel — text data is high-dimensional and often already linearly separable

Answer

Polynomial kernel (degree=3) — text has inherent cubic relationships between word frequencies

Answer

Sigmoid kernel — it mimics the neural activation used in modern NLP models

Answer

RBF kernel — it converts sparse text vectors into dense representations automatically

Question 18

What is the `coef0` parameter in polynomial and sigmoid kernels?

Accepted Answer

coef0 is the independent constant term in polynomial and sigmoid kernel formulas

Answer

coef0 is the learning rate used to update kernel parameters during training

Answer

coef0 is a threshold below which kernel similarity values are clipped to zero

Answer

coef0 is the coefficient scaling the C regularization penalty in soft-margin SVM

Question 19

Why is the sigmoid kernel not always a valid Mercer kernel?

Accepted Answer

The sigmoid kernel's Gram matrix is only positive semi-definite for specific parameter values

Answer

The tanh function outputs values outside [−1, 1] for large inputs, causing overflow in SVM

Answer

Sigmoid kernels require computing a matrix inverse that can become singular

Answer

The sigmoid function is non-differentiable at zero, preventing gradient-based kernel fitting

Question 20

What is kernel PCA, and how does it differ from standard PCA?

Accepted Answer

Kernel PCA applies PCA in the kernel-induced feature space, capturing nonlinear structure

Answer

Kernel PCA reduces dimensions by selecting the k features with the smallest eigenvalues

Answer

Kernel PCA divides the PCA output into separate sub-kernels for each class label

Answer

Kernel PCA normalizes PCA scores using the RBF kernel bandwidth as a scale factor

Question 21

How do you use GridSearchCV in Python to select the best SVM kernel through cross-validation?

Accepted Answer

Use GridSearchCV with cv=5 to evaluate all kernel and hyperparameter combinations via cross-validation

Answer

Call model.score(X_test, y_test) for each kernel on the test set and pick the highest

Answer

Always choose RBF since it is a universal approximator and always performs best

Answer

Split data into three equal parts, train each kernel on one part, then compare

Question 22

What does the `degree` parameter control in the polynomial kernel?

Accepted Answer

The polynomial order of the decision boundary in feature space

Answer

The number of support vectors allowed to define the margin

Answer

The number of gradient descent iterations used to fit the polynomial kernel

Answer

The exponent applied to scale each feature dimension before the dot product

Question 23

How does sklearn's SVC handle multi-class classification problems?

Accepted Answer

SVC trains C(C−1)/2 binary classifiers using the One-vs-One strategy by default

Answer

SVC naturally extends to k classes using a k-dimensional separating hyperplane

Answer

SVC applies softmax probabilities to convert its binary output to multi-class predictions

Answer

SVC requires multi-class targets to be pre-encoded as one-hot vectors before training

Question 24

What is a string kernel, and where is it applied in practice?

Accepted Answer

A kernel computing similarity via shared substrings or subsequences, used in NLP and bioinformatics

Answer

A kernel that encodes numeric features as strings to handle missing data entries

Answer

A kernel that serializes the feature vector to a compressed string for storage efficiency

Answer

A kernel specifically designed for string-labeled targets like sentiment categories

Question 25

What happens to model behavior when gamma is set to a very large value in RBF-SVM?

Accepted Answer

High gamma creates a narrow kernel that memorizes training data and overfits

Answer

High gamma broadens the kernel influence, making the decision boundary smoother

Answer

High gamma has no effect on the boundary if C is simultaneously reduced

Answer

High gamma reduces the support vector count to exactly one per class

Question 26

Why does the dual formulation of SVM mathematically enable the kernel trick?

Accepted Answer

The dual SVM depends only on pairwise dot products xᵢ·xⱼ, replaceable by K(xᵢ, xⱼ)

Answer

The dual reduces the number of optimization variables, speeding up kernel evaluation

Answer

The dual eliminates slack variables, making the kernel SVM a hard-margin classifier

Answer

The dual converts the kernel matrix to its pseudo-inverse for faster optimization

Question 27

What is the computational complexity of training an SVM with a kernel like RBF?

Accepted Answer

O(n²) to O(n³) in the number of training examples, due to the kernel matrix

Answer

O(n log n) using fast Fourier transform approximations of the kernel function

Answer

O(n × d) where d is the number of input features

Answer

O(1) because only support vectors are retained after training completes

Question 28

How do the decision boundaries of polynomial kernel SVM with degree=2 and degree=5 differ?

Accepted Answer

Degree-2 gives smooth quadratic boundaries; degree-5 gives complex, overfitting-prone boundaries

Answer

Degree-2 creates linear decision boundaries; degree-5 creates perfect circles

Answer

Degree-5 always outperforms degree-2 because greater expressiveness always helps

Answer

Both produce identical boundaries since polynomial kernels only differ in computation speed

Question 29

What Python code visualizes the decision boundary of a trained RBF-SVM on 2D data?

Accepted Answer

Build a fine mesh grid, predict labels at each point, and fill regions with plt.contourf

Answer

Call model.decision_function() on training data and plot it as a histogram

Answer

Access model.coef_ to get boundary weight vectors and draw them as directional arrows

Answer

Plot the raw kernel values K(xᵢ, x) for each training point as a heat map overlay

Question 30

How do you define and use a custom kernel function in sklearn's SVC?

Accepted Answer

Pass a callable to kernel= or use kernel='precomputed' with a precomputed n×n matrix

Answer

Custom kernels must be registered in sklearn's kernel_registry dictionary before use

Answer

You must subclass SVC and override the internal _kernel_matrix method

Answer

Custom kernels are not supported — only the four built-in kernels are available in SVC

Test yourself