ML Coding (from scratch) quiz

Question 1

Implement linear regression using the normal equations from scratch. What is the time complexity?

Accepted Answer

O(nd² + d³)

Answer

O(nd)

Answer

O(n²d)

Answer

O(n³)

Question 2

Implement k-means clustering from scratch. How is convergence defined and what is its time complexity per iteration?

Accepted Answer

O(nkd) per iteration

Answer

O(n²k) per iteration

Answer

O(nk²d) per iteration

Answer

O(nd) per iteration

Question 3

Implement Gaussian Naive Bayes from scratch and explain the log-likelihood computation.

Accepted Answer

Log space is used to prevent numerical underflow

Answer

Log space is used to prevent numerical overflow

Answer

Naive Bayes assumes features are correlated within a class

Answer

Training in Naive Bayes requires gradient descent

Question 4

Implement batch normalization forward and backward passes from scratch.

Accepted Answer

dγ = sum(dY * X_hat, axis=0)

Answer

dγ = sum(dY, axis=0)

Answer

dγ = sum(dY * γ, axis=0)

Answer

dγ = mean(dY, axis=0)

Question 5

Implement the attention mechanism (scaled dot-product attention) from scratch.

Accepted Answer

Scale by √d_k to prevent softmax saturation

Answer

Scale by d_k to prevent softmax saturation

Answer

Scale by d_k² to ensure unit variance

Answer

No scaling is needed for attention scores

Question 6

Implement gradient checking to verify a manual backpropagation implementation.

Accepted Answer

Gradient checking uses central differences for O(ε²) accuracy

Answer

Gradient checking uses forward differences only

Answer

Relative error > 1e-3 is acceptable

Answer

Gradient checking should be used during every training step

Question 7

Implement the GINI impurity split criterion for a decision tree and compare with entropy.

Accepted Answer

Gini impurity avoids log computation compared to entropy

Answer

Gini impurity is always higher than entropy

Answer

Entropy and Gini always produce identical trees

Answer

Gini impurity range is [0, 1] for all K

Question 8

Implement the support vector machine (SVM) primal problem using sub-gradient descent.

Accepted Answer

SVM primal uses hinge loss: max(0, 1-y(w·x+b))

Answer

SVM primal uses cross-entropy loss

Answer

SVM dual is harder to solve than the primal

Answer

Hinge loss gradient is always w (the weight vector)

Question 9

Implement Kernel Ridge Regression from scratch and demonstrate the kernel trick.

Accepted Answer

The kernel trick replaces dot products with kernel evaluations in dual form

Answer

The kernel trick requires explicit feature computation

Answer

Kernel ridge regression is only for linear kernels

Answer

The kernel matrix is always n×d

Question 10

Implement a numerically stable log-sum-exp function and explain its role in probabilistic models.

Accepted Answer

Subtract the maximum for numerical stability in log-sum-exp

Answer

Subtract the minimum for numerical stability in log-sum-exp

Answer

Log-sum-exp is computed as log(sum(x)) directly

Answer

Log-sum-exp is only needed for negative values

Question 11

Implement the DBSCAN clustering algorithm from scratch.

Accepted Answer

DBSCAN automatically detects outliers without specifying k

Answer

DBSCAN requires specifying the number of clusters

Answer

DBSCAN only finds spherical clusters

Answer

DBSCAN has O(n log n) complexity always

Question 12

Implement t-SNE dimensionality reduction: the high-dimensional affinity computation step.

Accepted Answer

t-SNE uses Gaussian in high-d and Student-t in low-d to prevent crowding

Answer

t-SNE uses Gaussian kernels in both high and low dimensions

Answer

t-SNE minimizes KL(Q||P)

Answer

Perplexity in t-SNE is a fixed constant independent of the data

Question 13

Implement the scaled dot-product multi-head attention from scratch.

Accepted Answer

d_k = d_model / h for each head

Answer

d_k = d_model × h for each head

Answer

Multi-head attention is sequential, not parallel

Answer

All heads use the same weight matrices

Question 14

Implement the k-means++ initialization algorithm and explain why it outperforms random initialization.

Accepted Answer

k-means++ selects each new centroid proportional to D(x)²

Answer

k-means++ guarantees finding the global optimum

Answer

k-means++ always selects the farthest point from all centroids

Answer

k-means++ is slower than random init with no quality benefit

Question 15

Implement the ReLU activation and its derivative from scratch.

Accepted Answer

ReLU derivative is 1 for x>0, 0 for x<0

Answer

ReLU derivative is 1 for all x

Answer

ReLU derivative is the sigmoid function

Answer

ReLU has no derivative anywhere

Question 16

Implement L1 and L2 regularization and show how they modify the gradient update.

Accepted Answer

L1 promotes weight sparsity; L2 shrinks weights proportionally

Answer

L2 promotes weight sparsity; L1 shrinks weights proportionally

Answer

Both L1 and L2 produce sparse weights

Answer

L1 and L2 regularization are identical in effect

Question 17

Implement the Perceptron learning algorithm from scratch.

Accepted Answer

Perceptron convergence is guaranteed for linearly separable data

Answer

Perceptron converges for any dataset

Answer

Perceptron uses a sigmoid activation

Answer

Perceptron update uses the Hessian

Question 18

Implement layer normalization from scratch.

Accepted Answer

Layer norm normalizes over the feature dimension per sample

Answer

Layer norm normalizes over the batch dimension

Answer

Layer norm requires running statistics at inference

Answer

Layer norm and batch norm produce identical outputs

Question 19

Implement the forward pass of a transformer encoder block from scratch.

Accepted Answer

FFN expansion factor is typically 4, giving 4d hidden dim

Answer

FFN expansion factor is always 2

Answer

Residual connections are not used in transformers

Answer

Layer norm is applied after the attention output only

Question 20

Implement the triplet loss for metric learning from scratch.

Accepted Answer

Triplet loss enforces d(a,p) - d(a,n) + margin ≤ 0

Answer

Triplet loss uses pairwise comparisons only

Answer

Triplet loss guarantees absolute distances

Answer

Margin α=0 makes triplet loss non-informative

Question 21

Implement the Focal Loss for addressing class imbalance in object detection.

Accepted Answer

Focal loss down-weights easy examples via (1-p_t)^gamma

Answer

Focal loss increases the loss for easy examples

Answer

Focal loss is identical to cross-entropy when gamma=1

Answer

Focal loss requires separate handling for each class

Test yourself