Linear algebra for ML quiz

Question 1

What is a scalar, a vector, and a matrix, and how do they differ in terms of dimensions?

Accepted Answer

Scalar is 0-D, vector is 1-D, matrix is 2-D

Answer

Scalar is 1-D, vector is 2-D, matrix is 3-D

Answer

All three are 2-D objects

Answer

Scalar is 2-D, vector is 1-D, matrix is 0-D

Question 2

If a dataset has 100 samples and each sample has 5 features, what is the shape of the data matrix X?

Accepted Answer

(100, 5)

Answer

(5, 100)

Answer

(500,)

Answer

(100, 100)

Question 3

What does the dot product of two vectors compute, and what does it mean geometrically?

Accepted Answer

A scalar equal to |a||b|cos(theta)

Answer

A vector pointing between the two inputs

Answer

A matrix combining both vectors

Answer

The angle theta itself in degrees

Question 4

When are two matrices conformable for multiplication, and what is the shape of the product?

Accepted Answer

Columns of A must equal rows of B; result is (m x p)

Answer

Rows of A must equal rows of B; result is (n x p)

Answer

Both matrices must be square; result is (m x m)

Answer

Columns of A must equal columns of B; result is (m x n)

Question 5

Is matrix multiplication commutative? Explain with reasoning.

Accepted Answer

No, AB generally does not equal BA

Answer

Yes, AB always equals BA

Answer

Only when the matrices are rectangular

Answer

Yes, but only for 2x2 matrices

Question 6

What is the identity matrix and what role does it play in matrix multiplication?

Accepted Answer

A diagonal matrix of 1s; AI = IA = A

Answer

A matrix of all ones; it doubles A

Answer

A matrix of all zeros; it cancels A

Answer

A matrix of random values; it scrambles A

Question 7

What is the transpose of a matrix, and how does it affect the shape?

Accepted Answer

It swaps rows and columns; shape becomes (n x m)

Answer

It rotates A by 90 degrees; shape stays (m x n)

Answer

It inverts every element; shape stays (m x n)

Answer

It reverses row order; shape becomes (m x n)

Question 8

In a linear model y = Xw + b, what do X, w, and b represent and why is this a matrix-vector operation?

Accepted Answer

X is data, w is weights, b is the bias

Answer

X is weights, w is data, b is the output

Answer

X, w, and b are all scalars

Answer

X is the output, w is the bias, b is the data

Question 9

What is the L2 norm (Euclidean norm) of a vector and why is it used in ML?

Accepted Answer

Square root of the sum of squared components

Answer

Sum of absolute values of components

Answer

The largest component value

Answer

The product of all components

Question 10

How does the L1 norm differ from the L2 norm, and what effect does L1 regularization have on weights?

Accepted Answer

L1 sums absolute values and encourages sparsity (zeros)

Answer

L1 sums squares and encourages large weights

Answer

L1 and L2 are identical in effect

Answer

L1 always produces denser models than L2

Question 11

What does it mean for two vectors to be orthogonal, and how do you check it?

Accepted Answer

Their dot product equals 0

Answer

Their dot product equals 1

Answer

Their sum equals 0

Answer

Their lengths are equal

Question 12

What is a linear combination of vectors?

Accepted Answer

Scaling each vector and summing the results

Answer

Multiplying all vectors together element-wise

Answer

Taking only the largest vector

Answer

Computing the dot product of all vectors

Question 13

What is the span of a set of vectors?

Accepted Answer

The set of all linear combinations of the vectors

Answer

The longest vector in the set

Answer

The dot product of the vectors

Answer

The number of vectors in the set

Question 14

What does it mean for a set of vectors to be linearly independent?

Accepted Answer

No vector can be written as a combination of the others

Answer

Every vector is a multiple of the first one

Answer

All vectors point in the same direction

Answer

All vectors have the same length

Question 15

What is the rank of a matrix and why does it matter in ML?

Accepted Answer

The number of linearly independent rows or columns

Answer

The total number of elements in the matrix

Answer

The sum of the diagonal entries

Answer

The largest entry in the matrix

Question 16

What is the inverse of a square matrix, and when does it exist?

Accepted Answer

It exists when A is square and its determinant is nonzero

Answer

It always exists for any matrix

Answer

It exists only for rectangular matrices

Answer

It exists only when A equals the identity

Question 17

What does the determinant of a matrix tell you geometrically?

Accepted Answer

The factor by which area or volume is scaled

Answer

The number of rows in the matrix

Answer

The sum of all matrix entries

Answer

The average of the diagonal entries

Question 18

What are eigenvalues and eigenvectors of a matrix?

Accepted Answer

Vectors whose direction is preserved (only scaled) by A, with Av = lambda*v

Answer

Vectors that become zero under A

Answer

The rows of the inverse matrix

Answer

Vectors that rotate by 90 degrees under A

Question 19

How is Principal Component Analysis (PCA) related to linear algebra?

Accepted Answer

It uses eigenvectors of the covariance matrix to find max-variance directions

Answer

It randomly drops features to reduce dimensions

Answer

It computes the inverse of the data matrix

Answer

It sorts samples by their L1 norm

Question 20

What is the difference between element-wise (Hadamard) multiplication and matrix multiplication?

Accepted Answer

Element-wise multiplies entry by entry; matrix mult uses row-column dot products

Answer

They are the same operation

Answer

Element-wise requires matching inner dimensions only

Answer

Matrix multiplication multiplies entries entry by entry

Question 21

What is broadcasting in NumPy and how does it help with linear algebra operations?

Accepted Answer

It stretches compatible shapes so element-wise ops work without copying

Answer

It transposes arrays automatically

Answer

It sends arrays over a network

Answer

It always requires identical shapes

Question 22

Why is the dot product used to measure similarity between feature vectors, and what is cosine similarity?

Accepted Answer

Cosine similarity is the dot product divided by the product of the norms

Answer

Cosine similarity is the sum of the vectors

Answer

Cosine similarity ignores direction entirely

Answer

Cosine similarity is always between 0 and 100

Question 23

What is a diagonal matrix and why is it computationally convenient?

Accepted Answer

A matrix with values only on the main diagonal; it scales each coordinate

Answer

A matrix with values only in the first row

Answer

A matrix of all equal values

Answer

A matrix with no zero entries

Question 24

What is a symmetric matrix and where does it appear in ML?

Accepted Answer

A matrix where A equals its transpose

Answer

A matrix where A equals -A

Answer

A matrix with all entries equal

Answer

A matrix that is always invertible

Question 25

In gradient descent for linear regression, why is the gradient naturally expressed using matrix operations?

Accepted Answer

Because matrices let you compute all partial derivatives at once, e.g. X^T(Xw - y)

Answer

Because matrices make the loss larger

Answer

Because gradients require the inverse of X

Answer

Because matrices remove the need for a learning rate

Question 26

What is the closed-form (normal equation) solution for linear regression, and what linear algebra does it require?

Accepted Answer

w = (X^T X)^-1 X^T y

Answer

w = X^T y

Answer

w = X^-1 y

Answer

w = X y^T

Question 27

Why do we prefer vectorized operations over Python loops in ML computations?

Accepted Answer

Vectorized ops use optimized libraries and are much faster than loops

Answer

Loops are always faster than vectorized code

Answer

Vectorized ops give different mathematical answers

Answer

Vectorization only works on scalars

Question 28

What is the trace of a matrix?

Accepted Answer

The sum of the main diagonal entries

Answer

The product of the diagonal entries

Answer

The number of nonzero entries

Answer

The largest eigenvalue

Question 29

What does it mean to one-hot encode a categorical variable, and why is it a vector operation?

Accepted Answer

It represents a category as a vector with one 1 and the rest 0s

Answer

It assigns each category a random number

Answer

It averages all categories into one value

Answer

It deletes categorical columns

Question 30

How does a fully connected (dense) neural network layer use linear algebra?

Accepted Answer

It computes Wx + b followed by an activation function

Answer

It sorts the inputs

Answer

It only adds the inputs together

Answer

It inverts the weight matrix

Test yourself