Expectation-Maximization quiz

Question 1

What does the abbreviation 'EM' stand for in machine learning?

Accepted Answer

Expectation-Maximization

Answer

Error Minimization

Answer

Estimate-Model

Answer

Exponential Mean

Question 2

What is the main purpose of the EM algorithm?

Accepted Answer

To estimate parameters when data has hidden/latent variables

Answer

To sort data quickly

Answer

To compress images

Answer

To reduce the number of features

Question 3

What is a 'latent variable' in the context of EM?

Accepted Answer

A hidden, unobserved variable that affects the data

Answer

A variable measured directly from sensors

Answer

The label printed in the dataset

Answer

A constant used to scale features

Question 4

What are the two main steps repeated in the EM algorithm?

Accepted Answer

Expectation and Maximization

Answer

Encode and Merge

Answer

Estimate and Multiply

Answer

Expand and Minimize

Question 5

In the E-step of EM, what is computed?

Accepted Answer

The expected values of the latent variables (responsibilities)

Answer

The final cluster centers

Answer

The learning rate

Answer

The number of clusters

Question 6

In the M-step of EM, what is updated?

Accepted Answer

The model parameters that maximize expected likelihood

Answer

The raw input data

Answer

The number of iterations

Answer

The latent variable definitions

Question 7

What is a 'responsibility' in EM for Gaussian Mixture Models?

Accepted Answer

The probability a point belongs to each cluster

Answer

A penalty term added to the loss

Answer

The distance to the nearest point

Answer

The number of features in a point

Question 8

What quantity does EM try to maximize over its iterations?

Accepted Answer

The log-likelihood of the observed data

Answer

The classification accuracy

Answer

The Euclidean distance

Answer

The number of clusters

Question 9

Is each iteration of EM guaranteed to increase the log-likelihood?

Accepted Answer

It never decreases the log-likelihood (it stays the same or increases)

Answer

No, it can decrease randomly

Answer

Only on even-numbered iterations

Answer

Only if the data is normalized

Question 10

What type of optimum is EM generally guaranteed to reach?

Accepted Answer

A local optimum (not guaranteed global)

Answer

The global optimum always

Answer

The worst possible solution

Answer

A saddle point only

Question 11

Why are multiple random initializations often used with EM?

Accepted Answer

To escape poor local optima and find a better solution

Answer

To make the algorithm slower on purpose

Answer

Because random numbers improve accuracy automatically

Answer

To increase the number of clusters

Question 12

Which popular clustering model is commonly trained using the EM algorithm?

Accepted Answer

Gaussian Mixture Models

Answer

Linear Regression

Answer

Decision Trees

Answer

K-Nearest Neighbors

Question 13

How is K-Means related to EM?

Accepted Answer

K-Means is a hard-assignment special case of EM

Answer

They are completely unrelated

Answer

EM is a special case of K-Means

Answer

K-Means uses no iterations while EM does

Question 14

What is the key difference between 'hard' and 'soft' cluster assignments?

Accepted Answer

Hard assigns to one cluster, soft gives probabilities across clusters

Answer

Hard uses probabilities, soft uses single labels

Answer

They are identical

Answer

Soft assignment is only used in regression

Question 15

What is a common stopping criterion for the EM algorithm?

Accepted Answer

When the log-likelihood change becomes very small

Answer

When all data points are deleted

Answer

When accuracy reaches exactly 50%

Answer

When the learning rate hits zero

Question 16

In a Gaussian Mixture Model, what does the 'mixing weight' (or mixing coefficient) represent?

Accepted Answer

The prior probability of a component, summing to 1

Answer

The variance of a cluster

Answer

The number of data points

Answer

The learning rate of EM

Question 17

During the M-step of a GMM, how is the mean of a component typically updated?

Accepted Answer

A responsibility-weighted average of all data points

Answer

The median of all points

Answer

A random point from the dataset

Answer

The point farthest from the origin

Question 18

Why can't we simply use ordinary maximum likelihood directly when latent variables are present?

Accepted Answer

Because summing over hidden states makes direct optimization intractable

Answer

Because the data is always too small

Answer

Because likelihood does not exist with latent variables

Answer

Because computers cannot multiply probabilities

Question 19

What is the 'complete-data log-likelihood' in EM?

Accepted Answer

The log-likelihood assuming both observed and latent variables are known

Answer

The likelihood using only test data

Answer

The likelihood of a single data point

Answer

The error rate of the model

Question 20

For soft clustering with scikit-learn, which class implements EM for Gaussian mixtures?

Accepted Answer

sklearn.mixture.GaussianMixture

Answer

sklearn.linear_model.LinearRegression

Answer

sklearn.tree.DecisionTreeClassifier

Answer

sklearn.svm.SVC

Question 21

In scikit-learn's GaussianMixture, which method returns the soft responsibilities for each point?

Accepted Answer

predict_proba

Answer

fit_transform

Answer

score_samples

Answer

get_params

Question 22

What does it mean that EM performs 'coordinate ascent' on a lower bound of the log-likelihood?

Accepted Answer

It alternately tightens and maximizes a lower bound on the log-likelihood

Answer

It randomly guesses parameters

Answer

It descends the gradient of the loss

Answer

It removes data points each step

Question 23

Which mathematical inequality is central to deriving the EM lower bound?

Accepted Answer

Jensen's inequality

Answer

The triangle inequality

Answer

Cauchy-Schwarz inequality

Answer

The Pythagorean theorem

Question 24

How does EM handle missing data values in a dataset?

Accepted Answer

It treats missing values as latent variables and estimates them

Answer

It deletes any row with a missing value

Answer

It replaces them all with zero permanently

Answer

It cannot handle missing data

Question 25

Consider EM applied to fair-vs-biased coin flips where which coin was used is unknown. What is the latent variable?

Accepted Answer

Which coin was used for each set of flips

Answer

The number of flips

Answer

The color of the coin

Answer

The temperature of the room

Question 26

What happens to a Gaussian component during EM if its variance is allowed to shrink toward zero around a single point?

Accepted Answer

The likelihood can diverge to infinity (a singularity)

Answer

The model becomes perfectly accurate

Answer

Nothing changes

Answer

All points join that cluster

Question 27

Which of these is a real-world application of the EM algorithm?

Accepted Answer

Clustering and density estimation with mixture models

Answer

Sorting a list of numbers

Answer

Computing a matrix transpose

Answer

Printing text to a console

Question 28

Is EM a supervised or unsupervised learning method when used for clustering?

Accepted Answer

Unsupervised, because it discovers hidden structure without labels

Answer

Supervised, because it needs labels

Answer

Reinforcement learning

Answer

Semi-supervised only

Question 29

How is the mixing weight of a component updated in the GMM M-step?

Accepted Answer

The average responsibility of that component over all points

Answer

The maximum responsibility value

Answer

A fixed constant of 0.5

Answer

The variance of the responsibilities

Question 30

What is one key advantage of EM-based soft clustering over hard clustering like K-Means?

Accepted Answer

It provides probabilistic memberships and flexible cluster shapes

Answer

It is always faster than K-Means

Answer

It never needs initialization

Answer

It guarantees the global optimum

Test yourself