Bayesian methods quiz

Question 1

What does Bayes' theorem state, and what are its four components?

Accepted Answer

Posterior

Answer

Prior

Answer

Likelihood

Answer

Evidence

Question 2

In Bayesian inference, how does the posterior distribution differ from the prior distribution?

Accepted Answer

Prior × Likelihood ∝ Posterior

Answer

Prior × Evidence → Posterior

Answer

Posterior × Likelihood → Prior

Answer

Posterior / Likelihood → Prior

Question 3

What is the likelihood function in Bayesian inference, and how does it differ from a probability?

Accepted Answer

θ³(1−θ)² × C(5,3)

Answer

(1−θ)³θ² × C(5,3)

Answer

θ⁵

Answer

θ³

Question 4

What is a conjugate prior, and why is the Beta distribution conjugate to the Binomial likelihood?

Accepted Answer

Beta(6, 9)

Answer

Beta(4, 3)

Answer

Beta(2, 3)

Answer

Beta(6, 3)

Question 5

What is the fundamental philosophical difference between Bayesian and frequentist interpretations of probability?

Accepted Answer

There is a 90% probability that the coin's bias is between 0.4 and 0.6.

Answer

The 95% confidence interval contains the true mean in 95% of repeated samples.

Answer

The p-value is 0.03, so we reject H₀.

Answer

The sample mean is an unbiased estimator of the population mean.

Question 6

In a Beta-Binomial model with a weakly informative prior Beta(5,5) reflecting a fair-coin belief, what is the posterior after observing 7 heads in 10 flips, and what is the posterior mean?

Accepted Answer

0.60

Answer

0.50

Answer

0.70

Answer

0.55

Question 7

What is Maximum A Posteriori (MAP) estimation, and how does it differ from Maximum Likelihood Estimation (MLE)?

Accepted Answer

L2 (Ridge)

Answer

L1 (Lasso)

Answer

Dropout

Answer

Elastic Net

Question 8

What is the 'naive' assumption in a Naive Bayes classifier, and why is it useful despite often being violated?

Accepted Answer

Features are conditionally independent given the class label

Answer

All features follow a Gaussian distribution

Answer

Features are correlated with each other

Answer

There is only one relevant feature per class

Question 9

Describe how sequential Bayesian updating works: if P(device works) = 0.5 initially and you observe three independent positive test results with P(positive | works) = 0.9 and P(positive | fails) = 0.2, does the update order matter?

Accepted Answer

Commutativity of multiplication in the likelihood terms

Answer

Conjugacy of the prior

Answer

The law of large numbers

Answer

The central limit theorem

Question 10

What is the posterior predictive distribution, and why is it more informative than using a single point estimate of θ?

Accepted Answer

The parameter θ weighted by the posterior P(θ|x)

Answer

The observed data x

Answer

The likelihood function only

Answer

The prior distribution only

Question 11

Why is Markov Chain Monte Carlo (MCMC) needed in Bayesian inference, and what computational problem does it solve?

Accepted Answer

The normalizing constant P(data)

Answer

The likelihood P(data|θ)

Answer

The prior P(θ)

Answer

The posterior mode (MAP estimate)

Question 12

In the Metropolis-Hastings algorithm, what is the acceptance probability and how does it ensure the chain converges to the correct posterior?

Accepted Answer

Accepted with probability equal to the posterior density ratio

Answer

Always rejected

Answer

Always accepted

Answer

Accepted with probability 0.5

Question 13

How does Gibbs sampling differ from the Metropolis-Hastings algorithm, and when is Gibbs sampling preferred?

Accepted Answer

It always accepts proposals by sampling from the exact conditional distribution

Answer

It uses a uniform proposal distribution

Answer

It only works for discrete parameters

Answer

It requires computing the normalizing constant

Question 14

What is the difference between a Bayesian 95% credible interval and a frequentist 95% confidence interval?

Accepted Answer

Given the data, there is a 95% probability the parameter lies within it

Answer

95% of sample means fall within it

Answer

The true parameter lies within it in 95% of repeated experiments

Answer

It is always wider than the frequentist confidence interval

Question 15

What is the Bayes factor and how is it used for comparing two competing models?

Accepted Answer

The observed data is 15 times more probable under M₁ than under M₂

Answer

Model M₂ is 15 times more likely given the data

Answer

Model M₁ has 15% higher accuracy on the test set

Answer

The prior odds favour M₁ by a factor of 15

Question 16

How does a Naive Bayes spam filter use word frequencies to classify email, and what happens if a word never appears in the training spam emails?

Accepted Answer

The entire product ∏P(wordᵢ|spam) becomes 0, misclassifying any spam email containing that word

Answer

The classifier always predicts non-spam

Answer

The prior P(spam) becomes 0

Answer

The likelihood ratio explodes to infinity

Question 17

What is Laplace (add-1) smoothing in Naive Bayes, why is it necessary, and what Bayesian prior does it correspond to?

Accepted Answer

Dirichlet prior with all pseudo-counts equal to 1

Answer

Gaussian prior on word probabilities

Answer

Beta prior with α=β=0.5 (Jeffreys prior)

Answer

Jeffreys prior on the class probabilities

Question 18

What is the difference between an informative prior and a non-informative (flat) prior, and when should each be used?

Accepted Answer

Beta(50, 50)

Answer

Beta(1,1)

Answer

Beta(0.5, 0.5)

Answer

Beta(5,2)

Question 19

How does Bayesian linear regression differ from ordinary least squares (OLS), and what does it produce as output?

Accepted Answer

It outputs a full posterior distribution over weights, quantifying parameter uncertainty

Answer

It always achieves lower training error than OLS

Answer

It does not require any assumptions about the data distribution

Answer

It automatically selects the best subset of features

Question 20

What is a hierarchical Bayesian model and in what situation is it particularly useful compared to fully pooled or completely unpooled analysis?

Accepted Answer

Shrinkage or borrowing strength across groups

Answer

Conjugacy of the prior

Answer

MAP estimation of each group independently

Answer

Laplace smoothing applied to group means

Question 21

How does Bayesian A/B testing differ from frequentist A/B testing, and what direct probability statement can it make that frequentist methods cannot?

Accepted Answer

P(variant B conversion rate > variant A conversion rate)

Answer

The sample mean conversion rate of each variant

Answer

The standard error of the conversion rate difference

Answer

The minimum sample size needed for significance

Question 22

A disease affects 1% of the population. A test has 99% sensitivity and 95% specificity. Using Bayes' theorem, what is the probability that a person who tests positive actually has the disease?

Accepted Answer

The disease prevalence (prior) is very low, so false positives dominate

Answer

The specificity is too high, inflating false positives

Answer

The likelihood function was incorrectly defined

Answer

Bayes' theorem does not apply when prevalence is below 5%

Question 23

What are the mean and variance of a Beta(α, β) distribution, and how do α and β control its shape?

Accepted Answer

0.60

Answer

0.40

Answer

0.50

Answer

0.67

Question 24

What is a Bayesian network and how does its graph structure encode conditional independence relationships?

Accepted Answer

Product of each node's conditional given its parents P(Xᵢ|parents(Xᵢ))

Answer

Product of all marginal distributions P(Xᵢ)

Answer

Sum of all pairwise conditional distributions

Answer

Product of all pairwise joint distributions

Question 25

What is the Jeffreys prior and why is it considered more 'objective' than a flat uniform prior?

Accepted Answer

It is invariant under reparametrisation of θ

Answer

It is always proper (integrates to 1 over θ)

Answer

It always equals the posterior when data is uninformative

Answer

It is equivalent to Laplace smoothing in discrete settings

Question 26

Write a Python snippet using scipy.stats.beta that updates a Beta(2,2) prior after observing 7 heads in 10 coin flips and plots both the prior and posterior distributions.

Accepted Answer

Beta(9, 5)

Answer

Beta(7, 3)

Answer

Beta(7, 5)

Answer

Beta(9, 3)

Question 27

Implement the core training and prediction steps of a Naive Bayes classifier from scratch in Python for binary (0/1) features, including Laplace smoothing.

Accepted Answer

To prevent numerical underflow when multiplying many small probabilities together

Answer

Logarithms are faster to compute than multiplications on modern hardware

Answer

Because the posterior is always log-normally distributed

Answer

To convert probabilities into likelihoods

Question 28

What is the Dirichlet distribution, how does it generalise the Beta distribution, and where is it used in machine learning?

Accepted Answer

Uniform over all valid probability vectors that sum to 1

Answer

Concentrated at exactly (1/3, 1/3, 1/3)

Answer

Concentrated near (1, 0, 0)

Answer

A 3D Gaussian centred at (1/3, 1/3, 1/3)

Question 29

What is the marginal likelihood (evidence) P(data) in Bayes' theorem, what role does it play, and why is computing it often the hardest part of Bayesian inference?

Accepted Answer

It is the normalising constant ensuring the posterior is a valid probability distribution

Answer

It determines the most likely parameter value (MAP estimate)

Answer

It measures the spread (variance) of the prior distribution

Answer

It replaces the likelihood in the Bayes formula

Question 30

What is prior sensitivity analysis and why should Bayesian practitioners perform it before trusting conclusions?

Accepted Answer

The data are insufficient to dominate the prior; more data may be needed

Answer

The likelihood is incorrectly specified

Answer

The analysis is robust and conclusions can be fully trusted

Answer

Bayes' theorem was applied incorrectly

Test yourself