Advanced Probability quiz

Question 1

What is Bayes' theorem and how is it applied in practice?

Accepted Answer

Bayes theorem: posterior = likelihood * prior / evidence

Answer

The posterior is equal to the prior times the evidence

Answer

Bayes theorem only applies when events are independent

Answer

The prior must always be a uniform distribution

Question 2

What is the moment generating function (MGF) and why is it useful?

Accepted Answer

The k-th derivative of the MGF at t=0 gives the k-th raw moment

Answer

The MGF is defined as E[e^{-tX}]

Answer

The first derivative of the MGF at t=0 gives the variance

Answer

MGFs are only defined for discrete random variables

Question 3

What is Chebyshev's inequality and how does it bound tail probabilities?

Accepted Answer

Chebyshev's inequality: P(|X-μ| >= k*σ) <= 1/k^2

Answer

Chebyshev's inequality requires normal distribution assumption

Answer

P(|X-μ| >= k*σ) <= 1/k for Chebyshev

Answer

Chebyshev's inequality only applies to discrete distributions

Question 4

What is the Central Limit Theorem and what are its conditions?

Accepted Answer

The CLT states the standardized sample mean converges in distribution to N(0,1)

Answer

The CLT requires that the underlying distribution is symmetric

Answer

The CLT applies only when n > 1000

Answer

The CLT requires the underlying distribution to have finite variance

Question 5

What is the Dirichlet distribution and why is it used in NLP?

Accepted Answer

The Dirichlet is defined over probability simplices and is conjugate to the multinomial

Answer

The Dirichlet distribution is defined on the real line

Answer

The Dirichlet is a multivariate generalization of the Normal distribution

Answer

Small Dirichlet concentration parameters produce dense, uniform distributions

Question 6

What is the Gamma distribution and its relationship to the Exponential?

Accepted Answer

The sum of n i.i.d. Exponential variables follows a Gamma distribution

Answer

Gamma(1, β) equals the Normal distribution

Answer

The Gamma distribution is only defined for integer shape parameters

Answer

Gamma(n/2, 2) follows a Beta distribution

Question 7

What is a martingale and what does the optional stopping theorem say?

Accepted Answer

A martingale satisfies E[X_{t+1}|history] = X_t

Answer

A martingale satisfies E[X_{t+1}|X_t] > X_t for all t

Answer

The optional stopping theorem guarantees profits in any gambling strategy

Answer

A submartingale tends to decrease over time

Question 8

What are order statistics and why are they important?

Accepted Answer

The k-th order statistic has a known PDF in terms of the original CDF and PDF

Answer

The PDF of the k-th order statistic does not depend on the original distribution

Answer

Order statistics are only defined for discrete distributions

Answer

The minimum order statistic always follows an Exponential distribution

Question 9

What is the law of large numbers (LLN) and how does it differ from the CLT?

Accepted Answer

The CLT is a stronger result than the LLN

Answer

The LLN describes the distribution of the sample mean

Answer

The SLLN requires convergence in distribution, not almost sure

Answer

The LLN requires both finite mean and finite variance

Question 10

What is the Chernoff bound and when does it outperform Chebyshev?

Accepted Answer

Chernoff bounds are obtained by applying Markov to the MGF, giving exponential decay

Answer

Chernoff bounds decay polynomially like Chebyshev

Answer

Chernoff bounds require the variables to be normally distributed

Answer

Chernoff bounds are looser than Chebyshev for all distributions

Question 11

What is the Kullback-Leibler divergence and what are its properties?

Accepted Answer

KL divergence is non-negative and equals zero only when P = Q

Answer

KL divergence is symmetric: KL(P||Q) = KL(Q||P)

Answer

KL divergence satisfies the triangle inequality

Answer

KL divergence is always between 0 and 1

Question 12

What are copulas and why are they used in multivariate modeling?

Accepted Answer

Sklar's theorem states any joint CDF can be expressed using a copula and marginals

Answer

Copulas can only be used when all marginal distributions are normal

Answer

Gaussian copulas always capture tail dependence correctly

Answer

Copulas require that all variables have the same marginal distribution

Question 13

What is the Cauchy distribution and why does it lack a mean?

Accepted Answer

The Cauchy distribution has no finite mean because its tails decay too slowly

Answer

The Cauchy distribution has a finite mean but infinite variance

Answer

The LLN applies to Cauchy samples because the sample size is large

Answer

The Cauchy distribution is equivalent to a Normal with very large variance

Question 14

What is the Poisson limit theorem (law of rare events)?

Accepted Answer

Poisson approximation is valid when np → λ and p → 0 as n → ∞

Answer

Binomial(n,p) converges to Poisson when p stays fixed as n grows

Answer

Binomial(n,p) can be approximated by Poisson only when n < 30

Answer

Poisson approximation requires n*p^2 → λ

Question 15

What is the Bernstein-von Mises theorem?

Accepted Answer

BvM shows that for large n, the posterior converges to normal centered at the MLE

Answer

The BvM theorem states the posterior is always exactly normal

Answer

BvM applies in all nonparametric Bayesian models

Answer

BvM shows frequentist and Bayesian methods always give identical results

Question 16

What is a sufficient statistic and the factorization theorem?

Accepted Answer

T(X) is sufficient if the likelihood factors into a function of T and θ times a function of x only

Answer

T(X) is sufficient if P(X|T,θ) depends on θ

Answer

The factorization theorem requires the distribution to be in the exponential family

Answer

A sufficient statistic must always be the sample mean

Question 17

What is the Gumbel distribution and its role in extreme value theory?

Accepted Answer

The Gumbel distribution is the limiting distribution of the maximum for thin-tailed distributions

Answer

The Gumbel distribution models the minimum, not the maximum

Answer

The Gumbel distribution arises for heavy-tailed parent distributions

Answer

The Gumbel-softmax is used for continuous variable relaxation in neural networks

Test yourself