Feature crosses and interactions quiz

Question 1

What is a feature cross (feature interaction) in machine learning?

Accepted Answer

A synthetic feature encoding the joint effect of two or more features

Answer

A method to remove correlated features before training

Answer

A regularization penalty applied to feature weights

Answer

A technique for imputing missing values across columns

Question 2

Why do linear models specifically benefit from explicit feature crosses, while deep neural networks often do not require them?

Accepted Answer

Linear models are additive and cannot model interactions without explicit crosses

Answer

Linear models train faster only when crosses are present

Answer

Crosses reduce the variance of the gradient in linear models

Answer

Linear models require crosses to handle missing data

Question 3

When you cross two categorical features with cardinalities of 1,000 and 500, what is the maximum cardinality of the resulting crossed feature, and what is the practical concern?

Accepted Answer

500,000; risk of sparsity and dimensionality explosion

Answer

1,500; risk of multicollinearity

Answer

500; loss of information from the larger feature

Answer

250,000; risk of target leakage

Question 4

What problem does the hashing trick solve when building feature crosses, and what is its main downside?

Accepted Answer

Bounds dimensionality of crosses at the cost of hash collisions

Answer

Eliminates the need for embeddings entirely with no downside

Answer

Removes correlated crosses automatically but slows training

Answer

Guarantees one bucket per cross with no information loss

Question 5

How does an interaction term appear in a linear regression equation, and which coefficient quantifies the interaction?

Accepted Answer

b3 on the product term x1*x2 captures the interaction

Answer

b1 captures the interaction once x2 is added

Answer

b0 the intercept captures the interaction

Answer

The sum b1+b2 captures the interaction

Question 6

Do gradient-boosted decision trees require explicit feature crosses to capture interactions? Why or why not?

Accepted Answer

No; sequential splits along a path model interactions implicitly, up to tree depth

Answer

Yes; trees are additive and need explicit product features

Answer

No; but only if features are one-hot encoded first

Answer

Yes; trees can only model interactions among numeric crosses

Question 7

What does Friedman's H-statistic measure, and what is its range?

Accepted Answer

Interaction strength via unexplained PD variance; range 0 to 1

Answer

Correlation between two features; range -1 to 1

Answer

Mutual information between feature and target; range 0 to infinity

Answer

Multicollinearity severity; range 1 to 10 (VIF)

Question 8

How do Factorization Machines (FM) model pairwise feature interactions, and why is this efficient under sparsity?

Accepted Answer

Interaction weight = dot product of learned latent vectors, sharing info across sparse pairs

Answer

Each pair gets an independent weight learned by least squares

Answer

It crosses only the top-k most frequent features

Answer

It uses random projections to approximate all crosses

Question 9

What does a Field-aware Factorization Machine (FFM) add over a standard FM?

Accepted Answer

Per-field latent vectors so a feature's embedding depends on the interacting field

Answer

A deeper neural network on top of FM outputs

Answer

Automatic removal of redundant fields before crossing

Answer

Replacement of dot products with cosine similarity

Question 10

In the DeepFM architecture, what are the two components and what does each contribute?

Accepted Answer

FM for low-order plus deep MLP for high-order interactions, sharing embeddings

Answer

Two stacked FMs for second- and third-order crosses

Answer

A wide linear model plus boosted trees

Answer

An autoencoder plus a logistic regression head

Question 11

What is the core operation of a cross layer in the Deep & Cross Network (DCN)?

Accepted Answer

x_{l+1} = x_0 (x_l^T w_l) + b_l + x_l, building bounded-degree explicit crosses

Answer

x_{l+1} = ReLU(W x_l + b), a standard dense layer

Answer

x_{l+1} = softmax(x_l) elementwise

Answer

x_{l+1} = x_l + dropout(x_l)

Question 12

How does DCN-v2 improve upon the original DCN cross layer?

Accepted Answer

Uses a (low-rank) weight matrix instead of a vector for richer per-layer crosses

Answer

Removes the residual connection to speed convergence

Answer

Replaces crosses with attention only

Answer

Switches from sigmoid to softmax output

Question 13

In Google's Wide & Deep model, what roles do the wide and deep parts play with respect to feature interactions?

Accepted Answer

Wide crosses memorize seen combinations; deep embeddings generalize to unseen ones

Answer

Wide generalizes via embeddings; deep memorizes via crosses

Answer

Both parts only memorize; generalization comes from dropout

Answer

Wide handles numerics; deep handles only the label

Question 14

What mechanism does AutoInt use to automatically learn feature interactions?

Accepted Answer

Multi-head self-attention over feature embeddings to learn interactions

Answer

A genetic algorithm searching over product features

Answer

Mutual-information ranking of all feature pairs

Answer

Random forests with interaction constraints

Question 15

What do SHAP interaction values decompose, and how do they relate to a feature's main effect?

Accepted Answer

A feature's SHAP value into a main effect plus symmetric pairwise interaction effects

Answer

The total variance of the model into per-tree contributions

Answer

The correlation matrix into eigenvalues

Answer

Gradients into first- and second-order Taylor terms only

Question 16

What is the difference between a 2-way and a 3-way feature interaction, and why are higher-order interactions used cautiously?

Accepted Answer

2-way is the joint effect of two features; 3-way of three; higher orders risk overfitting and sparsity

Answer

3-way interactions are always more accurate and preferred

Answer

2-way and 3-way are identical up to scaling

Answer

Higher-order interactions reduce dimensionality

Question 17

Why is it common to bucketize (discretize) continuous features before crossing them, e.g., crossing latitude with longitude?

Accepted Answer

Bucketed crosses let the model learn independent weights per region, capturing nonlinear joint effects

Answer

Bucketizing is required by all libraries before any cross

Answer

It reduces memory by removing the larger feature

Answer

It guarantees the interaction is linear and monotone

Question 18

When creating a multiplicative interaction term in linear regression, why is it often recommended to center (mean-subtract) the constituent features first?

Accepted Answer

Centering reduces collinearity between the product and main effects, improving stability/interpretation

Answer

Centering changes the interaction coefficient to make it significant

Answer

Centering is required or the product term is undefined

Answer

Centering increases multicollinearity to boost R-squared

Question 19

What is the difference between an additive and a multiplicative (interaction) effect of two predictors on an outcome?

Accepted Answer

Additive: x1's effect is constant in x2; interaction: x1's effect varies with x2

Answer

Additive and multiplicative effects are mathematically identical

Answer

Multiplicative means the features are correlated

Answer

Additive means the model is nonlinear

Question 20

How can a dot product of two learned embeddings act as a learned feature cross in recommender systems?

Accepted Answer

The dot product sums per-dimension products, i.e., learned multiplicative crosses of latent factors

Answer

It measures Euclidean distance between users and items

Answer

It is a linear additive combination with no interaction

Answer

It only works for one-hot inputs, not embeddings

Question 21

Why are plain feed-forward MLPs considered inefficient at learning explicit multiplicative interactions like x1*x2, motivating cross networks and FMs?

Accepted Answer

MLPs only approximate products via many piecewise-linear units; explicit-cross models are more efficient

Answer

MLPs cannot use backpropagation on product terms

Answer

MLPs compute exact products in a single neuron

Answer

Products are impossible for any neural network to learn

Question 22

Which regularization is commonly applied to large, sparse crossed feature sets to perform implicit feature selection, and why?

Accepted Answer

L1/Lasso, because it zeroes out irrelevant crosses for sparse selection

Answer

L2/Ridge, because it removes features entirely

Answer

Dropout on the input, which is identical to L1 here

Answer

No regularization is needed for crossed features

Question 23

How do feature crosses contribute to the curse of dimensionality, and what is one mitigation?

Accepted Answer

Crosses blow up dimensionality and sparsity; mitigate via hashing/FM/regularization

Answer

Crosses reduce dimensionality, so no mitigation is needed

Answer

Dimensionality is unaffected because crosses replace originals

Answer

Only PCA can mitigate cross-induced dimensionality

Question 24

A 2D partial dependence plot (PDP) shows a clear non-flat, non-separable surface across two features. What does this indicate?

Accepted Answer

A non-separable 2D PDP surface signals an interaction between the two features

Answer

It indicates the two features are perfectly correlated

Answer

It indicates label noise in the dataset

Answer

It always indicates overfitting

Question 25

State the statistical definition of an interaction effect between two factors.

Accepted Answer

The effect of one factor depends on the level of another (non-additivity)

Answer

Two factors are statistically independent

Answer

Two factors have equal means

Answer

The factors are perfectly collinear

Question 26

In TensorFlow's feature columns, what argument must you supply to tf.feature_column.crossed_column, and why?

Accepted Answer

hash_bucket_size, to bound the hashed Cartesian product

Answer

learning_rate, to control the cross weights

Answer

embedding_dim, which is mandatory for crosses

Answer

dropout_rate, to regularize the cross

Question 27

What is the difference between a synergistic and an antagonistic (redundant) interaction between two features?

Accepted Answer

Synergy: joint effect exceeds the sum of mains; antagonism/redundancy: joint effect is less

Answer

Synergy means the features are uncorrelated; antagonism means correlated

Answer

They are two names for the same additive effect

Answer

Synergy only occurs in linear models

Question 28

How do XGBoost/LightGBM interaction constraints control feature crosses, and when are they useful?

Accepted Answer

They restrict which features may co-occur on a tree path, limiting allowed interactions

Answer

They force every feature pair to interact in every tree

Answer

They are a post-hoc visualization, not a training control

Answer

They only constrain the learning rate of interactions

Question 29

Using scikit-learn's PolynomialFeatures with degree=2 and interaction_only=True on features [a, b, c], what columns are produced?

Accepted Answer

[1, a, b, c, ab, ac, bc] — no squared terms

Answer

[1, a, b, c, a^2, b^2, c^2] — no cross terms

Answer

[ab, ac, bc] only

Answer

[1, a, b, c, ab, ac, bc, a^2, b^2, c^2]

Question 30

What is target (mean) encoding of a feature cross, and what is the primary risk?

Accepted Answer

Encode the cross by its target mean; main risk is leakage/overfitting, mitigated by OOF + smoothing

Answer

Encode by feature frequency; no risk of leakage

Answer

Replace the cross with random noise to regularize

Answer

One-hot encode the cross; risk is only memory

Test yourself