Design: Fraud Detection quiz

Question 1

What is the most effective approach for handling severe class imbalance in a fraud detection training dataset?

Accepted Answer

Use cost-sensitive learning with high weights on false negatives and evaluate with PR-AUC

Answer

Collect more legitimate transaction data to balance the classes naturally

Answer

Remove all non-fraud samples to create a balanced dataset

Answer

Treat fraud detection as a standard binary classification problem without modification

Question 2

What is the primary challenge with account takeover (ATO) fraud detection compared to transaction fraud?

Accepted Answer

ATO occurs at login with legitimate credentials, requiring different signals than transaction fraud

Answer

ATO fraud involves higher transaction amounts than standard fraud

Answer

ATO can only be detected after the fraudulent transaction is completed

Answer

ATO fraud is less common than transaction fraud and lower priority

Question 3

How should you select the decision threshold in a fraud detection system?

Accepted Answer

Select the threshold by minimizing total expected cost across false positives and false negatives

Answer

Always use 0.5 as the decision threshold for binary fraud classification

Answer

Set the threshold to maximize AUC-ROC across all operating points

Answer

Use the threshold that produces the highest recall regardless of precision

Question 4

How do you detect and prevent promotional abuse (promo fraud) at scale?

Accepted Answer

Cluster accounts by shared device fingerprint, IP, payment method, and address to detect multi-accounting

Answer

Only block users after they have claimed the same promotion more than 10 times

Answer

Require users to provide a government ID for every promotional claim

Answer

Disable promotions entirely to eliminate abuse risk

Question 5

What is the primary purpose of using a two-model approach (pre-screening plus full scoring) in fraud detection?

Accepted Answer

To reduce compute cost by quickly eliminating clearly legitimate transactions before expensive scoring

Answer

To maintain separate models for different currencies and geographies

Answer

To run A/B tests on two different fraud model architectures simultaneously

Answer

To separate chargeback prediction from real-time authorization decision

Question 6

How do you evaluate a fraud detection model beyond AUC-ROC?

Accepted Answer

PR-AUC, KS statistic, dollar-weighted recall, and detection rate at fixed FPR are all important complements

Answer

AUC-ROC is the best and only metric needed for fraud detection evaluation

Answer

Accuracy is the most reliable fraud detection metric due to class imbalance

Answer

Use F1 score at 0.5 threshold as the single evaluation metric

Question 7

What is the key challenge of using graph-based signals in real-time fraud detection?

Accepted Answer

Graph computations are batch-oriented, making real-time propagation challenging at transaction scale

Answer

Graph models are too inaccurate to use in fraud detection

Answer

Graph signals are only useful for money laundering, not transaction fraud

Answer

Graph databases cannot store more than one million nodes, limiting scale

Question 8

How should you design the model update strategy for a fraud detection system to minimize exposure during model transitions?

Accepted Answer

Use shadow deployment followed by canary release with automatic rollback on metric degradation

Answer

Deploy new models immediately to production to minimize the exposure window

Answer

Run the new model only during off-peak hours to limit financial exposure

Answer

Only deploy new models after collecting one full year of validation data

Question 9

Which technique is most effective for reducing false positives in a fraud detection system without significantly reducing recall?

Accepted Answer

Apply a second-stage verification model with additional features to re-score flagged transactions

Answer

Raise the decision threshold significantly to only flag the highest-confidence fraud

Answer

Remove all borderline cases from the training set to sharpen the decision boundary

Answer

Reduce model complexity to prevent overfitting to rare fraud patterns

Test yourself