How we test AI models quiz

Question 1

What is a test set in machine learning?

Accepted Answer

To measure how well the model handles new, unseen data

Answer

To teach the model new patterns during training

Answer

To store extra data for future training rounds

Answer

To speed up the training process

Question 2

What is a training set and what is it used for?

Accepted Answer

Data the AI model uses to learn patterns

Answer

Data kept secret until the model is fully tested

Answer

Data used only for final performance evaluation

Answer

Data downloaded randomly from the internet

Question 3

Why is it unreliable to test an AI model on the same data it was trained on?

Accepted Answer

The model may have memorized answers, giving a falsely high score

Answer

Training data is always too large to test on

Answer

Test data is always more accurate than training data

Answer

Training data always contains labelling errors

Question 4

What does 'accuracy' mean when we evaluate an AI model?

Accepted Answer

The percentage of predictions the model gets right out of all predictions

Answer

The speed at which a model processes a single input

Answer

The fraction of predictions the model gets wrong

Answer

The total number of examples in the training set

Question 5

What is overfitting and how can you spot it?

Accepted Answer

High accuracy on training data but low accuracy on test data

Answer

Low accuracy on both training and test data

Answer

Equal accuracy on both training and test data

Answer

Low accuracy on training data but high accuracy on test data

Question 6

What is underfitting in machine learning?

Accepted Answer

Low accuracy on both training and test data

Answer

Very high training accuracy but low test accuracy

Answer

Low training accuracy but high test accuracy

Answer

Perfect accuracy on every dataset

Question 7

What is a confusion matrix?

Accepted Answer

Which predictions were correct and wrong, broken down by class

Answer

How fast the model makes predictions

Answer

How many training examples the model saw

Answer

The model's internal layer weights

Question 8

What is a validation set, and how is it different from a test set?

Accepted Answer

During training, to guide adjustments before final testing

Answer

After training is completely finished

Answer

Instead of a test set when data is scarce

Answer

Only when the model makes errors on training data

Question 9

What is k-fold cross-validation and why is it more reliable than a single train/test split?

Accepted Answer

It gives a more reliable estimate by evaluating on every data point in turn

Answer

It trains the model k times faster

Answer

It requires much less data overall

Answer

It eliminates the need for any test set

Question 10

What does 'precision' mean in AI testing?

Accepted Answer

80%

Answer

16%

Answer

20%

Answer

64%

Question 11

What does 'recall' mean in AI testing?

Accepted Answer

80%

Answer

40%

Answer

50%

Answer

90%

Question 12

What is the F1 score and when is it the right metric to use?

Accepted Answer

It balances precision and recall, which accuracy can miss on skewed datasets

Answer

It makes the model train faster

Answer

It ignores incorrect predictions automatically

Answer

It requires less labelled data than other metrics

Question 13

What is data leakage and why is it a serious problem?

Accepted Answer

Test set information accidentally influencing the training process

Answer

Using too much data for training

Answer

Using a model that is too simple for the task

Answer

Forgetting to shuffle the dataset before splitting

Question 14

What is a benchmark dataset?

Accepted Answer

To compare different models fairly on the same data

Answer

To keep training data private from competitors

Answer

To make all models train faster

Answer

To store backup copies of training data

Question 15

What is a baseline model and why do we need one?

Accepted Answer

To act as a minimum standard so you know if your model adds real value

Answer

To eventually replace the final AI model

Answer

To automatically generate additional training data

Answer

To store all the model's hyperparameter settings

Question 16

What does 'generalization' mean in the context of AI models?

Accepted Answer

It performs well on new, unseen data

Answer

It performs well only on training data

Answer

It memorises all of its training examples

Answer

It never makes any prediction errors

Question 17

What is Mean Absolute Error (MAE) and how is it calculated?

Accepted Answer

3

Answer

1

Answer

2

Answer

4

Question 18

Why can accuracy be a misleading metric when the dataset is class-imbalanced?

Accepted Answer

95%

Answer

5%

Answer

50%

Answer

100%

Question 19

What is an ROC curve and what does the AUC score tell us?

Accepted Answer

The model performs no better than random guessing

Answer

The model is perfect

Answer

The model achieves exactly 50% accuracy

Answer

The model needs significantly more training data

Question 20

What is hyperparameter tuning, and why must it be done using the validation set rather than the test set?

Accepted Answer

Tuning on the test set makes the model overfit to it, giving a misleadingly high score

Answer

The test set is usually too small for tuning

Answer

Hyperparameters do not affect test set accuracy

Answer

The test set is reserved solely for baseline comparison

Question 21

What is an ablation study in AI research?

Accepted Answer

To find out how much each part of a model contributes to its performance

Answer

To reduce the model's file size

Answer

To gather more labelled training data

Answer

To speed up the model's inference time

Question 22

What is out-of-distribution (OOD) data and why does it matter for testing AI?

Accepted Answer

It performs poorly because snowy roads are out-of-distribution data

Answer

It performs just as well because AI generalises perfectly

Answer

It performs better because snow increases image contrast

Answer

It refuses to make predictions until retrained

Question 23

What is human evaluation and when is it used to test AI models?

Accepted Answer

When outputs like text or images are hard to score automatically

Answer

When the model trains very slowly

Answer

When the dataset is too large for automatic metrics

Answer

When the model is small and simple

Question 24

What is A/B testing in the context of comparing AI models?

Accepted Answer

Comparing two models with real users to see which performs better in practice

Answer

Training both models simultaneously on the same data

Answer

Using model A to automatically label data for model B

Answer

Averaging the predictions from both models together

Question 25

What is error analysis and how does it help improve an AI model?

Accepted Answer

It reveals patterns in mistakes so developers know what to improve

Answer

It automatically corrects all of the model's errors

Answer

It speeds up the model's training time

Answer

It replaces the need for a separate test set

Question 26

What is fairness testing in AI evaluation?

Accepted Answer

To ensure the model performs equally well for all groups of people

Answer

To make the model run faster on all devices

Answer

To reduce the size of the training dataset

Answer

To improve the model's accuracy on the test set

Question 27

What are adversarial examples and what do they reveal about an AI model?

Accepted Answer

Weaknesses in robustness that can be exploited by subtle input changes

Answer

How quickly the model learns during training

Answer

How many layers the neural network contains

Answer

The amount of memory the model uses at runtime

Question 28

What is Root Mean Square Error (RMSE) and how does it differ from MAE?

Accepted Answer

It squares the errors before averaging, so large errors count much more

Answer

It divides errors by a larger constant

Answer

It only counts errors that exceed a set threshold

Answer

It ignores any errors smaller than 1

Question 29

Why should you shuffle your data before splitting it into training and test sets?

Accepted Answer

One set might contain only one class, making the split unfair and misleading

Answer

The model would train far too quickly

Answer

The model would always overfit regardless

Answer

The test set would end up far too large

Question 30

What does it mean when a model's training loss and validation loss change differently during training?

Accepted Answer

The model is overfitting the training data

Answer

The model is generalising well to new data

Answer

The model needs a larger dataset

Answer

The model is underfitting

Test yourself