Workflow Orchestration quiz

Question 1

What is an Airflow DAG and what are its key configuration parameters?

Accepted Answer

A directed acyclic graph representing a workflow with no cycles

Answer

A database table storing task results

Answer

A Docker container running a scheduled job

Answer

A message queue for task distribution

Question 2

What is XCom in Airflow and what are its size limitations?

Accepted Answer

Stored in the metadata database; practical limit is a few MB — use external storage for larger payloads

Answer

Stored in Redis with a 512 MB limit

Answer

Passed via shared filesystem mount only

Answer

Encrypted end-to-end with a 100 KB limit

Question 3

What is a trigger rule in Airflow and what are the most commonly used rules?

Accepted Answer

all_success, all_failed, all_done, one_success, one_failed, none_failed

Answer

always, never, on_success, on_failure

Answer

success, failure, skip, retry

Answer

upstream_done, upstream_success, manual

Question 4

What is an Airflow Sensor and when should you use a deferrable operator instead?

Accepted Answer

Deferrable operators release the worker slot while waiting, unlike traditional sensors that hold it

Answer

Deferrable operators are identical to sensors but use async Python

Answer

Sensors support retries; deferrable operators do not

Answer

Deferrable operators require CeleryExecutor; sensors work with any executor

Question 5

What are Airflow Variables and how do they differ from Connections?

Accepted Answer

Variables store config key-value pairs; Connections store structured credentials for external systems

Answer

Variables are encrypted; Connections are plaintext

Answer

Variables are runtime-only; Connections persist in git

Answer

They are identical; Connections is the deprecated name

Question 6

How do task groups improve DAG organization in Airflow 2.x?

Accepted Answer

Logical and visual groupings that replace SubDAGs without their performance issues

Answer

A way to run tasks in parallel automatically

Answer

A method to share XCom data between groups

Answer

A SubDAG wrapper with better error handling

Question 7

What is the ExternalTaskSensor and what are its key parameters?

Accepted Answer

Waits for a task or DAG in another DAG to reach a specified state

Answer

Polls an external REST API for a condition

Answer

Monitors an S3 bucket for new files

Answer

Triggers a downstream DAG on success

Question 8

What is Airflow DAG serialization and how does it improve scheduler performance?

Accepted Answer

Stores a JSON DAG representation in the database, reducing parsing overhead and enabling webserver independence from DAG files

Answer

Compresses DAG Python files for faster S3 uploads

Answer

Enables running DAGs as Docker containers

Answer

Converts Python DAGs to YAML for version control

Question 9

What is the Airflow LocalExecutor and when is it appropriate?

Accepted Answer

Runs tasks as subprocesses on the scheduler machine, supporting parallelism without external infrastructure

Answer

Runs tasks inside Docker containers locally

Answer

Runs tasks in Lambda functions synchronously

Answer

An alias for SequentialExecutor with extra logging

Question 10

What is Airflow's Triggerer component and what role does it play?

Accepted Answer

Runs an asyncio event loop managing async triggers for deferrable operators without consuming worker slots

Answer

A message broker replacement for CeleryExecutor

Answer

A component that stores DAG trigger timestamps in the database

Answer

An alternative scheduler for real-time DAG execution

Question 11

How do you implement a fan-out and fan-in pattern in Airflow?

Accepted Answer

Set task_a >> [task_b, task_c] for fan-out; [task_b, task_c] >> task_d for fan-in

Answer

Use SubDAGs for fan-out and ExternalTaskSensor for fan-in

Answer

Fan-out requires CeleryExecutor; fan-in requires LocalExecutor

Answer

Use XCom to coordinate fan-out and fan-in tasks

Question 12

What is the purpose of Airflow's max_active_tasks_per_dag setting?

Accepted Answer

Limits concurrent running TaskInstances for a single DAG across all its active DagRuns

Answer

Sets the maximum number of DAG runs that can be queued

Answer

Controls the number of Celery workers for a DAG

Answer

Limits how many times a DAG can be triggered per hour

Question 13

What is the purpose of Airflow's logical_date (formerly execution_date) and how should tasks use it?

Accepted Answer

The start of the data interval processed by the DAG run, not the actual execution wall-clock time

Answer

The timestamp when the scheduler triggered the DAG run

Answer

The next scheduled run time after the current run

Answer

The UTC timestamp when the first task started

Test yourself