Run Apache Airflow DAG and Retrieve XCom Value

What this workflow does

This workflow integrates the Apache Airflow API DAGRun and XCom . It enables n8n to trigger Airflow DAGs and retrieve the execution results.

Preparation:

1. Update Airflow API Link Prefix

   • Navigate to the airflow-api node.
   • Update the prefix of the Airflow API link in the format: http(s)://ip:port .
   • Example: https://airflow.example.com

2. Configure Authentication

   • Go to the Airflow: dag_run node.
   • Update the Basic Auth credentials with your Airflow username and password.
   • Repeat this step for Airflow: dag_run - state and Airflow: dag_run - get result nodes.

   Security Note: Using Basic Authentication requires storing credentials in plaintext. If possible, consider using API Keys or Tokens for enhanced security.

   • An example is setting Airflow's API Authentication to basic_auth . Choose other authentication methods if needed.
   • Ensure the user account has the following permissions: can create on DAG Runs , can read on DAG Runs , can read on XComs , can edit on DAGs , and can read on DAGs .

How to Use:

To execute this workflow, use the Execute Sub-workflow node with the following input parameters:

• dag_id : The DAG ID (name) in Airflow that you want to trigger.
• task_id : The Task ID (name) from which you want to retrieve the XCom return_value.
• conf : Input data for the Airflow DAG run.
• wait : Delay (in seconds) between each Airflow: dag_run - state check.
• wait_time : The maximum time (in seconds) to wait for Airflow: dag_run - state before returning an error.

Output:

• The workflow returns the XCom result from Airflow: dag_run - get result .
• The XCom return_value is stored in the value field.