Apache Airflow Training

• How to deal with the core functionalities such as DAGs, Tasks, Operators, and Workflows?
• How to understand and apply advanced concepts of Apache Airflow such as XCOMs, Branching and SubDAGs?
• What is the difference between Sequential, Local and Celery Executors
• How to use Sequential, Local and Celery Executors?
• How to use Apache Airflow in a Big Data ecosystem with the use of PostgreSQL, Hive, Elasticsearch?
• How to create plugins to add functionalities to Apache Airflow?
• How to use Docker with Airflow and different executors?
• How to implement solutions using Airflow to real data processing problems?
• How to install and configure Apache Airflow?

• Recorded Videos After Training
• Digital Learning Material
• Course Completion Certificate
• 24x7 After Training Support

• Data engineers who want to deploy their pipelines with the use of Airflow
• Engineers who want to switch their careers from conventional schedulers
• Professionals who want to add a hot & demanding technology to their resume
• Professionals who want to learn basic and advanced concepts about Apache Airflow

• You should have prior work experience in programming or scripting to pursue this Apache Airflow Training. You need at least 8 gigabytes of memory and VirtualBox installed. A VM of 3Gb needs to be downloaded. However, working experience in Python will help you a lot.

• PR Tech Skills will provide you with a training completion certificate after completing this Apache Airflow Training.

Module 1: Course Introduction 

• Important Prerequisites 
• Course Objectives 
• Who am I? 
• Development Environment

Module 2: Getting Started with Airflow

• Introduction
• Why Airflow?
• What is Airflow? 
• How Airflow works?
• Installing Airflow 2.0
• The CLI in Airflow 2.0
• What you need to know about the UI

Module 3: Coding Your First Data Pipeline with Airflow 

• Introduction
• What is DAG? 
• Time to code your first DAG
• DAG Skeleton
• What is an Operator?
• Creating Table
• The secret weapon!
• The Providers
• Is API available?
• Extracting users
• Processing users
• Storing users
• Order matters!
• You data pipeline in action!
• Dag scheduling
• Backfilling and catchup

Module 4: Databases and Executors

• Introduction
• The default configuration 
• Start scaling with the Local Executor
• Scale to the infinity with the Celery Executor
• Scaling Airflow with the Celery Executor in action!
• Changing the executor
• Concurrency, the parameters you must know!
• Concurrency in practice

Module 5: Implementing Advanced Concepts in Airflow 

• Introduction
• Adios repetitive patterns
• Minimising DAGs with SubDAGs
• Adios SubDAGs, Welcome TaskGroups!
• Sharing data between tasks with XComs
• XComs in action!
• Choosing a specific path in your DAG
• Executing a task according to a condition
• Trigger rules or how tasks get triggered
• Changing the way your tasks are triggered

Module 6: Creating Airflow Plugins with Elasticsearch and PostgreSQL

• Introduction
• Installing Elasticsearch
• How the plugin system works?
• Creating a hook interacting with Elasticsearch
• Creating the PostgresToElasticOperator

Module 7: Using Apache Airflow with Docker 

• Introduction
• Quick Reminder About Docker
• Running Airflow on Docker with the Celery Executor
• Running Airflow on Docker with the Local Executor

Module 8: Airflow 2.0

• What to expect from Airflow 2.0?