Comparte tu proyecto de Platzi explora el espacio con Airflow y certifícate

Clase 29 de 29 • Curso de Fundamentos de Apache Airflow

Carlos Estevez

student•

Hay un problema acá, el botón que redirige al examen de certificación esta oculto, la barra para hacer scroll no esta. Presiona click derecho y dirígete a inspeccionar pagina y reduce el ancho de la pagina a 766px o menos, allí aparecerá el botón.

Christian Manuel Campos Navarro

student•

Apache Airflow is an open-source platform designed for programmatically authoring, scheduling, and monitoring workflows. It is widely used for orchestrating data pipelines and other batch-oriented processes.

Key aspects of Airflow include:

Workflows as DAGs: Workflows in Airflow are defined as Directed Acyclic Graphs (DAGs). A DAG represents a collection of tasks and their dependencies, dictating the order in which they should be executed.
Tasks and Operators: Individual units of work within a DAG are called Tasks. These tasks are implemented using Operators, which are pre-built or custom components that perform specific actions (e.g., executing a Python script, running a shell command, transferring data between systems).
Scheduler: The Airflow scheduler is responsible for triggering and managing the execution of DAGs and their tasks based on defined schedules and dependencies.
Web UI: Airflow provides a rich web-based user interface for visualizing, monitoring, and managing workflows. This UI allows users to track task progress, view logs, and troubleshoot issues.
Extensibility: Airflow's Python-based framework allows for extensive customization and integration with various technologies and services through its provider system and custom operators.
Scalability: Airflow can be deployed in various configurations, from a single-process setup for development to a distributed system capable of handling large-scale workloads.

Christian Manuel Campos Navarro

student•

Gracias.

Comparte tu proyecto de Platzi explora el espacio con Airflow y certifícate

Introducción a Apache Airflow

Introducción al curso

¿Para qué sirve airflow?

¿Por qué usar airflow?

Resumen módulo 1

Conceptos básicos

DAG

Tasks y Operators

Scheduler

Instalación y configuración

Instalación de Apache Airflow usando Docker

Posibles configuraciones

Variables y conexiones

Implementando un DAG

Implementando un DAG

Bash Operator

Python Operator

Definiendo dependencias entre tareas

Custom Operator

Orquestar y monitorizar procesos

Orquestando un DAG I

Orquestando un DAG II

Monitoring

Task Actions

Trigger Rules

Sensores

¿Qué son los sensores?

ExternalTaskSensor

FileSensor

Templates con Jinja

¿Qué son los templates con Jinja?

XComs

¿Qué son los Xcoms?

BranchPythonOperator

BranchPythonOperator

Proyecto

Definición del proyecto

Cierre del curso

Cierre del curso

Comparte tu proyecto de Platzi explora el espacio con Airflow y certifícate