Prefect

Prefect is an open-source orchestration engine that turns your Python functions into production-grade data pipelines with minimal friction

Ref: https://docs.prefect.io/v3/get-started

Used for : Task Management, Flow, Scheduling, Monitoring,

Task Management: allow to break workflow into smaller tasks.


Videos:

PAL 101: Prefect Basics – Write Workflows You Can Schedule and Monitor

https://www.youtube.com/watch?v=sZaHczwu74k


Mastering Workflow Automation: Using Prefect with Python for Data Pipelines

https://www.youtube.com/watch?v=VR4UjPOADEQ


Leave a comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.