This course covers the learning outcomes and goals of transitioning from traditional workflow managers to second-generation tools for big data. Participants will explore Python-based open-source workflow managers such as Luigi, Pinball, and Airflow, which offer agility and extensibility by describing workflows in code. The course aims to equip learners with the skills to effectively manage and schedule jobs in the evolving big data landscape. The teaching method involves examining the features of these tools to understand their capabilities and advantages. This course is intended for individuals seeking to enhance their workflow management skills in the context of big data processing.
Leave a Reply