Data Engineer roadmap
Build reliable data pipelines: SQL, Python, Spark, orchestration and warehouses.
Interview questions for this role- 1
Foundations
SQL and Python first.
SQL including joins and window functions
basic~40hWhy it matters: SQL is tested in every data engineering interview.
Python for data
basic~40hWhy it matters: Pipelines and glue code are written in Python.
Linux and Git
basic~15hWhy it matters: Jobs run on servers and ship through version control.
Milestone: Can you load a CSV into a database with Python and answer business questions in SQL?
- 2
Core
Modelling, ETL and big data.
Data modelling and warehousing
intermediate~25hWhy it matters: Star schemas and normalisation decide analytics quality.
Apache Spark
intermediate~40hWhy it matters: Spark is the standard for large-scale processing.
Orchestration with Airflow
intermediate~25hWhy it matters: Scheduling, retries and dependencies make pipelines reliable.
Milestone: Can you build a scheduled pipeline that loads, transforms and validates data daily?
- 3
Job-ready
Cloud and streaming.
One cloud data stack
intermediate~35hWhy it matters: Employers want S3/GCS, a warehouse and managed compute.
Streaming with Kafka
intermediate~25hWhy it matters: Real-time ingestion is increasingly expected.
Milestone: Can you design an end-to-end batch plus streaming pipeline on one cloud?