In this data engineering course you learn to design data pipelines, model data for analytics and run them reliably. It is aimed at developers, technically minded analysts and platform teams who bring data from several sources into a warehouse or lake. You work with orchestration in Apache Airflow, processing with Spark, SQL transformations with dbt, and tests, monitoring and repeatability. You should bring working knowledge of SQL and Python.
What you’ll take away.
- Distinguish batch and streaming scenarios and choose a suitable pipeline architecture
- Define workflows as DAGs in Apache Airflow with dependencies, schedules and retries
- Design data models with fact and dimension tables and staging layers
- Implement transformations with dbt and Spark and secure them with automated tests
- Run pipelines reliably with idempotency, alerting and data quality checks
Your training in detail
- Who is it for?
- Developers, analysts with programming experience, BI developers and platform teams who operate data flows in production.
- Prerequisites
- Solid SQL and basic Python and command-line skills. Experience with Git is helpful.
- Format & planning
- Suggested format: 3 to 4 days live online or on site. Scope depends on prior knowledge.
Agenda: from understanding to application
Our independently developed agenda proposal. Focus areas, lab environment and scope are agreed before booking.
Architecture: ETL and ELT, data warehouse, data lake and lakehouse, batch versus streaming and how the layers interact.
Orchestration with Apache Airflow: DAGs, tasks, operators, schedules, backfills, retries, variables and connections.
Data modelling: star schema, slowly changing dimensions, staging, core layer and data marts, plus naming conventions.
Processing with Apache Spark: DataFrames, partitioning, joins on large tables, Parquet as a format and Spark SQL.
Transformation with dbt: models, the ref function, incremental models, tests, documentation and lineage.
Operations: idempotent runs, data quality tests, monitoring, alerting, CI for pipelines and handling schema changes.
Suggested exercise: You build a pipeline that ingests order data daily, transforms it into a star schema with dbt and raises an alert when quality tests fail.
Exams, licences & credentials
The course is vendor-neutral and does not prepare for a specific exam. Cloud and platform vendors offer their own data engineering certifications. Fees are only included if confirmed in the offer. Attending alone does not award a certification.
Vendor references & review date
Sources checked on 2026-11-03. These sources describe the technology or external exam requirements. They do not imply a partnership with NextNowa.
Your learning content
These objectives describe the training topic. The specific syllabus, duration, dates, trainer and price are agreed before booking.
Frequently asked questions
What prior knowledge do I need for data engineering?
You should be confident in SQL and able to read and write Python. Basics of the command line and Git are also needed. Cloud knowledge is useful but not essential, since the exercises can also run locally. If you are still learning SQL, start with the SQL course instead.
What is the difference between ETL and ELT?
With ETL, data is transformed outside the target before loading. With ELT, pipelines load raw data into the warehouse first and transform it there using SQL, for example with dbt. ELT is common with modern cloud warehouses because compute scales and raw data stays traceable.
Airflow or dbt: which do I need?
They solve different problems. Airflow orchestrates workflows: it starts tasks in the right order, retries failures and schedules runs. dbt transforms data in the warehouse using SQL models and tests. In many projects they work together, with Airflow triggering dbt runs.
Does Data engineering include a certificate?
The course is vendor-neutral and does not prepare for a specific exam. Cloud and platform vendors offer their own data engineering certifications. Fees are only included if confirmed in the offer. Attending alone does not award a certification.