Skip to content
Natnael Masresha Zerihun

Traffic Data Warehouse (ELT)

10 Academy · 2022 · Academic project · Individual

A fully dockerized ELT warehouse for vehicle-trajectory data: Airflow-orchestrated loading into PostgreSQL, dbt transformations with hosted docs, and Redash dashboards.

  • Airflow
  • dbt
  • PostgreSQL
  • Redash
  • Docker

Overview

An end-to-end ELT pipeline for drone-captured vehicle trajectory data, built to demonstrate warehouse design: raw loads orchestrated by Airflow, transformation logic in dbt with generated documentation, and analyst-facing dashboards in Redash, all running in Docker Compose.

Media

Redash dashboard with traffic charts and tables
Redash dashboard over the warehouse: vehicle counts, speeds, and distances from the pNEUMA trajectory dataset.
Second Redash dashboard view
Further dashboard views for analysts.
Airflow DAGs list view
Airflow DAG list: the extractor/loader running daily alongside the dbt transformation DAG.
Airflow directed acyclic graph view
Task graph of the loading pipeline.
dbt lineage graph
dbt model lineage for the transformation layer.
dbt docs site showing project models
Hosted dbt documentation generated for the warehouse models.

Resources