This project follows the Medallion Architecture:
- Raw data ingestion
- Schema inference and storage as Delta tables
- Data cleaning and standardization
- Type casting and validation
- Dimensional Data Model (Business Transformation)
- Ready for BI and analysis
- Databricks
- Apache Spark
- PySpark
- Spark SQL
- Delta Lake
- Unity Catalog
