This project implements a Modern Data Warehouse using PostgreSQL, structured with the Medallion Architecture (Bronze, Silver, and Gold layers). It integrates and analyzes structured data from CRM (Customer Relationship Management) and ERP (Enterprise Resource Planning) systems.

• Combining structured CRM and ERP data into a unified data warehouse.

• Extract, Transform, and Load (ETL) processes implemented using SQL to ensure data is cleaned, transformed, and loaded efficiently.

• Utilizes a Star Schema with fact and dimension tables for optimized query performance and analytical reporting.

• Designed to handle large datasets and support efficient analytical queries, ensuring scalability as data grows.

1. Bronze Layer: Raw data ingestion from CRM and ERP systems.
2. Silver Layer: Cleaned and transformed data, ready for analysis.
3. Gold Layer: Aggregated and enriched data for business reporting and insights.

• Database: PostgreSQL
• ETL: SQL-based pipelines
• Data Modeling: Star Schema
• Data Sources: CRM & ERP systems

1. Clone the Repository:

   git clone https://github.com/DataBells/data_warehouse_sql.git
   

• Download and install PostgreSQL from the official website: https://www.postgresql.org/download/.
• Follow the installation wizard to complete the setup.

• Open pgAdmin or the psql command-line tool.
• Run the following SQL command to create a new database:

  CREATE DATABASE modern_data_warehouse;

• Update the connection details in the project configuration file (e.g., config.json or environment variables) with the following:

  {
   "host": "localhost",
   "port": 5432,
   "database": "modern_data_warehouse",
   "user": "your_username",
   "password": "your_password"
  }

• Open the scripts/ folder in your project directory.

• Run the SQL scripts in the following order to load data into the Bronze, Silver, and Gold layers

This project is licensed under the MIT License.

Contributions are welcome! Please open an issue or submit a pull request for any improvements.

This markdown provides a structured overview of your Modern Data Warehouse project, including key features, architecture, tools, and setup instructions. Let me know if you need further customization! 🚀