I’m Smars Hu. A data engineer & minimalist based in Toronto 🇨🇦

GIF

God help those who help themselves

自救者,人恒救之

🔨 3 Years of Experience in Data Engineering

💼 Data Engineer @ ScotiaBank, Toronto, Canada🇨🇦

🎓 Master of Science in Big Data Analytics @ Trent University, Ontario, Canada 🇨🇦

📝 My website

📞 Book a meeting with me

associate-badge-de 0_atCqIOGA5HoUwVl0

Simulated an enterprise-level on-premise self-managed big data distributed cluster using Docker containers. Integrated components include Hadoop, Zookeeper, Spark, Hive, MySQL, Airflow, Prometheus, ClickHouse, and Power BI. Developed a data warehouse for an e-commerce backend based on dimensional modeling theory and built a BI analytics system for reporting and data analysis.

HTML tutorial

Reproduced a modern enterprise-grade Azure cloud data engineering architecture widely adopted in North America. Leveraged technologies such as Databricks, PySpark, ADLS Gen2, Unity Catalog, Delta Lake, Power BI, and Azure Data Factory (ADF) to develop cloud-native data pipelines on Azure and perform exploratory data analysis (EDA).

HTML tutorial

☘️ Languages

Python SQL Java Scala R

☘️ Distributed Computation & Data Warehouse

Apache Hadoop Apache Spark Apache Hive Delta Lake Apache ZooKeeper

☘️ Streaming & Lakehouse Architecture

Apache Flink Apache Kafka Strcutured Streaming

☘️ Data Engineering Practices

• Theory: Dimensional Modelling Lakehouse Architecture Schema Evolution

• Data Orchestration: Apache Airflow ADF databricks workflow

• Data Quality: Delta Live Tables

• Data Governance: Unity Catalog

• Data Visualization & BI: Power BI Tableau

• Data RESTful API Services & Integration: Spring Boot Postman

☘️ Databases: OLAP, OLTP & NoSQL

MySQL Oracle (OLTP)

ClickHouse Static Badge (OLAP)

Redis Elastic Search Kibana (NoSQL/Search)

☘️ Cloud-Native Data Engineering, Containerization & Platform Tools

Azure Databricks (Synapse, ADLS Gen2, Databricks, Data Factory)

AWS (S3, Lambda)

☘️ DevOps & Monitoring:

Docker Kubernetes Prometheus Grafana

GitLab CI GitLab CI Bitbucket

☘️ Basic Tools

CHAT GPT Claude 3.7 DeepSeek Cursor (AI)

Linux Git Apache Maven Anaconda (OS, Version Control, API, Dev environment)

Jira Markdown (Project Management, Doc)