Data Engineer Handbook Save
This is a repo with links to everything you'd ever want to learn about data engineering
Project README
The Data Engineering Handbook
This repo has all the resources you need to become an amazing data engineer!
Make sure to check out the projects section for more hands-on examples!
Make sure to check out the interviews section for more advice on how to pass data engineering interviews!
Resources
Great books:
- The Fundamentals of Data Engineering
- Designing Data-Intensive Applications
- Designing Machine Learning Systems
- The Hundred Page Machine Learning Book
- Kimball - The Data Warehouse Toolkit
- Data Mesh
- Machine Learning System Design Interview
- Streaming Systems
- High Performance Spark
- Building Evolutionary Architectures, 2nd Edition
- Data Management at Scale, 2nd Edition
- Deciphering Data Architectures
- 97 Things Every Data Engineer Should Know: Collective Wisdom from the Experts
- Data Governance: The Definitive Guide
- Trino: The Definitive Guide
- Delta Lake: The Definitive Guide
- Hadoop: The Definitive Guide
- Modern Data Engineering with Apache Spark: A Hands-On Guide for Building Mission-Critical Streaming Applications
- Data Engineering with dbt: A practical guide to building a dependable data platform with SQL
- Data Engineering with AWS
- Practical DataOps: Delivering Agile Date Science at Scale
- Data Engineering Design Patterns
- Snowflake Data Engineering
- Unlocking dbt
Communities:
- Seattle Data Guy Discord
- EcZachly Data Engineering Discord
- Chip Huyen MLOps Discord
- Data Engineer Things Slack
- DBT Community
- r/dataengineering
- Microsoft Fabric Community
- r/MicrosoftFabric
- Data Talks Club Slack
- Data Engineering Wiki
Companies:
- Orchestration
- Data Lake / Cloud
- Data Warehouse
- Data Quality
- Education Companies
- Analytics / Visualization
- Data Integration
- Modern OLAP
Data Engineering blogs of companies:
- Netflix
- Uber
- Databricks
- Airbnb
- Amazon AWS Blog
- Microsoft Data Architecture Blogs
- Microsoft Fabric Blog
- Oracle
- Meta
- Onehouse
Data Engineering Whitepapers:
- A Five-Layered Business Intelligence Architecture
- Lakehouse:A New Generation of Open Platforms that Unify Data Warehousing and Advanced Analytics
- Big Data Quality: A Data Quality Profiling Model
- The Data Lakehouse: Data Warehousing and More
- Spark: Cluster Computing with Working Sets
- The Google File System
- Building a Universal Data Lakehouse
Great YouTube Channels:
- 100k+ subscribers
- 10k+ subscribers
- 1k+ subscribers
Great Podcasts
- The Data Engineering Show
- Data Engineering Podcast
- DataTopics
- The Data Engineering Side Of Data
- DataWare
- The Data Coffee Break Podcast
- Thd datastack show
- Intricity101 Data Sharks Podcast
- Drill to Detail with Mark Rittman
- Analytics Power Hour
- Catalog & cocktails
- Datatalks
- Data Brew by Databricks
- The Data Cloud Podcast by Snowflake
- What's New in data
- Open||Source||Data by Datastax
- Streaming Audio by confluent
- The Data Scientist Show
- MLOps.community
- Monday Morning Data Chat
- The Data Chief
Newsletters:
- DataEngineer.io Newsletter
- Seattle Data Guy
- Joe Reis
- Data Engineering Weekly
- Data Engineering Central
- Dutch Engineer
- ByteByteGo
- Start Data Engineering
- Developing Dev
- High Growth Engineer
- Learn Analytics Engineering
- Marvelous MLOps
- medium Data Engineering Newsletter
- Benn Stancil
- Metadata Weekly
- Technically
- Blef.fr Data News
- All Hands on Data
- Modern Data 101
- SELECT Insights
- Interesting Data Gigs
Glossaries:
- Data Engineering Vault
- Airbyte Data Glossary
- Data Engineering Wiki by Reddit
- Seconda Glossary
- Glossary Databricks
- Airtable Glossary
- Data Engineering Glossary by Dagster
- 100k+ Followers
- 50k+ Followers
- 10k+ Followers
- 5k+ Followers
- 1k+ Followers
Twitter / X
- Zach Wilson
- Seattle Data Guy
- Sumit Mittal
- Joseph Machado
- Alex Xu
- Eric Roby
- Andreas Kretz
- Marc Lamberti
- Dipankar Mazumdar
- Start Data Engineering
- Data Cyborg
- Simon Späti
- Marcos Ortiz
TikTok
Design Patterns
- Cumulative Table Design
- Microbatch Deduplication
- The Little Book of Pipelines
- Data Developer Platform
Courses / Academies
- DataExpert.io course use code HANDBOOK10 for a discount!
- LearnDataEngineering.com
- Technical Freelancer Academy Use code zwtech for a discount!
- IBM Data Engineering for Everyone
- Qwiklabs
- DataCamp
- Udemy Courses from Shruti Mantri
- Rock the JVM teaches Spark (in Scala), Flink and others
- Data Engineering Zoomcamp by DataTalksClub
Certifications Courses
- Google Cloud Certified - Professional Data Engineer
- Databricks - Data Engineer Professional
- Azure Data Engineer Associate
- Microsoft Fabric Analytics Engineer Associate
- Exam DP-203: Data Engineering on Microsoft Azure
- AWS Certified Data Engineer - Associate
Conferences
Open Source Agenda is not affiliated with "Data Engineer Handbook" Project. README Source: DataExpert-io/data-engineer-handbook
Stars
8,524
Open Issues
2
Last Commit
1 month ago
Repository
Open Source Agenda Badge
