We're building a Real-World Fabric Platform (10 week build!)
Hey everyone, it's Will here. I hope you are having a wonderful week! Just wanted to share some exciting updates from our second community Fabric Dojo - we are just about to start building a Real-World Fabric Solution, over a 10-week period. So if anyone is looking to gain real-world experience with Fabric (to help you get/ excel in a job) - then this will be a great opportunity for you! This project is designed to simulate the type of Fabric solution you would build within a real enterprise environment. You will be challenged to think, design, and deliver like a professional Fabric Data Engineer — from strategy through to implementation. By the end of this project, you will: - Understand how all engineering components of Microsoft Fabric interact as part of a coherent, end-to-end data platform - Develop the ability to move from business outcomes and client requirements, through solution architecture design, and into the build and delivery phase - Experience a realistic project lifecycle, including iteration, validation, refinement, and presentation Here's the schedule for the next 10 weeks or so: - Project Kick-off (15 Oct - but you can catch the recording): Introduction to the real-world client, project context, and goals - Sprint 1 (20–27 Oct): Capacity & Workspace Design - Define Fabric capacities and workspace strategy based on client needs - Sprint 2 (27 Oct – 3 Nov): CI/CD & Automation - Establish GitHub repository & develop CI/CD & Automation strategy, based on the client requirements. - Sprint 3 (3–10 Nov): Data Architecture & Data Stores - Design data architecture and implement Fabric data stores - Sprint 4 (10–17 Nov): Data Extraction - Build ingestion pipelines from client source systems (in some instances, synthetic data will be provided to protect sensitive data) - Sprint 5 (17–24 Nov): Data Transformation - Design a scalable transformation framework across architectural layers - Sprint 6 (24 Nov – 1 Dec): Data Validation & Quality - Implement data validation framework and centralised DQ monitoring - Sprint 7 (1–8 Dec): Orchestration - Build orchestration layer to integrate extraction, transformation, and validation processes - Sprint 8 (8–15 Dec): Logging & Alerting - Implement operational logging and alerting across the platform - Extension Sprint 9 (15 Dec – 5 Jan): Further Development - For those that want to push their skills even further, I'll be providing a number of extension tasks to be completed over the Christmas period. These will include: developing a end-to-end testing framework & developing Infrastructure-as-Code templates to support future platform evolution. - Project Showcase & Closeout (5 Jan): Present your final solution to “the client” (the community and Will) & Celebration and next steps discussion