Activity
Mon
Wed
Fri
Sun
Jan
Feb
Mar
Apr
May
Jun
Jul
Aug
Sep
Oct
Nov
Dec
What is this?
Less
More

Memberships

Learn Microsoft Fabric

14.1k members • Free

15 contributions to Learn Microsoft Fabric
We're building a Real-World Fabric Platform (10 week build!)
Hey everyone, it's Will here. I hope you are having a wonderful week! Just wanted to share some exciting updates from our second community Fabric Dojo - we are just about to start building a Real-World Fabric Solution, over a 10-week period. So if anyone is looking to gain real-world experience with Fabric (to help you get/ excel in a job) - then this will be a great opportunity for you! This project is designed to simulate the type of Fabric solution you would build within a real enterprise environment. You will be challenged to think, design, and deliver like a professional Fabric Data Engineer — from strategy through to implementation. By the end of this project, you will: - Understand how all engineering components of Microsoft Fabric interact as part of a coherent, end-to-end data platform - Develop the ability to move from business outcomes and client requirements, through solution architecture design, and into the build and delivery phase - Experience a realistic project lifecycle, including iteration, validation, refinement, and presentation Here's the schedule for the next 10 weeks or so: - Project Kick-off (15 Oct - but you can catch the recording): Introduction to the real-world client, project context, and goals - Sprint 1 (20–27 Oct): Capacity & Workspace Design - Define Fabric capacities and workspace strategy based on client needs - Sprint 2 (27 Oct – 3 Nov): CI/CD & Automation - Establish GitHub repository & develop CI/CD & Automation strategy, based on the client requirements. - Sprint 3 (3–10 Nov): Data Architecture & Data Stores - Design data architecture and implement Fabric data stores - Sprint 4 (10–17 Nov): Data Extraction - Build ingestion pipelines from client source systems (in some instances, synthetic data will be provided to protect sensitive data) - Sprint 5 (17–24 Nov): Data Transformation - Design a scalable transformation framework across architectural layers - Sprint 6 (24 Nov – 1 Dec): Data Validation & Quality - Implement data validation framework and centralised DQ monitoring - Sprint 7 (1–8 Dec): Orchestration - Build orchestration layer to integrate extraction, transformation, and validation processes - Sprint 8 (8–15 Dec): Logging & Alerting - Implement operational logging and alerting across the platform - Extension Sprint 9 (15 Dec – 5 Jan): Further Development - For those that want to push their skills even further, I'll be providing a number of extension tasks to be completed over the Christmas period. These will include: developing a end-to-end testing framework & developing Infrastructure-as-Code templates to support future platform evolution. - Project Showcase & Closeout (5 Jan): Present your final solution to “the client” (the community and Will) & Celebration and next steps discussion
We're building a Real-World Fabric Platform (10 week build!)
0 likes • Oct 17
Hi @Will Needham, I have a couple of questions: 1. During the period, how much time would we need to put in either daily or each sprint to take part? 2. Will you prepare recordings of a template approach for an option to just watch summaries of each sprint's outputs?
Databricks vs MS Fabric or Hybrid
Hi all, I have a client that is currently using databricks and power bi. They have multiple clusters of databricks running and want to consolidate into one analytics platform. I've suggested Fabric, what would be the reasons for migrating over to fabric instead of using databricks? or would a hybrid approach be better, by using fabric and databricks? Will Fabric be cheaper to run as one platform? Please let me now your thoughts.
0 likes • Feb 27
I'd welcome someone sharing some links to articles and blogs covering these concerns. I'm on the cusp of a full commitment to a Fabric migration (not from Databricks) and am thus obviously concerned
I took the DP-700 today and I passed! 🥳
Hey all, I took the DP-700 today and passed! I will write a proper post with my experiences when I get home later today, but I’ll say it’s definitely a thorough test of your skills! Look forward to now switching to thinking about how best to teach the content to all of you! 🙌🏽
I took the DP-700 today and I passed! 🥳
1 like • Jan 22
Congratulations Will!
SQL in Fabric | Can you use SQL Server replication direct to Fabric
I'm looking into getting on-prem SQL tables into Fabric. We're looking at mirroring but that currently seems to rely on a sort of workaround using replication to Azure SQL before mirroring: Mirroring SQL Server database to Fabric | Microsoft Fabric Blog | Microsoft Fabric as Will posted here: Mirroring for on-prem SQL Server · Learn Microsoft Fabric. Given there is now SQL directly in Fabric: Announcing SQL database in Microsoft Fabric (Preview) | Microsoft Fabric Blog | Microsoft Fabric. Can you not set up replication for the on-prem SQL Server directly to SQL in Fabric? Has anyone tried it? I currently am trying it but hitting a roadblock because our SQL Server is pre-2022 and doesn't support Entra Authentication. I'm wondering whether it is worth pushing the relevant team to upgrade but don't want to if it won't work anyway.
1 like • Jan 8
Just to confirm for anyone else interested. If your SQL Server Version supports Entra Authentication (Version 2022 onward), you can replicate directly from an on prem DB into Fabric, right now. We've just done it. It skips the need to go via Azure SQL at all. We have the data in parquet in Fabric within seconds. I think we'll then shortcut to our Bronze Lakehouse to treat all sources consistently
0 likes • Jan 8
I didn't personally do it, but the steps at a high level are below. There are good article like this that cover it: SQL Server Replication: Overview & Step-by-Step Configuration @Shaun Edwards actually did the work, but not sure how active he is on this platform. @Shaun Edwards - Did I miss anything or are there any Fabric idiosyncrasies people should know? 1. Ensure source on-prem SQL Server is on the right version (2022 onward) to allow Entra Authentication. 2. Create SQL in Fabric that will be the subscriber DB 3. Create a distribution: Define the distribution (name, etc.). 4. Create a publication: Define the database articles to be replicated (tables, views, stored procedures). 5. Configure subscribers. 6. Initialize subscribers. 7. Start replication.
Current Limitations on Mirroring | Table Names and Column Names
Hi there all, Happy New Year! I've reviewed documentation here: Limitations and Behaviors for Fabric Mirrored Databases From Azure SQL Database - Microsoft Fabric | Microsoft Learn and here: Limitations for SQL database (preview) - Microsoft Fabric | Microsoft Learn I'm under the impression that: - Table Names are now supported with spaces: Fabric September 2024 Monthly Update | Microsoft Fabric Blog | Microsoft Fabric and thus, mirroring with Tables with spaces should work too. - Column Names with spaces or other special characters is still not supported and thus, mirroring Tables with such columns will not work. Do I have that right? I don't know if there is any desire from Microsoft to try and accommodate them? does anyone know about any workaround to allow mirroring of a DB that does have special characters/spaces in column names?
1-10 of 15
Simon Tidd
3
44points to level up
@simon-tidd-9978
I am a manager of a Project Office and a Business Intelligence team, for a health retail company. I live and work from Guernsey.

Active 2d ago
Joined Jul 25, 2024
ESTJ
Guernsey
Powered by