Today, I decided to make a small benchmark between these 3 tools 🎯 Benchmark setup 2 data volumes: 100k and 5M rows 3 scenarios: Simple Intermediate (adding columns, filters, etc.) Advanced (joins, complex calculated columns, window functions, indexes…) 3 tools compared: Dataflow Gen2 Spark notebook T‑SQL in the warehouse The detailed results are shown in the attached chart.​ 📌 Key takeaways 1) T‑SQL is the clear winner Across all scenarios, it’s either the fastest or very close to the top. Even for complex transformations (window functions, aggregations, joins), the Fabric SQL engine is extremely efficient. 2) Dataflow Gen2: very sensitive to volume 100k rows: 35–41 s → acceptable. 5M rows: 39–105 s → execution time explodes. 3) Spark notebook: stable but never first Execution time is fairly consistent (~40–53 s), but it doesn’t dominate the advanced 5M‑row scenario like I expected. It’s also penalized by ~10 s of cluster startup… 4) Scale completely changes the picture At 100k rows, everything looks “fast” (3–41 s). At 5M rows, the gap grows to 1× vs 6× between tools. 5) CU consumption: similar Across the tests, all three options sit in a similar range of about 9–11 CU for the workloads. The big difference: if you don’t stop the Spark notebook, it keeps consuming CUs at the same level even when it’s just “waiting”, whereas T‑SQL and Dataflow Gen2 stop consuming once the workload is done. My personal verdict Strong SQL team → T‑SQL Fastest, most stable, and very competitive even for complex workloads. Need low‑code / Power Query team → Dataflow Gen2 Great tool, but you really need to watch Query Folding carefully. Need predictable scalability / data engineering use cases → Spark notebook Stable performance, ideal when you want to avoid surprises. Any of you already make this kind of benchmark ? What was your results ? Ps: if you want to support me, do not hesistate to share or like my linkedin post ;-)

Datamarts have met their inevitable end... after a 3-year Public Preview. Read more here: https://powerbi.microsoft.com/en-us/blog/unify-datamart-with-fabric-data-warehouse/ The author of this article should go into politics! 😉 What are your thoughts?!

Hey everyone - yesterday the DP-700 exam became generally available, PLUS Microsoft announced 20,000 50% off vouchers to take the exam... so you might be wondering, what's the plan?! With the previous DP-600 exam, we managed to help hundreds, or maybe even thousands of you certified, which is awesome! And for the DP-700, I have plans to go even bigger! Therefore, this is a summary of what's planned: 📽️ Like with the DP-600 exam, I will be producing videos for the YouTube channel to walk you through each section of the DP-700 study guide - these videos will also be added to the Classroom here in the LMF community, and discussion posts made for conversations about the topics! I hope to start releasing these in the next few weeks. As well as the videos, this time, I'll also be hosting a DP-700 bootcamp (from 19th Jan to 30th March) within Fabric Dojo for those that want more support, this includes: - Access to 11 LIVE (online) study sessions held between 19th Jan and 30th March - Access to summary study notes for each section of the exam. - Plus, as a member of Fabric Dojo, you will get access to the 90+ hands-on tutorials and projects in the Dojo, 60 of which are relevant for DP-700 learning. Practical experience really makes a difference when you're sitting the exam. This is what I spent 6 months working on last year! It's nearly 30 hours of exclusive content and hands-on learning related to Fabric Data Engineering and Analytics Engineering.