I’ve been pretty quiet the past couple of months because I’ve been deep in Python, AI engineering skills, while staying steady with Fabric. I’m grateful to share that I’ve accepted the Senior AI Engineering role at my institution — something I’ve been working toward relentlessly since January. Thanks to everyone who shares their work here; it’s been a huge part of how I’ve kept moving.

Hey everyone! Have you been keeping up-to-date with the latest generation of AI models? If you HAVEN'T, then I recommend checking them out - they are insane to be honest - I'm talking Claude Opus 4.5, Gemini 3.0, GPT 5.1 etc. Has huge implications for how we will work in the future. I'm just figuring out the best workflows for implementing these models in Fabric workflows, I'll share more of my thinking and research in the following weeks/ months, but I'm interested - have you been keeping up-to-date with the LLM updates? Have you been thinking about how to integrate into Fabric - let me know what you've found so far!

Hey everyone, I wanted to know if anyone can provide some advice to a problem I have encountered. Scenario I conducting a review our institutions recruitment campaigns. To do this I have been given access to a SharePoint folder which contains a daily extract of recruitment data. These extracts are csv files and are around 50MB. Every day at around 5am a csv extract appears with yesterdays recruitment data. I want to create from these files a simple dataset, with the application, offer and new registration numbers for each date and possibly show this with some other dimensions, like domicile country etc. Data Flow Gen 2 won't work as it can't handle such a large volume of data. I thought a data pipeline would work as if I can perform the aggregation to one csv extract I could then use the for each loop to perform this on all of the other extracts. Any help please?

Hi Guys, I have noticed a problem with a solution I have deployed using Fabric. I am using a trial capacity (FT1). I am using a medallion architecture with a data pipeline to orchestrate this process. On the bronze layer, I have a dataflow that pulls the latest data from a web source as a raw table and store this data in a lakehouse (this seems to always work). However there is another dataflow that pulls the data from the bronze lakehouse and then performs cleansing on this source. This is where the problems start as some of the time this works just fine and the latest raw data from the bronze lakehouse is cleansed into my silver lakehouse. However there are times where it doesn't bring in the latest data pulled from the bronze lakehouse. This is the M-code to access the bronze lakehouse data: let Source = Lakehouse.Contents([]), #"Navigation 1" = Source{[workspaceId = "c79766a3-4f30-43d3-942c-d1fa4e84b64d"]}[Data], #"Navigation 2" = #"Navigation 1"{[lakehouseId = "72ab087e-c6f4-40dd-8149-95bd9c8e5bb5"]}[Data], #"Navigation 3" = #"Navigation 2"{[Id = "Files", ItemKind = "Folder"]}[Data], It is very frustrating as it makes no sense.

Microsoft has released the official self-paced course for DP-700. Check it out here👉🏽 Course DP-700T00-A: Microsoft Fabric Data Engineer This course will eventually become the recommended self-paced material on the Fabric Data Engineer Associate certification page. So, if you plan on taking the exam once it goes GA, you might as well take advantage of this course and start preparing now. Also, find some helpful DP-700 discussions below: - DP-700 beta exam experiences and tips (by Ali Stoops) - DP 700 - Study Guide & Learn Collection When are you planning to take the DP-700 exam? Let us know how you’re preparing below👇🏽