Activity
Mon
Wed
Fri
Sun
Jun
Jul
Aug
Sep
Oct
Nov
Dec
Jan
Feb
Mar
Apr
May
What is this?
Less
More

Memberships

Learn Microsoft Fabric

Public • 1.5k • Free

9 contributions to Learn Microsoft Fabric
Copy Data from Lakehouse in Another Workspace Using Data pipeline
The Problem: out-the-box Fabric Data Pipelines can't write from a Lakehouse in one workspace to another Lakehouse in another workspace - this is something I noticed a while back and logged a Fabric Idea (which apparently they are now working on a solution). In the meantime though, Microsoft released an article today with an interesting workaround, which looks like it solves the above problem: read more here: https://blog.fabric.microsoft.com/en-US/blog/copy-data-from-lakehouse-in-another-workspace-using-data-pipeline/
17
12
New comment 7d ago
0 likes • 8d
Would handy is there was also a way to reference views from workspace to workspace similar to shortcuts but can’t see a way so far
1 like • 7d
@Julio Ochoa Hi, yes you are correct, views are just logical joins of tables so in theory to be able to reference them from other workspaces would mean you need to bring in all the underlining tables used by the view as shortcuts too. However there are scenarios when using warehouses (more likely in Gold layer) where you want to reference just a view from another location workspace/warehouse especially if you need to expose a flat data view for business users when using a semantic model may not be ideal. My understanding is shortcuts are just persisted tables which are copied over from the source in the background and managed by delta log changes thereafter, in theory making views also persisted in this way should be possible.
Current Open Job Listings mentioning Fabric
There's been a few questions/ discussions about future jobs within Fabric. We're beginning to see companies recruiting for Fabric roles... exciting! Here's a few I found: Lead Data Modeler - Microsoft Fabric Senior Data Engineer (Microsoft Fabric) Data Engineer (Microsoft Fabric) GERMAN Microsoft Fabric Engineer (BELGIUM) Senior Consultant - Databricks and Microsoft Fabric Data & Analytics Engineer Principal and Senior consultant level Microsoft Fabric Data & Analytics Consultants I had the idea to create a video reviewing some of these job ads and picking out the common themes, to help people understand what they might need to focus on to get jobs in the future - what do you think of that concept? What you find that interesting? Let me know!
24
16
New comment 7d ago
4 likes • 9d
I recently finished a contract working on a Fabric implementation and have been trying to find a new contract opportunity but have been surprised with the lack or opportunities in UK at the moment, it seems it’s still just a bit too new for demand to be realised, hopefully will change over the next few months 👌
0 likes • 8d
@Justin Stevenson I think that’s a fair point however I also believe there is a fundamental gap in skill inbetween the data engineering side of things and the Power BI development side. I’ve worked a lot with the arguably now outdated SSIS/SSAS toolset and one area I’ve seen being somewhat neglected is the modelling side of things, whether that’s due to lack of experience from developers or because it’s easier to cover up poor performance from bad model design with increased cpu power that the cloud offers I’m not sure, I suspect a little bit of both. The one thing Fabric offers which I think is understated at the moment is that SQL Managment studio experience wrapped over the performance benefits of a delta lake. This makes it easier for old school seasoned SSAS developers to break into the modern stack of Fabric and be able to create well modelled warehouse fact/dimension tables outside of PBI and leave the PBI developers to focus on the DAX and visual side of things, right now it just seems data goes from Data engineers and then gets picked up PBI developers with a lack of attention for the modelling side of things. Quite a ramble so apologies 😆
#dataengineering vs # data analytics
is azure data engineering still in demand in market or with unified approach of microsoft fabrics it reduced data engineering jobs in market any one please kindly answer im confused which one to choose for
3
11
New comment 19d ago
3 likes • Apr 18
I also think Fabric hugely increases adoptability of cloud/data lake implementation for businesses that maybe didn’t have the in-house python and architecture skill sets for developing a data lake solution. One of the biggest wins for Fabric is that it effectively gives you a SQL server wrapper over the data/delta lake solution, making it very flexible and much easier for experienced on-prem SQL server developers to make that shift, just my thoughts anyhow 🙂
0 likes • 19d
I do think there is an in between role that is valid, an Analytics engineer, someone who knows modelling and is responsible to bring in the data from silver layers into gold layer/reporting layer. Data engineers may not always appreciate the nuances of data transformation from silver data into a star schema model, data analysts may also not be experienced enough in the specifics of how best to design star schema tables but are experts in using the modelled data and applying more complex calculations on top of it.
Meta data driven pipelines
Hi Will, What are your thoughts on meta data drive approach to creating data pipelines from a lakehouse table? You mentioned this point in one of your videos. Perhaps an idea for one of your next videos with worked out examples?
6
10
New comment 29d ago
1 like • Apr 18
@Eivind Haugen thanks for the link, we currently working on a meta data driven solution to improve our implementation. However it would be good to look at actually potential creating all objects for an environment purely in code, so you could actually create/spin up an entire environment at will just from meta data. I think being able to create workspaces from some form of base template would be interesting, currently I see a gap in being able to easily create test workspaces with data for development purposes
0 likes • Apr 19
That’s interesting will check it out, many thanks!
Is P1 Capacity still an option?
I have a client who just recently bought a P1 Capacity to handle incremental loading through their Dataflows. They are new to Power BI and are looking to use the Dataflows to pull data from Oracle ADW to migrate their existing reports into Power BI. My argument is they should opt for Fabric now (still green field for them) to take advantage of Gen 2, etc... BUT their CIO is wary of changes, and they just made a change to Power BI and don't want to change further if they can help it. Question - Should I call for an emergency halt? Or let them continue down their path of using P1 knowing they'll need to switch at a later date? Again - they are very new to BI, with their main focus of migrating their existing reports into Power BI. Once done, then look towards a new BI modernization play...
1
9
New comment 9d ago
1 like • Apr 19
Fabric has CDC functionality in Delta tables, might make sense making use of that for their implementation
1-9 of 9
Rafal Andrearczyk
3
43points to level up
@rafal-andrearczyk-6581
MS BI Developer with 15years experience, now moving into the world of Fabric.

Active 6d ago
Joined Apr 3, 2024
powered by