Activity
Mon
Wed
Fri
Sun
Jan
Feb
Mar
Apr
May
Jun
Jul
Aug
Sep
Oct
Nov
Dec
What is this?
Less
More
Memberships
Learn Microsoft Fabric
14.3k members • Free
2 contributions to Learn Microsoft Fabric
Apr '24 •
How do you manage updates/deletes in batch processing
Hi all, I'm new here and in data engineering as well. We are trying Fabric for our organization and I need some advice/recommendations from you. I need to copy data from Azure SQL Database and Azure CosmosDb for Mongo into Microsoft Fabric for further analysis. I'm using Data Pipeline and the Copy action that runs every 12 hours and copies the data (where CreatedAt >= (now-21h). It copies the new data into Warehouse but what about updated data or deleted? How to keep my Warehouse in sync with the SQL and Mongo data? What are the best practices here? I'm not using Spark and want to avoid it if possible. Thank you in advance
1 like • Apr '24
@Will Needham Thanks. Looking forward to see your video with best practices for such cases
0 likes • Apr '24
@Vinayak K thank you very much. could you also suggest about this "get all changed data along with change type details (insert,update,delete) as a column from source" - how to do that?
Apr '24 •
Upsert/incremental refresh using Microsoft Fabric
Microsoft Fabric is still under development for Alter Row Transformation how do you do upsert without this? Also, How can I handle incremental refresh using Fabric?
0 likes • Apr '24
@Matt Roberts could you pls describe it in more details? maybe some example ...
1-2 of 2
