I'm trying to migrate an SSIS packakage that populates a fact table in MS SQL to Fabric. This is part of a larger migration project where I've successfully used Pipelines and Gen2 dataflows to populate the staging and dimension tables in MS SQL. I'm now trying to use that data to populate the Fact table but experiencing issues. All my source tables are from a single server in MS SQL. My staging table has 40+ columns and 2M+ rows. Transformations include 20+ left-joins, replacing nulls and adding sys tracking columns (date and string) In SSIS we merged the staging table with each dimension table but recreating this in a Gen2 dataflow is causing performance issues and does not run at all when triggered within a pipeline. I'm wondering if I should consider using notebook or spark definition. Pipelines don't seem to allow me to perform the more complex transformations required and the Gen2 Dataflows don't seem capable of handling my data volume.