Data Pipeline running as incorrect user

Hi all, we have an issue on one of our data pipelines where it seems to be stuck running under a single users' credentials. We use a service account to trigger the pipeline, and the service account has contributor access to the workspace. However, we get an error "Lakehouse failed for forbidden which may be caused by user account or service principal doesn't have enough permission to access Lakehouse".

The run is submitted successfully by the service account, and all other aspects of the pipeline are successful, other than writing to the lakehouse. The "Run by" details on ALL run history shows only my colleagues username. I think this is where things are falling down, as our personal accounts have access automatically removed at the end of each day (hence the use of the service account).

Other pipelines writing to other lakehouses in the same workspace are successful using the same service account, and correctly show the service account in the "Run by" details.

I'm about to submit a MS support ticket but wondering if anyone here has had a similar issue.

Cheers,

Mark

3 comments