Spark connector for SQL databases (Preview)

Using PySpark, we can now read from and write to Azure SQL Database, Azure SQL Managed Instance, and Fabric SQL Database using the built-in Spark connector. For on-premises SQL Server, connectivity is currently supported only via Azure VM. Direct support for other on-prem setups may be introduced in the future.

Key insights from the blog:

The Spark connector is now preinstalled in the Fabric runtime—no manual setup needed.
It supports PySpark natively, eliminating previous workarounds.
Security features like RLS, CLS, and OLS are respected during access.
Multiple authentication methods are supported for flexibility
https://blog.fabric.microsoft.com/fr/blog/spark-connector-for-sql-databases-preview?ft=Data-engineering:category

0 comments