Inconsistent behavior while reading from and writing to delta parquet tables using Pyspark in notebooks

Inconsistent behavior while reading from and writing to delta parquet tables using Pyspark in notebooks. I am using the built-in notebooks to work on the lakehouse data and the same pyspark script to read from or write to a delta table will work sometimes and not at other times. Anyone else having this issue? and the errors are cryptic and not particularly useful.

6 comments