I am using a notebook to load an Excel file (downloaded from a website) into a folder in the Files area of a lakehouse - I thought this would be pretty straightforward, but I must be missing something:
from datetime import datetime
import pandas as pd
url = "https://<url_of_excel_file"
output_path = "Files/sales_targets/" + datetime.now().strftime("%Y%m%d")
# load Excel file from URL and replace spaces in column names
df = pd.read_excel(url)
df.columns = df.columns.str.replace(' ','')
# create directory if it doesn't exist
mssparkutils.fs.mkdirs(output_path)
df.to_excel(output_path + "/targets.xlsx")
Is df.to_excel the correct method here, or should I be using PySpark instead?