Mismatch in location of files saved via direct vs scheduled runs


I have a notebook that has a function to write a dataframe into a .csv file. It’s as simple as df.to_csv('file_name.csv').

When I run the notebook directly, the file will be saved under /data/notebook_files/file_name.csv.

However, when the notebook runs on a schedule, the .csv file will be saved under /data/workspace_files/file_name.csv instead. Even when I specify the path as df.to_csv('/data/notebook_files/file_name.csv'), it will still be saved under workspace_files instead of notebook_files.

This seems like a bug to me. The file should be saved in whatever path that is specified in the notebook, regardless of whether it’s a direct or scheduled run.