databricks throws OSError: [Errno 95] Operation not supported when appending to azure blob file

Question

Doing some file manipulation on databricks and it's failing due to the following error when using open('\mnt\file\','a')

OSError: [Errno 95] Operation not supported

however, it will succeed if open('\mnt\file\','w'). How can "a" not be supported on azure blob? Any workaround here?

Answer 1

That is because the files in Azure blob storage are immutable, which means you can not append them. but you can overwrite it which will rewrite the file after the changes.

Example Python code

`file_path = '/mnt/file/'  # Your Azure Blob storage mount path

# Read the existing data
with open(file_path, 'r') as file:
    existing_data = file.read()

# Modify the existing_data as needed
modified_data = existing_data + "New data to append"

# Write the modified data back to the file, effectively overwriting it
with open(file_path, 'w') as file:
    file.write(modified_data)

`