Best practice for writing big files

Question

I need to write a big file in my project.

---

What I learned:

• I should NOT write the big file directly to the destination path, because this may leave a incomplete file in case the app crash while writing it.

• Instead, I should write to a temporary file and move (rename) it. (called atomic file operation)

---

My code snippet:

[NotNull]
public static async Task WriteAllTextAsync([NotNull] string path, [NotNull] string content) 
{
    string temporaryFilePath = null;
    try {
        temporaryFilePath = Path.GetTempFileName();
        using (var stream = new StreamWriter(temporaryFilePath, true)) {
            await stream.WriteAsync(content).ConfigureAwait(false);
        }            

        File.Delete(path);
        File.Move(temporaryFilePath, path);
    }
    finally {
        if (temporaryFilePath != null) File.Delete(temporaryFilePath);
    }
}

---

My Question:

• The file will be missing if the app crashes between File.Delete and File.Move. Can I avoid this?

• Is there any other best practice for writing big files?

• Is there any suggestion on my code?

Answer 1

The file will be missing if the app crashes between File.Delete and File.Move. Can I avoid this?

Not that I'm aware of, but you can detect it - and if you use a more predictable filename, you can recover from that. It helps if you tweak the process somewhat to use three file names: the target, a "new" file and an "old" file. The process becomes:

• Write to "new" file (e.g. foo.txt.new)
• Rename the target file to the "old" file (e.g. foo.txt.old)
• Rename the "new" file to the target file
• Delete the "old" file

You then have three files, each of which may be present or absent. That can help you to detect the situation when you come to read the new file:

• No files: Nothing's written data yet
• Just target: All is well
• Target and new: App crashed while writing new file
• Target and old: App failed to delete old file
• New and old: App failed after the first rename, but before the second
• All three, or just old, or just new: Something very odd is going on! User may have interfered

Note: I was unaware of File.Replace before, but I suspect it's effectively just a simpler and possibly more efficient way of doing the code you're already doing. (That's great - use it!) The recovery process would still be the same though.