Search code examples
delphiunicodedelphi-xe7

How to save classic Delphi string to disk (and read them back)?


I want to achieve a very very basic task in Delphi: to save a string to disk and load it back. It seems trivial but I had problems doing this TWICE since I upgraded to IOUtils (and one more time before that... this is why I took the 'brilliant' decision to upgrade to IOUtils).

I use something like this:

procedure WriteToFile(CONST FileName: string; CONST uString: string; CONST WriteOp: WriteOperation);    
begin
   if WriteOp= (woOverwrite)
   then IOUtils.TFile.WriteAllText (FileName, uString)  //overwrite
   else IOUtils.TFile.AppendAllText(FileName, uString); //append
end;    

Simple right? What could go wrong? Well, I recently stepped into a (another) bug in IOUtils. So, TFile is buggy. The bug is detailed here.

Anyone has can share an alternative (or simply your thoughts/ideas) that is not based on IOUtils and it is known to work? Well... the code above also worked for a while for me... So, I know if difficult to guaranty that a piece of code (no matter how small) will really work!

Also I would REALLY like to have my WriteToFile procedure to save the string to an ANSI file when it is possible (the uString contains only ANSI chars) and as Unicode otherwise.
Then the ReadAFile function should automagically detect the encoding and correctly read the string back.
The idea is that there are still text editors out there that will wrongly open/interpret an Unicode/UTF file. So, whenever possible, give a good old ANSI text file to the user.

So:
- Overwrite/Append
- Save as ANSI when possible
- Memory efficient (don't eat 4GB of ram when the file to load is 2GB)
- Should work with any text file (up to 2GB, obviously)
- No IOUtils (too buggy to be of use)


Solution

  • Then the ReadAFile function should automagically detect the encoding and correctly read the string back.

    This is not possible. There exists files that are well-formed if interpreted as any text encoding. For instance see The Notepad file encoding problem, redux.

    This means that your goals are unattainable and that you need to change them.

    My advice is to do the following:

    • Pick a single encoding, UTF-8, and stick to it.
    • If the file does not exists, create it and write UTF-8 bytes to it.
    • If the file exists, open it, seek to the end, and append UTF-8 bytes.

    A text editor that does not understand UTF-8 is not worth supporting. If you feel inclined, include a UTF-8 BOM when you create the file. Use TEncoding.UTF8.GetBytes and TEncoding.UTF8.GetString to encode and decode.