Search code examples
c#delphistreamreaderstreamwriter

Is it possible to read from and write to the same text file in C#

Here is the situation. I have over 1500 SQL (text) files that contain a "GRANT some privilege ON some table TO some user after the CREATE TABLE statement. I need to remove them from the original file and put the GRANT statements in their own file. Sometimes the Grants are on a single line and sometimes they are split across multiple lines. For example:

  GRANT SELECT ON XYZ.TABLE1 TO MYROLE1 ;
  GRANT
  SELECT ON XYZ.TABLE1 TO MYROLE2 ;
  GRANT
  DELETE,
  INSERT,
  SELECT,
  UPDATE ON XYZ.TABLE1 TO MYROLE3;

I am reading through the file until I get to the GRANT and then building a string containing the text from the GRANT to the semicolon which I then write out to another file. I have an app I wrote in Delphi (Pascal) and this part works great. What I would like to do is after I read and have processed the line I want, I would like to delete the line from the original text file. I can't do this in the Delphi. The only solution there is to read the file line by line and write the file back out to another file excluding the lines I don't want while also writing the GRANTS to yet another file. Then delete the original and rename the new. Way too much processing and risk.

I looked at using the StreamReader and StreamWriter in C# but it appears to be similar situation to Delphi. I can Read or I can Write but I can't do both to the same file at the same time.

I would appreciate any suggestions or recommendations.

Thanks

Solution

  • If you think there's "way too much processing and risk" in generating the a new temporary file without the lines you don't want and replacing the original: then consider the alternative you're hoping to achieve.

        Line 1
    Line 2
    Delete this line
+-->Line 4
|   Line 5
|
+- Read position marker after reading line to be deleted

    If you immediately delete the line while reading, the later lines have to be moved back into the "empty space" left behind after the 3rd line is deleted. In order to ensure you next read "Line 4", you'd have to backtrack your read-position-marker. What's the correct amount to backtrack? A "line" of variable length, or the number of characters of the deleted line?

    What you perceive to be the "risky" option is actually the safe option!

    If you want to delete while processing you can use an abstraction that gives you that impression. But you lose the benefits of stream processing and don't really eliminate any of the risk you were worried about in the first place.

    E.g. Load your entire file into a list of strings; such as an array, vector or TStringList (in Delphi). Iterate the list and delete the items you don't want. Finally save the list back to file.

    This approach has the following disadvantages:

    • Potential high memory overhead because you load the entire file instead of small buffer for the stream.
    • You're at risk of mid-process failure with no recovery, because your job is all-or-nothing.
    • You have to deal with the nuances of the particular container you choose to hold your list of strings.
      • In some cases (e.g. TStringList) you might still need to backtrack your position marker in a similar fashion to the earlier description.
      • For arrays you'd have to copy all lines back 1 position every time you delete something with a huge performance cost. (The same happens in TStringList though it's hidden from you.)
      • Iterators for some containers are invalidated whenever you modify the list. This means you'd have to copy to a new list without the 'deleted lines' in any case. More memory overhead.

    In conclusion, take the safe option. Use a separate read and write stream; write to a temporary file, and rename when done. It will save you headaches.