Replace regex in a file, in-place, with Common Lisp

Question

I am trying to write the Common Lisp version of Python's regex search and replace, with in-place modification of files:

import fileinput, re

for line in fileinput.input(inplace=1, backup='.bak'):
    line = re.sub(r"foo", "bar", line, re.M)
print (line)

This is the Common Lisp code I was able to think up:

(require :cl-ppcre)

(defun in-place-subst (file)
  (with-open-file (stream file :direction :io :if-exists :overwrite)
    (loop for line = (read-line stream nil)
       while line do
         (write-line (cl-ppcre:regex-replace-all "foo" line "bar") stream))))

It works, sort of. Right now the replacement text will be appended at the end of the file. My immediate problem is that I can't figure out how to replace the content.

To better explain, if file.txt contains:

1 foo
2 bar
3 foobar

after calling

(in-place-subst "file.txt")

I get:

1 foo
2 bar
3 foobar
1 bar
2 bar
3 barbar

Instead of the right replacement:

1 bar
2 bar
3 barbar

I tried with all possible with-open-file options (from Successful Lisp), with no success:

Keyword      Value                Action if File Exists
----------   ------------------   ---------------------------------------
:IF-EXISTS   NIL                  return NIL
:IF-EXISTS   :ERROR               signal an error
:IF-EXISTS   :NEW-VERSION         next version (or error)
:IF-EXISTS   :RENAME              rename existing, create new
:IF-EXISTS   :SUPERSEDE           replace file upon CLOSE
:IF-EXISTS   :RENAME-AND-DELETE   rename and delete existing, create new
:IF-EXISTS   :OVERWRITE           reuse existing file (position at start)
:IF-EXISTS   :APPEND              reuse existing file (position at end)

Could somebody please send me in the right direction, so that the function will render file.txt in the correct way?

Also, what would be the Common Lisp idiomatic way for doing this, assuming of course cl-ppcre is available?

Is there a more succinct way of doing an in-place regex substitution with Common Lisp?

Answer 1

In Python there is no primitive operation that modifies a file “in-place”; instead, there is a function of an helper class fileinput, that gives the illusion of modifying a file in place by first copying the file to a backup file, and then reading the backup file and writing the result of processing it to the original one. From the manual:

Optional in-place filtering: if the keyword argument inplace=1 is passed to fileinput.input() or to the FileInput constructor, the file is moved to a backup file and standard output is directed to the input file (if a file of the same name as the backup file already exists, it will be replaced silently). This makes it possible to write a filter that rewrites its input file in place. If the backup parameter is given (typically as backup='.'), it specifies the extension for the backup file, and the backup file remains around; by default, the extension is '.bak' and it is deleted when the output file is closed. In-place filtering is disabled when standard input is read.

So, the way of doing this operation in Common Lisp is to mimic the Python code, by first copying the file to a backup file, for instance using this function my-copy-file, then writing the following code:

(defun in-place-subst (file)
  (let ((backup-file (concatenate 'string file ".bak")))
    (my-copy-file file backup-file)
    (with-open-file (in-stream backup-file)
      (with-open-file (out-stream file :direction :output :if-exists :supersede)
        (loop for line = (read-line in-stream nil)
           while line do
             (write-line (cl-ppcre:regex-replace-all "foo" line "bar") out-stream))))))