Search code examples
common-lispslime

Function redefinitions during execution in lisp


Let us suppose we have two functions fct1 and fct2 :

  • fct1 calls fct2,
  • fct1 sets some object O1 in the application to the state A,
  • fct2 sets some object O2 in the application to the state B.

Let us suppose the following constraint which MUST be true all the times :

  • (01 is in A state AND 02 is in B state),
  • XOR (01 is in not(A) state AND 02 is in not(B) state).

What happens if, during a call to fct1 :

  • Redefinition : fct1 now sets some object 01 to the state not(A),
  • Redefinition : fct2 now sets some object 02 to the state not(B).

Can it "breaks" the constraint by setting 01 to the state A and 02 to the state not(B) ?

I've found this answer : https://stackoverflow.com/a/20477763/9614866

If a recursive function redefines itself, the recursive calls still to be made in the same invocation may keep going to the same body. [...] More generally, Common Lisp allows compilers to generate efficient calls among functions that are in the same file. So you normally have to think of replacement of running code as being at the module level rather than individual function level. If functions A and B are in the same module, and A calls B, then if you merely replace B without replacing A, A may continue calling the old B (because B was inlined into A, or because A doesn't go through the symbol, but uses a more direct address for B). You can declare functions notinline to suppress this.

My questions are :

  • Can this phenomenon occur (i.e. 01 set to A state and 02 set to not(B) state) ? Does it have a name ?

If "yes" :

  • Is it implementation-dependent ?
  • Is there a way to force the correct behavior, for instance by inlining a function ?
  • What tool(s) can I use to test if the functions work the correct way ? It seems a pain to test : I have no idea how to test redefinitions without altering the base source code.
  • How can I detect portions of my code which can present this problem ?

Solution

  • Here is an example of how what you describe can happen, in a multithreaded environment:

    (progn
    
      (defun f2 (o2)
        (setf (car o2) :b))
    
      (defun f1 (o1 o2)
        (setf (car o1) :a)
        ;; the sleep here is to increase the risk of data-race
        (sleep 3)
        (f2 o2))
    
      ;; call the functions in a separate thread
      (sb-thread:make-thread
       (lambda () 
         (let ((o1 (list 0))
               (o2 (list 0)))
           (f1 o1 o2)
           (print (list o1 o2)))))
    
      ;; in parallel, redefine f2, then f1
      (defun f2 (o2)
        (setf (car o2) :not-b))
    
      (defun f1 (o1 o2)
        (setf (car o1) :not-a)
        (f2 o2)))
    

    After 3 seconds, the REPL prints

    ((:A) (:NOT-B))
    

    If you add (declaim (inline f2)) before f2 is defined and test again, then the code from old f2 is still executed from old f1, inside the thread, which prints the following after 3 seconds:

    ((:A) (:B)) 
    

    Further invocations of the updated function f1 give:

    ((:NOT-A) (:NOT-B)) 
    

    What tool(s) can I use to test if the functions work the correct way ? It seems a pain to test : I have no idea how to test redefinitions without altering the base source code.

    Maybe you are updating a running server with new code, and you want to avoid having the server use partial redefinitions of functions while you load the definitions.

    Like every other aspects of your infrastructure, it is important to plan ahead how to make backups and updates reliably (databases, configuration, etc.).

    One possible way is to update things package by package. You could suffix your package with a version number:

    (in-package :web-0 ...)
    (defun f2 () ...)
    (defun f1 () ...)
    
    ;; new version
    (in-package :web-1 ...)
    (defun f2 () ...)
    (defun f1 () ...)
    

    When it is time to update, you can compile and load the code for :web-1 without interfering with the running code. Then you should be able to change the callers to use the new implementation:

    ;; somewhere in the main server package
    (handle-request (request)
       (web-0:handle request))
    
     ;; update it
    (handle-request (request)
      (web-1:handle request))
    

    It should even work without version numbers, if you first delete the package then recreate it with the same name, but then you cannot recover easily.

    Some places might also need a global lock that you have to manage both in your application and during the update.