Why is this global variable only able to be set twice, and no more?

Question

I was trying to make a simple parser in elisp, and I encountered a problem where I defvar a global, then later I setq a new value into it. This works the first time. However, later setq's fail every time.

The following code is a simplication of the problem:

(defvar buf '(BUF))
(defvar head nil)

(defun pparse (seq)

  (defun status ()
    (princ (format "Parse: %40s || %-20s\n"
           (prin1-to-string seq) (prin1-to-string buf))))

  (while seq
    (status)
    (setq head (car seq))
    (setq seq (cdr seq))
    (cond ((equal "x" head)
       (nconc buf (list head)))
      ((equal "," head)
       (setq buf '(BUF))
       ;;(setcdr buf nil) <- fixes it but doesn't answer my question
      )))

  (status))

(pparse '("x" "," "x" "," "x" "," "x"))

Which produces this output:

Parse:            ("x" "," "x" "," "x" "," "x") || (BUF)               
Parse:                ("," "x" "," "x" "," "x") || (BUF "x")           
Parse:                    ("x" "," "x" "," "x") || (BUF)               
Parse:                        ("," "x" "," "x") || (BUF "x")           
Parse:                            ("x" "," "x") || (BUF "x")           
Parse:                                ("," "x") || (BUF "x" "x")       
Parse:                                    ("x") || (BUF "x" "x")       
Parse:                                      nil || (BUF "x" "x" "x")   

As you can see, the second column was clipped once, but grew every subsequent time.

If you uncomment the setcdr line, this works as expected (output is as follows). You can even dump the setq. I understand why this fixes it, but not why the original error happens in the first place.

Parse:            ("x" "," "x" "," "x" "," "x") || (BUF)               
Parse:                ("," "x" "," "x" "," "x") || (BUF "x")           
Parse:                    ("x" "," "x" "," "x") || (BUF)               
Parse:                        ("," "x" "," "x") || (BUF "x")           
Parse:                            ("x" "," "x") || (BUF)               
Parse:                                ("," "x") || (BUF "x")           
Parse:                                    ("x") || (BUF)               
Parse:                                      nil || (BUF "x")           

BTW, the behavior is the same even if i turn off lexical scoping.

Answer 1

You cannot mutate literal data, such as '(BUF), and expect sane results. And in your code, the nconc is a mutating operation.

In terms of the "why" you're seeing the behaviour you're seeing, it's due to the (setq buf '(BUF)) expression. That sets it to the same object each time, because it's a literal datum—which you're not supposed to mutate with the likes of nconc. If you changed it to (setq buf (list 'BUF)), then it would generate a new object each time, and you can safely nconc that.