Dynamic Binding and Macros

Consider the following code:

CL-USER> (defmacro sum (a b)
           (+ a b))
SUM
CL-USER> (let ((alpha 3) (beta -1))
           (sum alpha beta))
; in: LET ((ALPHA 3) (BETA -1))
;     (SUM ALPHA BETA)
; 
; caught ERROR:       
;   during macroexpansion of (SUM ALPHA BETA). Use *BREAK-ON-SIGNALS* to intercept.
;   
;    Argument X is not a NUMBER: ALPHA

;     (LET ((ALPHA 3) (BETA -1))
;       (SUM ALPHA BETA))
; 
; caught STYLE-WARNING:
;   The variable ALPHA is defined but never used.
; 
; caught STYLE-WARNING:
;   The variable BETA is defined but never used.
; 
; compilation unit finished
;   caught 1 ERROR condition
;   caught 2 STYLE-WARNING conditions
; Evaluation aborted on #<SB-INT:COMPILED-PROGRAM-ERROR {10030E4223}>.

There are basically two reasons (that I could think of), which contribute to the failure of this code:
1. The macro sum is first evaluated upon two variables alpha and beta, which are sent as symbols into the macro. So, the code to be evaluated inside the macro is:

(+ 'alpha 'beta)

Which won't work, because we can't add two symbols.
2. The variables alpha and beta are lexically bound, because of which, the code of the macro can't access the symbolic values of alpha and beta.
Thus, redefining sum:

CL-USER> (defmacro sum (a b)
           (+ (symbol-value a) (symbol-value b)))
WARNING: redefining COMMON-LISP-USER::SUM in DEFMACRO
SUM

Here, the macro sum will evaluate the value of the symbols provided to it. It can only do it, if it is in the scope of the symbols. So, in order to do that, we can make alpha and beta dynamically bound.
Furthermore, in order to check if the dynamic binding is working, we can make a function dynamic-checker, which is defined below:

CL-USER> (defun dynamic-checker ()
           (+ alpha beta))

; in: DEFUN DYNAMIC-CHECKER
;     (+ ALPHA BETA)
; 
; caught WARNING:
;   undefined variable: ALPHA
; 
; caught WARNING:
;   undefined variable: BETA
; 
; compilation unit finished
;   Undefined variables:
;     ALPHA BETA
;   caught 2 WARNING conditions
DYNAMIC-CHECKER

And, finally we can evaluate this code in the REPL:

CL-USER> (let ((alpha 3) (beta -1))
           (declare (special alpha))
           (declare (special beta))
           (print (dynamic-checker))
           (sum alpha beta))

Which gives us the error:

; in: LET ((ALPHA 3) (BETA -1))
;     (SUM ALPHA BETA)
; 
; caught ERROR:
;   during macroexpansion of (SUM ALPHA BETA). Use *BREAK-ON-SIGNALS* to intercept.
;   
;    The variable ALPHA is unbound.
; 
; compilation unit finished
;   caught 1 ERROR condition

2 ; Evaluation aborted on #<SB-INT:COMPILED-PROGRAM-ERROR {1003F23AD3}>.
CL-USER>

Note the 2 in the end of the error code. This is returned by the function dynamic-checker, which adds alpha and beta, even though they aren't it's parameters, which proves that the variables alpha and beta can be accessed dynamically by the members of let.
Therefore, the macro sum should've worked now, because both the problems that occurred earlier are resolved.
But this clearly isn't the case, and I'm missing something.
Any help appreciated.

Solution

Interpreter and Compiler vs. interactivity

Common Lisp allows both interpreters and compilers. That you can interactively use a read-eval-print-loop does not mean that the implementation uses an interpreter. It could just incrementally compile the code and then invoke the compiled code. A Lisp interpreter runs the code from the Lisp representation. SBCL doesn't use an interpreter by default. It uses a compiler.

Using an Interpreter

LispWorks has an interpreter. Let's use it:

CL-USER 8 > (defun test ()
              (let ((alpha 3) (beta -1))
                (declare (special alpha))
                (declare (special beta))
                (print (dynamic-checker))
                (sum alpha beta)))
TEST

CL-USER 9 > (test)

2 
2

Thus the code works, since the Lisp interpreter executes the forms and when it sees a macro, then it expands it on the go. The bindings are available.

Let's use the LispWorks stepper, which uses the interpreter. :s is the step command.

(step (test))

(TEST) -> :s
   (LET ((ALPHA 3) (BETA -1))
     (DECLARE (SPECIAL ALPHA))
     (DECLARE (SPECIAL BETA))
     (PRINT (DYNAMIC-CHECKER))
     (SUM ALPHA BETA)) -> :s
      3 -> :s
      3 
      -1 -> :s
      -1 
      (PRINT (DYNAMIC-CHECKER)) -> :s
         (DYNAMIC-CHECKER) -> :s
            (+ ALPHA BETA) -> :s
               ALPHA -> :s
               3 
               BETA -> :s
               -1 
            2 
         2 
2                                ; <- output
      2 
      (SUM ALPHA BETA) <=> 2     ; <- macro expansion to 2
      2 -> :s                    
      2                          ; 2 evaluates to itself
   2 
2 
2

Compilation fails

But we can't compile your code:

CL-USER 10 > (compile 'test)

Error: The variable ALPHA is unbound.
  1 (continue) Try evaluating ALPHA again.
  2 Return the value of :ALPHA instead.
  3 Specify a value to use this time instead of evaluating ALPHA.
  4 Specify a value to set ALPHA to.
  5 (abort) Return to level 0.
  6 Return to top loop level 0.

Type :b for backtrace or :c <option number> to proceed.
Type :bug-form "<subject>" for a bug report template or :? for other options.

The compiler tries to expand the macro, but it does not run the code. Since the LET form isn't executed (the compiler only compiles it to something else, but does not execute it) there is no binding for alpha.

Style

Generally it is a good idea to avoid writing macros which need runtime state from the enclosing code. Such macros would only be executable by an interpreter.