I'm trying to fully understand the limitations of compile-time macros.
Here is a macro (I'm fully aware that this is not a best-practice macro):
(defmacro emit (language file &body body)
(print language)
(print file)
(print body)
(with-open-file (str file :direction :output :if-exists :supersede)
(princ (cond ((eq language 'html)
(cl-who:with-html-output-to-string (s nil :prologue t :indent t) body))
((eq language 'javascript)
(parenscript:ps body))
((eq language 'json)
(remove #\; (parenscript:ps body))))
str)))
I compile the macro:
; processing (DEFMACRO EMIT ...)
PROGRAM>
I compile this form:
PROGRAM> (compile nil (lambda () (emit json "~/file" (ps:create "hi" "hello") (ps:create "yo" "howdy"))))
JSON
"~/file"
((PARENSCRIPT:CREATE "hi" "hello") (PARENSCRIPT:CREATE "yo" "howdy"))
#<FUNCTION (LAMBDA ()) {5367482B}>
NIL
NIL
PROGRAM>
The compile-time print output is what I expect.
However, if I look at ~/file:
body
It appears that ((PARENSCRIPT:CREATE "hi" "hello") (PARENSCRIPT:CREATE "yo" "howdy")) was never substituted in for the parameter body, and thus never processed.
Why is this?
& what would be the best literature to read on this subject?
Why should it substitute? You never substituted anything.
A macro defines a macro substitution function, which is applied to the actual form in the code to produce another form which is then compiled. When you apply your macro definition to those parameters, it will at macroexpansion time do all kinds of things (write a file etc.) before returning what princ returned, which is exactly its first argument, and this returned form is then compiled. I don't think that is what you want.
It seems that what you actually want to do is to expand to a form that interprets the body in one of a variety of ways, indicated by the first argument.
What you need to do is to return the new form, so that
(emit 'html "foo.html"
(:html (:head) (:body "whatever")))
expands to
(with-open-file (str "foo.html" :direction :output :etc :etc)
(cl-who:with-html-output (str)
(:html (:head) (:body "whatever")))
For that, we have a template syntax: the backtick.
`(foo ,bar baz)
means the same as
(list 'foo bar 'baz)
but makes the structure of transformed code a bit clearer. There is also ,@ to splice things into a list.
`(foo ,@bar)
means the same as
(list* 'foo bar)
i. e. the contents of bar, when they are a list, are spliced into the list. This is especially useful for bodies such as in your macro.
(defmacro emit (language file &body body)
`(with-open-file (str ,file :direction :output :if-exists :supersede)
(princ (cond ((eq ,language 'html)
(cl-who:with-html-output-to-string (s nil :prologue t :indent t)
,@body))
((eq ,language 'javascript)
(parenscript:ps ,@body))
((eq ,language 'json)
(remove #\; (parenscript:ps ,@body))))
str)))
Note where I introduced the backtick to create a template and commata to put outer arguments into it. Note also that the arguments are forms.
This has a few problems: there are hardcoded symbols that the user of the macro has no way of knowing. In one case (str) they have to pay attention not to shadow it, in the other (s) they have to know it in order to write to it. For this, we use either generated symbols (for str so that there is no conflict possible) or let the user say what they want to name it (for s). Also, this cond can be simplified to a case:
(defmacro emit (language file var &body body)
(let ((str (gensym "str")))
`(with-open-file (,str ,file
:direction :output
:if-exists :supersede)
(princ (case ,language
('html
(cl-who:with-html-output-to-string (,var nil
:prologue t
:indent t)
,@body))
('javascript
(parenscript:ps ,@body))
('json
(remove #\; (parenscript:ps ,@body))))
,str)))
However, you might want to determine the output code already at macro expansion time.
(defmacro emit (language file var &body body)
(let ((str (gensym "str")))
`(with-open-file (,str ,file
:direction :output
:if-exists :supersede)
(princ ,(case language
('html
`(cl-who:with-html-output-to-string (,var nil
:prologue t
:indent t)
,@body))
('javascript
`(parenscript:ps ,@body))
('json
`(remove #\; (parenscript:ps ,@body))))
,str)))
Here, you can see that the case form is already evaluated at macro expansion time, and an inner template is then used to create the inner form.
This is all completely untested, so removing the little errors is left as an exercise ^^.
One book that has a lot of things to say about macro writing is »On Lisp« by Paul Graham. The freely available »Practical Common Lisp« by Peter Seibel also has a chapter about it, and there are also some recipes in »Common Lisp Recipes« by Edi Weitz.