I need to do a big trick and am keen on hearing your suggestions. What I need is a macro that takes ordinary clojure code peppered with a special "await" form. The await forms contains only clojure code and are supposed to return the code's return value. Now, what I want is that when I run whatever is being produced by this macro, it should stop executing when the first "await" form is due for evaluation.
Then, it should dump all the variables defined in its scope so far to the database (I will ignore the problem that not all Clojure types can be serialised to EDN, e.g. functions can't), together with some marker of the place it has stopped in.
Then, if I want to run this code again (possibly on a different machine, another day) - it will read its state from the DB and continue where it stopped.
Therefore I could have, for example:
(defexecutor my-executor
(let [x 7
y (await (+ 3 x))]
(if (await (> y x))
"yes"
"no")))
Now, when I do:
(my-executor db-conn "unique-job-id")
the first time I should get a special return value, something like
:deferred
The second time it should be like this as well, only the third time a real return value should be returned.
The question I have is not how to write such executor, but rather how to gather information from within the macro about all the declared variables to be able to store them. Later I also want to re-establish them when I continue execution. The await forms can be nested, of course :)
I had a peek into core.async source code because it is doing a similar thing inside, but what I have found there made me shiver - it seems they employ the Clojure AST analyser to get this info. Is this really so complex? I know of &env variable inside a macro, but do not have any idea how to use it in this situation. Any help would be appreciated.
And one more thing. Please do not ask me why I need this or that there is a different way of solving a problem - I want this specific solution.
I will ignore the problem that not all Clojure types can be serialised to EDN, e.g. functions can't
If you ignore this, it will be very restrictive for the kinds of Clojure expressions you can handle. Functions are everywhere, e.g. in the implementation of things like doseq and for. Likewise, a lot of interesting programs will depend on some Java object like a file handle or whatever.
The question I have is not how to write such executor, but rather how to gather information from within the macro about all the declared variables to be able to store them.
If you manage to write such an executor, I suspect its implementation will need to know about local variables anyway. So you can put off this question until you are done implementing your executor - you will probably find it obsolete, if you can implement your executor.
I had a peek into core.async source code because it is doing a similar thing inside, but what I have found there made me shiver - it seems they employ the Clojure AST analyser to get this info. Is this really so complex?
Yes, this is very intrusive. You are basically writing a compiler. Thank your lucky stars they wrote the analyzer for you already, instead of having to analyze expressions yourself.
I know of &env variable inside a macro, but do not have any idea how to use it in this situation.
This is the easy part. If you like, you can write a simple macro that gives you all the locals in scope. This question has been asked and answered before, e.g. in Clojure get local lets.
And one more thing. Please do not ask me why I need this or that there is a different way of solving a problem - I want this specific solution.
This is generally an unproductive attitude when asking a question. It's admitting you're posing an XY problem, and still refusing to tell anyone what the Y is.