I'm missing an obvious part of ASDF / quickload / sly, and it's starting to drive me nuts.
It's mostly about the developer experience, but as I can't go past the very first steps, I'm unable to work on anything more than short examples in a short-lived repl.
So, don't hesitate to assume that I'm misunderstanding something very basic.
Also, I've spend a couple of days trying to wrap my head around this, so at this point I've probably been looking "past" the solutions a couple of time (reading exactly the right sentences of the right manual several times, but not managing to make my brains register.) So, really, have mercy on the noob.
Following the "Working with projects" section of the Common Lisp cookbook, I was able to create an asdf project.
https://lispcookbook.github.io/cl-cookbook/getting-started.html#working-with-projects
The project has three source files:
The asd file contains two defsystem clauses.
In particular, one of the clauses defines a dependency on the "rove" library for test runner.
(defsystem "foobar/tests"
:author ""
:license ""
:depends-on ("foobar"
"rove")
:components ((:module "tests"
:components
((:file "main"))))
:description "Test system for foobar"
:perform (test-op (op c) (symbol-call :rove :run c)))
There is one source file for tests:
(defpackage foobar/tests/main
(:use :cl
:foobar
:rove))
(in-package :foobar/tests/main)
;; NOTE: To run this test file, execute `(asdf:test-system :foobar)' in your Lisp.
(deftest test-target-1
(testing "should (= 1 1) to be true"
(ok (= 1 1))))
If I open the foobar/test/main.lisp file, and I want to run the tests, then as I understand it :
One way to do that is:
(ql:quickload "rove") in the REPLSo it seems, like, if my project depends on 10 libraries, it would mean that I would start every work session by quickload-ing all 10 libraries, then asdf-loading things for 5 minutes (including multiple typos in the reply.)
Is that correct ? (Or is there a proper sequence of repl / slime / sly commands that will trigger all the quicklisp and ASDF functions ?
Is that how people work ? Open up emacs, go through all their files, and quickload stuff as they go, ASDF load system as they go, etc... If I put the content of those files on a repo, and I tell you to extend the code, what do you do past 'git clone' ?
So it seems, like, if my project depends on 10 libraries, it would mean that I would start every work session by quickload-ing all 10 libraries, then asdf-loading things for 5 minutes
Normally you should only have to quickload your project, and all the transitive dependencies will be loaded.
There is also the convention of having a separate test system that exists because supposedly during tests you may need more dependencies that the ones for the library you publish. Knowing that, if you want to run all tests, you need only call:
(ql:quickload "foobar/tests")
And since system "foobar" is a dependency, it will also be loaded automatically.
Once it is loaded, you can execute various operations on systems, like asdf:test-system.
Is that correct ? (Or is there a proper sequence of repl / slime / sly commands that will trigger all the quicklisp and ASDF functions ?
There might be Slime shortcuts but I don't use them so I can't say, but you only need to quickload the root element of your dependency tree.
Is that how people work ? Open up emacs, go through all their files, and quickload stuff as they go, ASDF load system as they go, etc... If I put the content of those files on a repo, and I tell you to extend the code, what do you do past 'git clone' ?
So I went to the Awesome CL Software page and picked something relatively small (large projects tend to have more setup requirements, hopefully they document it). I cloned ALIW (A Lisp in Wonderland), some Wiki software which I don't know. It seems to be a bit old and not maintained.
I go to the local-projects directory for quicklisp, which will simplify the discovery of the system. I could have used a different approach but this is the simplest:
$ cd $HOME/quicklisp/local-projects
There, I clone the project:
$ git clone https://github.com/vy/aliw
I see that there is an aliw.asd file which defines a system named "aliw", with a lot of dependencies. I happen to have a Lisp instance already running, and from the Slime REPL, I write the following:
CL-USER> (ql:quickload "aliw")
Then, a lot of text is emitted showing that quicklisp is fetching, compiling and loading code (all the dependencies), until I reach my first error:
; caught ERROR:
; READ error during COMPILE-FILE:
;
; Package HUNCHENTOOT-MP does not exist.
;
; Line: 74, Column: 27, File-Position: 3006
;
; Stream: #<SB-INT:FORM-TRACKING-STREAM for "file /home/christophe/quicklisp/local-projects/aliw/src/specials.lisp" {1005C5C7E3}>
There are also available restarts:
COMPILE-FILE-ERROR while
compiling #<CL-SOURCE-FILE "aliw" "src" "specials">
[Condition of type UIOP/LISP-BUILD:COMPILE-FILE-ERROR]
Restarts:
0: [RETRY] Retry compiling #<CL-SOURCE-FILE "aliw" "src" "specials">.
1: [ACCEPT] Continue, treating compiling #<CL-SOURCE-FILE "aliw" "src" "specials"> as having been successful.
....
I do not select any restart option, instead I'll fix the code and resume loading later.
The repository was archived some time ago was not kept up-to-date, there is effectively no hunchentoot-mp package. Seeing that function being called was make-lock, which is used to make a mutex, I fix this by editing the file and writing instead: bt:make-lock, referencing the bordeaux-threads library, which is well-known and provides locks. I also edit the .asd file to add bordeaux-threads library.
(asdf:defsystem :aliw
:version +aliw-version+
:depends-on (:bordeaux-threads ;; << here
:cl-difflib
:cl-fad
:cl-ppcre
:cl-who
:flexi-streams
:hunchentoot
:md5
...
Note that Hunchentoot already depends on bordeaux-threads but it is better to document the dependencies that you really need in the project, not rely on hidden transitive dependencies.
Since it is unlikely to be the only error, I also fix all other occurences of make-lock and use bt:with-recursive-lock-held wherever necessary. In the buffers that I visit in Emacs, slime-mode allows me to autocomplete things that are already loaded in the system at this point.
Now, I can resume loading the system, and I do invoke the Retry ASDF operation restart because I edited the .asd file.
SBCL warns me about a symbol being exported, due to this pattern:
(defpackage :aliw-asd
(:use :cl :asdf))
(in-package :aliw-asd)
(defparameter +aliw-version+ "0.1.5")
(export '+aliw-version+)
When you load the file twice, the defpackage notices that +aliw-version+ is exported, but this is not declared by the package declaration. A better way to do this is to write:
(defpackage :aliw-asd
(:use :cl :asdf)
(:export #:+aliw-version+))
(in-package :aliw-asd)
(defparameter +aliw-version+ "0.1.5")
(asdf:defsystem :aliw ...)
I restart again after making this change, and fix the remaining errors about hunchentoot:start-server (which has been renamed start for a long time). I do the same for stop-server.
There is a warning about hunchentoot:*handle-http-errors-p* being undefined, so now I have another Yak to shave, I clone Hunchentoot and see that it was removed in 2011. I remove the code in "aliw" that uses it and make a note to send a patch to Hunchentoot.
Now, everything is loaded. Since I hacked a bit the code while loading it, I want to make sure that everything works. If I open a new terminal and start sbcl, I can quickload "aliw" almost instantly (compilation produces object files that can be loaded), without error, so my fixes seem to be good (nb. after having looked at it more, the code compiles without problem, but I would have to make more changes to have something that runs).
There was a lot of stuff to do, but notice that I was able to fix the code while I was in the process of loading it, and I only had to invoke restarts to continue compiling the code as necessary. Usually with libraries that are distributed by Quicklisp this works out of the box because the maintainer tests the distribution.
I hope this makes sense.