How to make defsystem use "everything"?

I am working on project euler problems in SBCL and keep a short file for every solution. Every problem has some 5am-based tests, which are referenced from a "main" test suite. These tests are run when "tests.lisp" is run. Since I got bored with maintaining the list of files by hand, I wrote some code to do it for me:

(defpackage #:euler/asdf
  (:use :cl :asdf))
(in-package #:euler/asdf)

;; directory containing the problem files
(defparameter +dir+ "/home/stefan/quicklisp/local-projects/euler")

;; build file list for package components
(defun files-for-problems (dir)
  (mapcar #'(lambda (p) (list :file (pathname-name p) :depends-on '("package")))
      (directory (concatenate 'string dir "/e????.lisp"))))

;; build dependency list for all tests component
(defun depends-on-problems (dir)
  (mapcar #'pathname-name
      (directory (concatenate 'string dir "/e????.lisp"))))

;; define euler system
(defsystem euler
    :name "euler"
    :author "Stefan Schmiedl"
    :description "Solutions to problems at http://projecteuler.net"
    :depends-on ("iterate" "fiveam" "cl-csv")
    :components #.`((:file "package")
            ,@(files-for-problems +dir+)
         #.`(:file "tests" :depends-on ,(depends-on-problems +dir+))))

In short, defsystem euler uses all e????.lisp files as components and tests.lisp depends on all of these files.

Is this a good idea? Is there an "official" way to make defsystem use all files in a directory or all files matching a given filename pattern?

I feel like I'm missing something elementary here, especially after reading some ELS slides on github about a "more declarative defsystem" where the thing I've done above would probably be frowned upon.

After some fiddling with Fare's suggestion here is what I now have:

;; define private package for defsystem
(defpackage #:euler-system
  (:use :cl :uiop :asdf))
(in-package #:euler-system)


;; define euler system
(defsystem "euler"
  :author "Stefan Schmiedl"
  :description "Solutions to problems at http://projecteuler.net"
  :depends-on ("iterate" "fiveam" "cl-csv")
  :components ((:module "package"
                        :pathname ""
                        :components ((:file "package")))
               (:module "problems"
                        :pathname ""
                        :depends-on ("package")
                        :components #.(mapcar #'(lambda (p) (list :file (pathname-name p)))
                                              (directory-files (pathname-directory-pathname
                                                                (uiop/lisp-build:current-lisp-file-pathname))
                                                               "e*.lisp")))
               (:module "tests"
                        :pathname ""
                        :depends-on ("package" "problems")
                        :components ((:file "tests")))))

Thanks for the feedback.

Solution

For the directory part, I recommend using relative pathnames. You could do it several ways.

1- Thou shalt not use an absolute pathname. Use relative pathname like that, possibly via a variable: (subpathname (current-file-pathname) #p"e????.lisp")

2- I'm not sure how portable ? is as a wildcard character — if you can live with it, * is much more portable.

3- uiop:directory-files is safer than cl:directory in this and many contexts.

4- for an "official" way of handling wildcard patterns without #. or (eval `...), get inspiration from asdf/contrib/wild-modules.lisp — that said, for a one-off, #. is totally acceptable, especially since we're so far from purely declarative .asd files.

5- for grouped dependencies, you could use

(defsystem "euler"
  :depends-on ("iterate" "fiveam" "cl-csv")
  :serial t
  :components
   ((:module "package" :pathname ""
       :components ((:file "package")))
    (:module "problems" :pathname "" :depends-on ("package")
       :components #.(mapcar ...))
    (:module "tests" :pathname ""
       :components ((:file "tests")))))

6- Instead of modules, you could be using secondary systems, at which point system-relative-pathname would be available:

(defsystem "euler" :depends-on ("euler/tests"))
(defsystem "euler/tests"
  :depends-on ("euler/package")
  :components ((:file "package")))
(defsystem "euler/problems"
  :depends-on ("euler/package")
  :components
    #.(mapcar ... (directory-files (system-relative-pathname "euler" #p"e*.lisp")))))
(defsystem "euler/tests"
  :depends-on ("euler/problems")
  :components ((:file "tests")))

7- In the above I assume asdf3, and that you use uiop without prefix:

(defpackage :euler-system (:use :cl :uiop :asdf))
(in-package :euler-system)

If you don't define any function or variable or class, you could directly (in-package :asdf)

I'm glad you enjoyed my talk at ELS 2013. I gave another one at ELS 2014, in the same repository.