I'm trying to optimize a function by changing an argument from type string to simple-base-string (among other things). But this produces a compiler error:
(declaim (ftype (function (simple-base-string) (or nil t)) dictionary-compatible))
(defun dictionary-compatible ($new-cross-str) ;eg, "A?R??" of length 5
"Tests if a string (with uppercase alphabetic and ? characters)
is compatible with the dictionary words (symbols) in a hash-table."
(declare (optimize (speed 3)))
(iter (with len = (length $new-cross-str))
(declare (fixnum len))
(for chr in-vector $new-cross-str)
(declare (base-char chr))
(for index from 0)
(declare (fixnum index))
(when (char/= chr #\?)
(collect (gethash (format nil "~D~C~D" len chr index)
*lci-dictionary*) into dict-words))
(finally (return (reduce #'intersection dict-words)))))
; in: DEFUN DICTIONARY-COMPATIBLE
; (ITERATE:ITER
; (ITERATE:WITH WOULDWORK-PKG::LEN = (LENGTH WOULDWORK-PKG::$NEW-CROSS-STR))
; (DECLARE (FIXNUM WOULDWORK-PKG::LEN))
; (ITERATE:FOR WOULDWORK-PKG::CHR WOULDWORK-PKG::IN-VECTOR
; WOULDWORK-PKG::$NEW-CROSS-STR)
; (DECLARE (BASE-CHAR WOULDWORK-PKG::CHR))
; (ITERATE:FOR WOULDWORK-PKG::INDEX WOULDWORK-PKG::FROM 0)
; (DECLARE (FIXNUM WOULDWORK-PKG::INDEX))
; (WHEN (CHAR/= WOULDWORK-PKG::CHR #\?)
; (ITERATE:COLLECT
; (GETHASH
; (FORMAT NIL "~D~C~D" WOULDWORK-PKG::LEN WOULDWORK-PKG::CHR
; WOULDWORK-PKG::INDEX)
; WOULDWORK-PKG::*LCI-DICTIONARY*)
; WOULDWORK-PKG::INTO WOULDWORK-PKG::DICT-WORDS))
; (ITERATE:FINALLY
; (RETURN (REDUCE #'INTERSECTION WOULDWORK-PKG::DICT-WORDS))))
; --> BLOCK BLOCK TAGBODY PROGN IF PROGN
; ==>
; WOULDWORK-PKG::DICT-WORDS
;
; note: deleting unreachable code
;
; caught WARNING:
; Constant NIL conflicts with its asserted type BASE-CHAR.
; See also:
; The SBCL Manual, Node "Handling of Types"
Does the macroexpansion of the iter statement offer any clues to the problem(s)?
* (macroexpand-1 '(iter (with len = (length $new-cross-str))
(declare (fixnum len))
(for chr in-vector $new-cross-str)
(declare (base-char chr))
(for index from 0)
(declare (fixnum index))
(when (char/= chr #\?)
(collect (gethash (format nil "~D~C~D" len chr index)
*lci-dictionary*) into dict-words))
(finally (return (reduce #'intersection dict-words)))))
(LET* ((LEN (THE FIXNUM (LENGTH $NEW-CROSS-STR)))
(#:SEQUENCE414 NIL)
(#:LIMIT415 NIL)
(CHR NIL)
(#:INDEX413 NIL)
(INDEX 0)
(DICT-WORDS NIL)
(#:END-POINTER416 NIL)
(#:TEMP417 NIL))
(DECLARE (FIXNUM LEN))
(DECLARE (BASE-CHAR CHR))
(DECLARE (FIXNUM INDEX))
(BLOCK NIL
(BLOCK #:ITERATE142
(TAGBODY
(PROGN (SETQ #:SEQUENCE414 $NEW-CROSS-STR) (SETQ #:LIMIT415 (LENGTH #:SEQUENCE414)) (SETQ #:INDEX413 -1) (SETQ INDEX -1))
LOOP-TOP-NIL
(PROGN
(SETQ #:INDEX413 (+ #:INDEX413 1))
(IF (>= #:INDEX413 #:LIMIT415)
(GO LOOP-END-NIL))
(SETQ CHR (AREF #:SEQUENCE414 #:INDEX413))
(SETQ INDEX (+ INDEX 1))
(IF (CHAR/= CHR #\?)
(PROGN
(SETQ #:TEMP417 (LIST (GETHASH (FORMAT NIL "~D~C~D" LEN CHR INDEX) *LCI-DICTIONARY*)))
(SETQ #:END-POINTER416
(IF DICT-WORDS
(SETF (CDR #:END-POINTER416) #:TEMP417)
(SETQ DICT-WORDS #:TEMP417)))
DICT-WORDS)))
(PROGN)
(GO LOOP-TOP-NIL)
LOOP-END-NIL
(PROGN (RETURN (REDUCE #'INTERSECTION DICT-WORDS))))
NIL)))
T
This is a bug in iterate which is fairly common (series had it, long ago, as well). The problem is that it binds your chr variable too early, and in particular before it has worked out that it can bind it to the first element of the vector (because the vector might not have any elements). So it needs to find some default value ... and picks nil, which is not correct.
This is sometimes hard to avoid: in the series case I think there were clearly cases where it was difficult to deal with. This is not one of those.
A horrid workaround is to declare types as (or null <type-you-want>). loop also gets this right in SBCL (by, if it has to, turning the type declaration in something like (loop ... for x of-type fosh across ... ...) into (or null fosh)). This is better than not working at all, but it does mean that the type declaration for the variable is not what you expect it to be by looking at the code, so in particular something like (loop for x of-type fosh across ... do (setq x nil) ...) will not be picked up as an error.
As a note: this is why I avoid complicated iteration constructs: they almost always have obscure bugs like this. loop probably has had most of them sorted out by now if you can bear it (I can't).