have a file of s-expressions, which includes foreign language characters, that I am reading in as follows:
(defun test (file)
(with-open-file (stream file)
(loop while (read stream nil nil))))
It reads the file without errors in ccl 1.8, but throws an error under 1.9:
? (test "/users/markklein/desktop/naples.text")
> Error: Reader error: Illegal symbol syntax.
> While executing: CCL::%PARSE-TOKEN, in process Listener(5).
> Type cmd-. to abort, cmd-\ for a list of available restarts.
> Type :? for other options.
1 >
Does anyone have any ideas what is going wrong, and how to fix it? I can send the data file on request.
Ben Hyde noted in a comment that R. Matthew Emerson spoke to this issue on the Clozure mailing list, pointing out that Clozure CL's default external format has been changed to :utf-8. As a result, he suggested this alternative:
We changed the default external format to :utf-8 in the 1.9 release. This is probably tripping you up. Try specifying an appropriate external-format explicitly, e.g.,
(defun test (file) (with-open-file (stream file :external-format :iso-8859-1) (loop while (read stream nil nil))))