Weird Common lisp SBCL query-io behavior

Currently learning Common Lisp with SBCL, and it's bugging me how query-io has some unexpected differences from standard input and output. I will assume that standard-output contains a newline, or is formatted with one, because

(format *query-io* "hello")

Contains no newline at all.

Now, using read-line gives me this behavior

(defun read (prompt)
  (format *query-io* "~a: " prompt)
  (read-line))

(read "message")
> hello
"hello"
message: NIL

I was told this is expected, as SBCL would wait for a newline to be inputted before printing the format, but what I don't get is the result of changing the second line

(defun read (prompt)
  (format *standard-output* "~a: " prompt)
  (read-line))

(read "message")
> hello
message: 
"hello"
NIL

Why is the order different? I don't think the newline after the prompt variable affects the result

Solution

If you want things to behave in a predictable way you should write programs which ensure that:

don't name the function read (in fact I'm extremely convinced that SBCL wouldn't have allowed you to name the function read, unless perhaps you are using a prehistoric version);
be clear about what streams you are actually reading and writing to;
if you want output to appear at a given time, say you do.

So in your first function you are doing something equivalent to what I assume is this:

(defun f1 (prompt)
  (format *query-io* "~a: " prompt)
  (read-line))

This:

writes the prompt to *query-io*, which means that at some future time it may appear on whatever *query-io* points at;
reads a line from *standard-input*;
returns the result of reading that line.

So if you call this function it will read a line from *standard-input* which perhaps is the terminal but may not be. At some time after the call to format some output probably will appear on *query-io* which may also point at the terminal. There is no guarantee as to when or in fact even if that output will appear: for instance if you modify the function to be:

(defun f1.1 (prompt)
  (format *query-io* "~a: " prompt)
  (clear-output *query-io*)
  (read-line))

it is quite possible that the prompt will never appear.

When output actually appears from functions which write to streams if you don't explicitly say 'write it now' depends on many, many factors:

what the stream points to;
if there's a buffer;
how big the buffer is;
how full the buffer is;
whether you later called clear-output on the stream and whether clear-output actually does anything;
whether you later closed the stream with :abort t and whether that actually does anything;
the price of chocolate on the moons of Jupiter.

In your second function things are mostly the same:

(defun f2 (prompt)
  (format *standard-output* "~a: " prompt)
  (read-line))

This time format is writing to *standard-output*, not *query-io* and again you are reading a line from *standard-input*. And again, when and if the output from format appears is in the lap of the gods.

So what you need to do is

be clear about what streams you are using;
actually ask for output to appear when you want it, not later.

In particular never make assumptions that output written to some stream appears at any given point 'on its own' or 'after a newline', or anything like that.

If you want to prompt and read from *query-io* you do it like this:

(defun f3 (prompt)
  (format *query-io* "~a: " prompt)
  (finish-output *query-io*)
  (read-line *query-io*))

If you want to prompt on *standard-output* and read from *standard-input* then:

(defun f4 (prompt)
  (format *standard-output* "~a: " prompt)
  (finish-output *standard-output*)
  (read-line *standard-input*))

In this last case you can actually leave out the streams (and use t for format) because they are the defaults: still it is probably better to be explicit.

f3 is probably better, since prompting and reading is what *query-io* is for.

Weird Common lisp SBCL *query-io* behavior

Weird Common lisp SBCL query-io behavior