Elisp: sleep-for doesn't block when running a test in ert

Question

I'm trying to set up some tests using ert that need to sleep for a background process to proceed. I've tried using sleep-for and accept-process-output. Neither is reliable. Here is a small example.

This test just sleeps for 5 seconds and then checks that at least 3 seconds have passed. Using sleep-for it finishes immediately and fails. If the shell-command is uncommented is takes the expected 5 seconds and succeeds! What is going on here?

(ert-deftest timetest ()
  (let ((now (cadr (current-time))))
    ;(shell-command "sleep 5")
    (sleep-for 5)
    (should (< now (- (cadr (current-time)) 3)))))

EDIT:

There must have been something strange in my environment when I tested the previous example. This slightly changed example contains a background process like the one I need to test, and fails. I tested it both interactively and using the command:

emacs --batch -l example.el -f ert-run-tests-batch-and-exit

(ert-deftest timetest ()
  (let ((now (cadr (current-time))))
    (start-process "echo" "*echo*" "echo" "hello world")
    (sleep-for 5)
    (should (< now (- (cadr (current-time)) 3)))))

Output is:

Test timetest condition:
    (ert-test-failed
     ((should
       (< now
          (- ... 3)))
      :form
      (< 55177 55174)
      :value nil))
   FAILED  1/1  timetest

Ran 1 tests, 0 results as expected, 1 unexpected (2013-02-05 09:57:29+0000)

1 unexpected results:
   FAILED  timetest

EDIT2:

New version that seems to indicate process output is sufficient to interrupt sleep-for:

(ert-deftest timetest ()
  (let ((now (cadr (current-time))))
    (start-process "yes" "*yes*" "yes")
    (sleep-for 1) ;; This sleep-for may be interrupted by process *output*
    (sleep-for 5) ;; This sleep-for is also interrupted
    (should (< now (- (cadr (current-time)) 3)))))

EDIT3:

With a heavy heart, I post another version:

(ert-deftest timetest ()
  (let ((now (cadr (current-time)))
        (process-connection-type nil))
    (start-process "tmp" "*tmp*" "bash" "-c" "sleep 1; echo hi")
    (sleep-for 5)
    (should (< now (- (cadr (current-time)) 3)))))

It seems clear that sleep-for cannot be relied upon to block.

Answer 1

I think that sleep-for is interrupted when process exits.

Following test is failed, because sleep-for is interrupted after 2 second.

(ert-deftest timetest ()
  (let ((now (cadr (current-time))))
    (start-process "sleep" "*sleep*" "sleep" "2") ;; Fail
    (sleep-for 5)
    (should (< now (- (cadr (current-time)) 3)))))

But following test is passed, because sleep-for is interrupted after 4 second.

(ert-deftest timetest ()
  (let ((now (cadr (current-time))))
    (start-process "sleep" "*sleep*" "sleep" "4") ;; Success
    (sleep-for 5)
    (should (< now (- (cadr (current-time)) 3)))))

So I think you can write test as below

(ert-deftest timetest ()
  (let ((now (cadr (current-time))))
    (start-process "echo" "*echo*" "echo" "hello world")
    (sleep-for 1) ;; This sleep-for may be interrupted by process exited
    (sleep-for 5) ;; This sleep-for can sleep 5 seconds
    (should (< now (- (cadr (current-time)) 3)))))

EDIT1

2nd test is passed by setting process-connection-type to nil. (But I cannot understand this behavior)

(ert-deftest timetest ()
  (let ((now (cadr (current-time)))
        (process-connection-type nil))
    (start-process "yes" "*yes*" "yes")
    (sleep-for 5)
    (should (< now (- (cadr (current-time)) 3)))))

Please see document

EDIT2

In 3rd test, we need 3 sleep-for because sleep-for is interrupted 2 times, But this is fragile. So I write test as below. This test can be passed if sleep-for is interrupted as many times. And I use float-time instead of current-time because 2nd element of current-time may wrap arround.

(ert-deftest timetest ()
  (let ((now (float-time))
        (process-connection-type nil))
    (start-process "tmp" "*tmp*" "bash" "-c" "sleep 1; echo hi")
    (while (< (- (float-time) now) 5)
      (sleep-for 1))
    (should (< now (- (float-time) 3)))))