async clojure http-kit client with callback writing to database sqlite3

Question

I am stucked with http-kit async behavior when it comes to write to an sqlite table.

The i/o to the database depends on whether I send the code to a boot repl or run it as a boot script. The i/o proceed only in the repl case. What am I missing? Here 's my code:

#!/usr/bin/env boot                                                                                                                                             

(defn deps [new-deps]                                                                                                                                           
  (boot.core/merge-env! :dependencies new-deps))                                                                                                                

(deps '[                                                                                                                                                        
        [http-kit  "2.2.0"]                                                                                                         
        [org.clojure/core.async  "0.2.395"]                                                                                                                     
        [org.clojure/java.jdbc  "0.7.0-alpha1"]                                                                                                                 
        [org.xerial/sqlite-jdbc  "3.16.1"]                                                                                                                      
        [org.slf4j/slf4j-nop "1.7.22"]                                                                                                                          
        ])                                                                                                                                                      

(require                                                                                                                                                        
         '[org.httpkit.client :as http]                                                                                                                         
         '[clojure.java.jdbc :as jdbc]                                                                                                                          
         )                                                                                                                                                      


(def db-spec                                                                                                                                                    
  {:classname "org.sqlite.JDBC"                                                                                                                                 
   :subprotocol "sqlite"                                                                                                                                        
   :subname "sqlite.db"})                                                                                                                                       

;(jdbc/db-do-commands                                                                                                                                           
  ;db-spec                                                                                                                                                      
  ;(jdbc/create-table-ddl "test" [[:msg :text]]))                                                                                                               

(def ins! (partial jdbc/insert! db-spec "test"))                                                                                                                

(http/get "http://locahost" {} (fn [_] (ins! {:msg "repl"})))                                                                                                   


(defn -main []                                                                                                                                                  
  (println (System/getProperty  "user.dir"))                                                                                                                    
  (http/get "http://locahost" {} (fn [_] (ins! {:msg "exec"}))))                                                                                                

Thanks

Answer 1

The issue with the script not working from command line is that http-kit asynchronous callbacks are handled by daemon threads and the only non-daemon thread is the main thread running your script.

When your -main function ends after submitting HTTP request to http-kit for async processing, the main thread terminates and causes that the JVM shutdowns before daemon threads processing your async callback get a chance to run.

You can check that by adding a sleep expression in the end of -main function to see that your callback is executed:

(defn -main []                                                                                                                                                  
  (println (System/getProperty  "user.dir"))                                                                                                                    
  (http/get "http://locahost" {} (fn [_] (ins! {:msg "exec"})))
  (Thread/sleep 60000))

The best way to make sure that the -main function will wait for the result to be processed is to keep the promise returned by http/get call. The promise eventually will contain the result produced by your callback function:

(let [result-promise (http/get "https://www.google.com" {} (fn [_] "Result"))]
  @result-promise)

@result-promise is a reader macro/shortcut for (deref result-promise).

The full form might be better when you would like to not block indefinitely - just call deref with timeout-ms and timeout-value arguments:

(deref result-promise 5000 "Didn't get response in 5 seconds. Giving up")