I get a NullPointerException when trying to access LazySeq the first time, line-items - which takes some time to evaluate, then it breaks with NullPointerException on the next line calculating total-exc:
(defn orders->invoice [db billing-account-id dt orders resource-name]
(let [line-items (mapcat (partial order->line-items db dt resource-name) orders) ;;lazy
total-exc (reduce + (map :invoice-item/total-exclusive line-items)) ;; NullPointerException
total-inc (reduce + (map :invoice-item/total-inclusive line-items))]
{:invoice/items line-items
:invoice/total-exclusive total-exc
:invoice/total-inclusive total-inc}))
There isn't data missing as if I put line where is breaks in try catch block swollow it and try the same in the catch block it will work:
(try
(reduce + (map :invoice-item/total-exclusive line-items))
(catch Exception e
(clojure.pprint/pprint (.getStackTrace e))
(reduce + (map :invoice-item/total-exclusive line-items))))
Stacktrace:
[[java.util.concurrent.FutureTask report "FutureTask.java" 122],
[java.util.concurrent.FutureTask get "FutureTask.java" 192],
[clojure.core$deref_future invokeStatic "core.clj" 2300],
[clojure.core$future_call$reify__8439 deref "core.clj" 6974],
[clojure.core$deref invokeStatic "core.clj" 2320],
[clojure.core$deref invoke "core.clj" 2306],
[clojure.core$map$fn__5851 invoke "core.clj" 2755],
[clojure.lang.LazySeq sval "LazySeq.java" 42],
[clojure.lang.LazySeq seq "LazySeq.java" 58],
[clojure.lang.RT seq "RT.java" 531],
[clojure.core$seq__5387 invokeStatic "core.clj" 137],
[clojure.core$filter$fn__5878 invoke "core.clj" 2809],
[clojure.lang.LazySeq sval "LazySeq.java" 42],
[clojure.lang.LazySeq seq "LazySeq.java" 51],
[clojure.lang.RT seq "RT.java" 531],
[clojure.core$seq__5387 invokeStatic "core.clj" 137],
[clojure.core$map$fn__5851 invoke "core.clj" 2746],
[clojure.lang.LazySeq sval "LazySeq.java" 42],
[clojure.lang.LazySeq seq "LazySeq.java" 51],
[clojure.lang.RT seq "RT.java" 531],
[clojure.core$seq__5387 invokeStatic "core.clj" 137],
[clojure.core$concat$cat__5480$fn__5481 invoke "core.clj" 734],
[clojure.lang.LazySeq sval "LazySeq.java" 42],
[clojure.lang.LazySeq seq "LazySeq.java" 58],
[clojure.lang.RT seq "RT.java" 531],
[clojure.core$seq__5387 invokeStatic "core.clj" 137],
[clojure.core$map$fn__5851 invoke "core.clj" 2746],
[clojure.lang.LazySeq sval "LazySeq.java" 42],
[clojure.lang.LazySeq seq "LazySeq.java" 51],
[clojure.lang.Cons next "Cons.java" 39],
[clojure.lang.RT next "RT.java" 709],
[clojure.core$next__5371 invokeStatic "core.clj" 64],
[clojure.core.protocols$fn__8144 invokeStatic "protocols.clj" 169],
[clojure.core.protocols$fn__8144 invoke "protocols.clj" 124],
[clojure.core.protocols$fn__8099$G__8094__8108 invoke "protocols.clj" 19],
[clojure.core.protocols$seq_reduce invokeStatic "protocols.clj" 27],
[clojure.core.protocols$fn__8131 invokeStatic "protocols.clj" 75],
[clojure.core.protocols$fn__8131 invoke "protocols.clj" 75],
[clojure.core.protocols$fn__8073$G__8068__8086 invoke "protocols.clj" 13],
[clojure.core$reduce invokeStatic "core.clj" 6824],
[clojure.core$reduce invoke "core.clj" 6810],
[user$eval106182 invokeStatic "form-init3664079349963806059.clj" 15],
[user$eval106182 invoke "form-init3664079349963806059.clj" 14],
[clojure.lang.Compiler eval "Compiler.java" 7176],
[clojure.lang.Compiler eval "Compiler.java" 7131],
[clojure.core$eval invokeStatic "core.clj" 3214],
[clojure.core$eval invoke "core.clj" 3210],
[clojure.main$repl$read_eval_print__9068$fn__9071 invoke "main.clj" 414],
[clojure.main$repl$read_eval_print__9068 invoke "main.clj" 414],
[clojure.main$repl$fn__9077 invoke "main.clj" 435],
[clojure.main$repl invokeStatic "main.clj" 435],
[clojure.main$repl doInvoke "main.clj" 345],
[clojure.lang.RestFn applyTo "RestFn.java" 137],
[clojure.core$apply invokeStatic "core.clj" 665],
[clojure.core$apply invoke "core.clj" 660],
[refactor_nrepl.ns.slam.hound.regrow$wrap_clojure_repl$fn__62865 doInvoke "regrow.clj" 18],
[clojure.lang.RestFn invoke "RestFn.java" 1523],
[clojure.tools.nrepl.middleware.interruptible_eval$evaluate$fn__53116 invoke "interruptible_eval.clj" 87],
[clojure.lang.AFn applyToHelper "AFn.java" 152],
[clojure.lang.AFn applyTo "AFn.java" 144],
[clojure.core$apply invokeStatic "core.clj" 665],
[clojure.core$with_bindings_STAR_ invokeStatic "core.clj" 1973],
[clojure.core$with_bindings_STAR_ doInvoke "core.clj" 1973],
[clojure.lang.RestFn invoke "RestFn.java" 425],
[clojure.tools.nrepl.middleware.interruptible_eval$evaluate invokeStatic "interruptible_eval.clj" 85],
[clojure.tools.nrepl.middleware.interruptible_eval$evaluate invoke "interruptible_eval.clj" 55][clojure.tools.nrepl.middleware.interruptible_eval$interruptible_eval$fn__53161$fn__53164 invoke "interruptible_eval.clj" 222],
[clojure.tools.nrepl.middleware.interruptible_eval$run_next$fn__53156 invoke "interruptible_eval.clj" 190],
[clojure.lang.AFn run "AFn.java" 22],
[java.util.concurrent.ThreadPoolExecutor runWorker "ThreadPoolExecutor.java" 1149],
[java.util.concurrent.ThreadPoolExecutor$Worker run "ThreadPoolExecutor.java" 624],
[java.lang.Thread run "Thread.java" 748]]
Returned value:
176805
This has been working for years with the only difference that there is much more data to process theferore it takes longer.
I've tried realising the LazySeq with doall - same thing, but only the first time.
I've never seen a behaviour like this before, so any help is greatly appriciated.
Your line-items has an exception in it, which doesn't surface until you try to realize it for summation. You will see the same behavior from a value like this:
user=> (def xs (take 10 (map #(/ 2 %) (iterate dec 3))))
#'user/xs
user=> (apply + xs)
Execution error (ArithmeticException) at user/fn (REPL:1).
Divide by zero
user=> (apply + xs)
11/3 ;; (which is 2/3 + 2/2 + 2/1)
The fact that you get a valid result the second try doesn't mean your input was valid. The exception just caused xs to throw away all the stuff after the last valid item.
So, instead of looking for the bug in the code you've posted, figure out how you ended up producing an invalid line-items.