I'm currently looking into Clojure and Incanter as an alternative to R. (Not that I dislike R, but it just interesting to try out new languages.) I like Incanter and find the syntax appealing, but vectorized operations are quite slow as compared e.g. to R or Python.
As an example I wanted to get the first order difference of a vector using Incanter vector operations, Clojure map and R . Below is the code and timing for all versions. As you can see R is clearly faster.
Incanter and Clojure:
(use '(incanter core stats))
(def x (doall (sample-normal 1e7)))
(time (def y (doall (minus (rest x) (butlast x)))))
"Elapsed time: 16481.337 msecs"
(time (def y (doall (map - (rest x) (butlast x)))))
"Elapsed time: 16457.850 msecs"
R:
rdiff <- function(x){
n = length(x)
x[2:n] - x[1:(n-1)]}
x = rnorm(1e7)
system.time(rdiff(x))
user system elapsed
1.504 0.900 2.561
So I was wondering is there a way to speed up the vector operations in Incanter/Clojure? Also solutions involving the use of loops, Java arrays and/or libraries from Clojure are welcome.
I have also posted this question to Incanter Google group with no responses so far.
UPDATE: I have marked Jouni's answer as accepted, see below for my own answer where I have cleaned up his code a bit and added some benchmarks.
Here's a Java arrays implementation that is on my system faster than your R code (YMMV). Note enabling the reflection warnings, which is essential when optimizing for performance, and the repeated type hint on y (the one on the def didn't seem to help for the aset) and casting everything to primitive double values (the dotimes makes sure that i is a primitive int).
(set! *warn-on-reflection* true)
(use 'incanter.stats)
(def ^"[D" x (double-array (sample-normal 1e7)))
(time
(do
(def ^"[D" y (double-array (dec (count x))))
(dotimes [i (dec (count x))]
(aset ^"[D" y
i
(double (- (double (aget x (inc i)))
(double (aget x i))))))))