Search code examples
pythonpython-3.xclojure

How do I benchmark (compare) performance of Python vs Clojure?


I'm learning Clojure and to get a better handle on my progress I decided to start solving the Project Euler problems in the language (some of which I've already solved in C++ and Python). Problem 1 goes as follows:

If we list all the natural numbers below 10 that are multiples of 3 or 5, we get 3, 5, 6 and 9. The sum of these multiples is 23.

Find the sum of all the multiples of 3 or 5 below 1000.

Here's my first run at the Clojure solution:

(defn main1
  ([n]
    (reduce +
      (filter
        #(or
          (= 0 (mod % 3))
          (= 0 (mod % 5)))
        (range n)))))

I then looked at my Python version of the code, which is as follows:

import sys
import operator
from functools import reduce


def main(n=1000):
    """
    returns solution up to but excluding n
    """
    genexp = (num for num in range(1, n) if ((num % 3 == 0) or (num % 5 == 0)))
    total = reduce(operator.add, genexp)
    return total


if __name__ == "__main__":
    if len(sys.argv) > 1:
        print(main(int(sys.argv[1])))
    else:
        print(main())

Ignoring the extra CLI-args stuff I've added in the Python version, the only major difference is that I used filter in Clojure instead of a generator. I guess I could've used a filter in Python too, but just for the sake of argument, I made my Clojure code more similar to the Python code:

(defn main2
  ([n]
    (reduce + (for [i (range n)
                  :let [div3 (= 0 (mod i 3))
                        div5 (= 0 (mod i 5))]
                  :when (or div3 div5)]
               i))))

That got me thinking - how can I benchmark these functions to compare them? For Python, it's easy enough:

$ time python python/p0001.py 10000000
23333331666668

real    0m2.693s
user    0m2.660s
sys 0m0.018s

$ time python python/p0001.py 100000000
2333333316666668

real    0m26.494s
user    0m26.381s
sys 0m0.050s

It goes up to n=100,000,000 in a reasonable amount of time (under 30 seconds). How can I run a similar test for my Clojure funcs? I imagine I'd have to compile the Clojure code first before running it. Would that even be a fair comparison, given the Python code isn't JIT-compiled here?

On another note, how idiomatic is my Clojure code (both versions)? And what are some good recommendations for styling the code? Is there a pep8-like style-guide? Or even something along the lines of a Clojure version of pep20: the "Zen of Python"?


Solution

  • The built-in function time is good for a first cut:

    (time (main1 100000000)) => 2333333316666668
    "Elapsed time: 15824.041487 msecs"
    

    You can also use the excellent criterium library

    (ns xyz.core
      (:require [criterium.core :as cc] ))
    
    (cc/quick-bench (main1 999))
    
    Evaluation count : 3894 in 6 samples of 649 calls.
                 Execution time mean : 154.680082 µs
        Execution time std-deviation : 750.591607 ns
       Execution time lower quantile : 153.982498 µs ( 2.5%)
       Execution time upper quantile : 155.870826 µs (97.5%)
                       Overhead used : 7.898724 ns
    
    Found 1 outliers in 6 samples (16.6667 %)
        low-severe   1 (16.6667 %)
     Variance from outliers : 13.8889 % Variance is moderately inflated by outliers