Search code examples
pythonnetwork-programmingcherrypy

Making multiple requests to get data from the CherryPy web service

I create a CherryPy web service which stores data in the map, receives the key from clients and returns the corresponding data:

import sys
import imp
import cherrypy

data_source = get_data() # get data from the database and store it in the map

class Provider:
    exposed = True

    def POST(self, key):
        global data_source
        data = data_source[key] # get stored data based on given key
        return data

if __name__ == '__main__':
    cherrypy.tree.mount(Provider(), '/Provider',{'/':
            {'request.dispatch': cherrypy.dispatch.MethodDispatcher()}
        })
    cherrypy.config.update({'server.socket_host': '0.0.0.0',
            'server.socket_port': 8080,
        })
    cherrypy.server.max_request_body_size = 1048576000
    cherrypy.engine.start()
    cherrypy.engine.block()

Then, on another machine, I create a script to request data from the provider. Using the script, it is possible to scpecify how many concurrent requests that I want to make:

import requests
import time
from threading import Thread

def make_request(id, key):
    start = time.time()
    r = requests.post("http://provider-host/Provider", {'key':key})
    end = time.time()
    print 'Thread {0} takes {1} seconds to finish with status code {2}'.format(id, end - start, r.status_code)

def start(num, key):
    ts = []
    for i in range(num):
        t = Thread(target=make_request, args=(i, key))
        ts.append(t)
    for t in ts: t.start()
    for t in ts: t.join()

Finally, I make a test to request the same key 10 times, with 2 different methods: sequential and concurrent.

Sequential method:

time for i in range(10): start(1, 'big_data_key')

The result is:

Thread 0 takes 2.51558494568 seconds to finish with status code 200
Thread 0 takes 2.47761011124 seconds to finish with status code 200
Thread 0 takes 2.66229009628 seconds to finish with status code 200
Thread 0 takes 2.47381901741 seconds to finish with status code 200
Thread 0 takes 2.4907720089 seconds to finish with status code 200
Thread 0 takes 2.93357181549 seconds to finish with status code 200
Thread 0 takes 2.47671484947 seconds to finish with status code 200
Thread 0 takes 2.40888786316 seconds to finish with status code 200
Thread 0 takes 2.6319899559 seconds to finish with status code 200
Thread 0 takes 2.77075099945 seconds to finish with status code 200
CPU times: user 1.79 s, sys: 1.06 s, total: 2.85 s
Wall time: 25.9 s

Concurrent methods:

time start('138.251.195.251', 10, 'big_data_key')

The result is:

Thread 5 takes 15.5736939907 seconds to finish with status code 200
Thread 1 takes 19.4057281017 seconds to finish with status code 200
Thread 7 takes 21.4743158817 seconds to finish with status code 200
Thread 8 takes 22.4408829212 seconds to finish with status code 200
Thread 0 takes 24.1915988922 seconds to finish with status code 200
Thread 2 takes 24.3175201416 seconds to finish with status code 200
Thread 6 takes 24.3368370533 seconds to finish with status code 200
Thread 4 takes 24.3618791103 seconds to finish with status code 200
Thread 9 takes 24.3891952038 seconds to finish with status code 200
Thread 3 takes 24.5536601543 seconds to finish with status code 200
CPU times: user 2.34 s, sys: 1.67 s, total: 4.01 s
Wall time: 24.6 s

It is clear that using the concurrent method, the time it takes to finish one request is higher than those in sequential method.

So, my question is: is the difference in download times caused by the bandwidth between 2 machines only or by something else, e.g. cherrypy related? If it is caused by something else, I would appreciate any suggestions to deal with it.

Solution

  • Then it is clear that your bottleneck is the network. Having 220MiB to transfer at 10.8MiB/s should take at least ~20 seconds. You're experiment takes ~25 seconds, which is 8.8MiB/s, i.e. effective ~74Mbit/s out of 100MBit/s of maximal theoretical capacity. It is good result, taking into account all possible errors of measure.

    The difference between serial and parallel case (~5%) shows that multiplexing doesn't help because the bottleneck is the network bandwidth, not individual connection limitation.

    To measure CherryPy impact, you can setup a web-server written in native code, I suggest nginx, put the file there, and try to download in 10 times. For parallel test you can try Apache ab, like ab -n 10 -c 10 http://provider-host/some-big-file.

    Also a couple of notes about CherryPy:

    • CherryPy is a threaded server, default thread pool, server.thread_pool, has 10 workers,
    • Try activate gzip compression, '/' : {'tools.gzip.on': True}, in your config, which will give a significant boost on plain text data.

    You may also take a look at this question, about handling big file downloads with CherryPy.