I am using Pyramid (1.5.7), waitress (0.8.9) with Cassandra (2.2.0). It seems like Waitress and Cassandra driver are using both asyncore and somehow they are stepping on each others toes. This is my code on the app/__init__.py file:
import logging.config
from .action.context import root_factory
from pyramid.config import Configurator
from cassandra.cluster import Cluster
from cassandra.query import named_tuple_factory
def main(global_config, **settings):
""" This function returns a Pyramid WSGI application."""
# support logging in python3
logging.config.fileConfig(
settings['logging.config'],
disable_existing_loggers=False
)
config = Configurator(settings=settings, root_factory=root_factory)
# Retrieves connection to Cassandra (Non SQL database)
def get_non_sql(request):
cluster = Cluster(['127.0.0.1'], port=9042)
session = cluster.connect('app')
def disconnect(request):
cluster.shutdown()
request.add_finished_callback(disconnect)
return cluster.connect('app')
#Adding scheduler configuration
config.configure_celery(global_config['__file__'])
config.add_request_method(get_non_sql, 'non_sql', reify=True)
config.scan()
return config.make_wsgi_app()
Generated errors:
2015-07-27 12:24:36,779 ERROR [waitress][cassandra_driver_event_loop] Socket error.
Traceback (most recent call last):
File "python3.4/site-packages/waitress-0.8.9-py3.4.egg/waitress/channel.py", line 167, in handle_read
data = self.recv(self.adj.recv_bytes)
File "python3.4/asyncore.py", line 379, in recv
data = self.socket.recv(buffer_size)
BlockingIOError: [Errno 35] Resource temporarily unavailable
2015-07-27 12:24:37,079 ERROR [waitress][MainThread] Unexpected exception when flushing.
File "python3.4/site-packages/waitress-0.8.9-py3.4.egg/waitress/server.py", line 154, in run
use_poll=self.adj.asyncore_use_poll,
File "python3.4/asyncore.py", line 208, in loop
poll_fun(timeout, map)
File "python3.4/asyncore.py", line 145, in poll
r, w, e = select.select(r, w, e, timeout)
OSError: [Errno 9] Bad file descriptor
2015-07-27 12:33:32,649 DEBUG [cassandra.io.asyncorereactor][cassandra_driver_event_loop] Asyncore event loop ended
Does someone know a workaround for this issue?
Waitress and cassandra-driver were using the same socket. I've deleted the version 0.8.9 of waitress and installed the latest development version (master branch)
pip uninstall waitress
pip install git+git://github.com/Pylons/waitress@master
The asyncore socket has been changed in this version so it works perfectly. Nonetheless I recommend using libev instead, it seems to have better performance than asyncore. In OS X it's as easy as installing the libev library:
brew install libev
Right after doing this, install the cassandra-driver (It will detect the libev library automatically)
pip install cassandra-driver
And then just use the LibevConnection class:
from cassandra.io.libevreactor import LibevConnection
from cassandra.cluster import Cluster
cluster = Cluster()
cluster.connection_class = LibevConnection
session = cluster.connect()