Search code examples
neo4jnginx-ingresspy2neo

Getting "TimeoutError: [Errno 110] Connection timed out" in py2neo when neo4j db is exposed through loadbalancer

I have deployed neo4j community edition(version 4.0.0) in k8s and it has been exposed (bolt and browsers) through nginx-ingress and a load balancer.

Now when tried to connect to this db using py2neo(version latest), its working fine. But I got exception message: broken after 20 or 25 minutes of being idle.

Please be informed the idle-timeout time in AWS load balancer has been changed to the 3600s and also made this same for Nginx ingress.

exeption message: broken.

full log:

Traceback (most recent call last):
  File "/tmp/test/py2neo/wiring.py", line 270, in send
    n = self.__socket.send(self.__output)
TimeoutError: [Errno 110] Connection timed out
During handling of the above exception, another exception occurred:
in query
    return self.graph.run(query).data()
  File "/tmp/test/py2neo/database.py", line 579, in run
    return self.auto().run(cypher, parameters, **kwparameters)
  File "/tmp/test/py2neo/database.py", line 918, in run
    result = self._connector.auto_run(self.graph.name, cypher, parameters, hydrant)
  File "/tmp/test/py2neo/client/__init__.py", line 996, in auto_run
    cx.sync(result)
  File "/tmp/tes/py2neo/client/bolt.py", line 318, in sync
    self._send()
  File "/tmp/test/py2neo/client/bolt.py", line 354, in _send
    sent = self._writer.send()
  File "/tmp/test/py2neo/client/packstream.py", line 652, in send
    return self._tx.send()
  File "/tmp/test/py2neo/wiring.py", line 273, in send
    raise WireError("Broken")
py2neo.wiring.WireError: Broken

The flow of the trafic is like this :

py2neo client ---> botl address(record in route53 against loadbalancer) ---> loadbalancer ---> nginx-ingress--->neo4j service---> neo4j docker container(standalone)

Can anyone please help me to solve this issue?

Solution

  • What is your expectation here? Because py2neo contains no facilities to countermeasure hostile network environments in which connections are unexpectedly closed on a whim by third party network components.

    Also, I don't understand your overall topology. You say you are using Community Edition, therefore you cannot be running a cluster, yet you mention a load balancer. What servers are you load balancing over? Community Edition can only run single instances. Also, why put nginx in there too?

    To me, you have answered your own question. You have inserted network components that time out connections and py2neo is reporting exactly that. If you want to resume after such a connection failure, you will need to build client code to detect this and retry.

    If you were using a casual cluster (and therefore Enterprise Edition) I would recommend the official Python driver instead. But I'm not sure that would help you much in this case.