We have a server running a Flask App, which uses GeoServer under Tomcat7 to serve map data internally to the App itself, and externally via WMS to authenticated users.
This was all working fine until after some minor code edit/reload, command-line updates/upgrades and reboot (including reboot from EC2 console) when things stopped working.
After a bit of digging and debugging, it seems as if all the elements of the system are working ok individually, but the crucial requests to Tomcat7 and GeoServer via port 8080 are not connecting and timeout (usually after around 2mins).
I’ve taken a look at lots of online help for similar problems and checked, implemented and tested several of the suggestions, but nothing seems to clear the blockage.
I’m no expert (though I’m learning lots in the process!) and I’ve tried all the obvious & recommended things as far as I can tell.
I now seem to be going round in circles (i.e. nowhere) on this, so any suggestions as to what I might try next would be most gratefully received.
Background Info:
Security Group configured and applied to EC2 instance with rules as follows:
INBOUND RULES
Type Protocol Port Range Source
HTTP TCP 80 0.0.0.0/0
HTTP TCP 80 ::/0
Custom TCP Rule TCP 8080 x.x.x.x/32
Custom TCP Rule TCP 5432 x.x.x.x/32
SSH TCP 22 x.x.x.x/32
Custom TCP Rule TCP 443 0.0.0.0/0
Custom TCP Rule TCP 443 ::/0
OUTBOUND RULES
Type Protocol Port Range Destination
All traffic All All 0.0.0.0/0
Domain name resolves Ok to the Public IP
Checked this post and updated /usr/share/tomcat7/bin/catalina.sh to add the following:
JAVA_OPTS=" $JAVA_OPTS -Djava.net.preferIPv4Stack=true -Djava.net.preferIPv4Ad=true "
Checked this post and updated /etc/tomcat7/server.xml to make the Connector string:
<Connector port="8080" protocol="HTTP/1.1"
connectionTimeout="20000"
URIEncoding="UTF-8"
redirectPort="8443"
address="0.0.0.0"
useIPVHosts="true" />
Checked what’s listening on different ports using netstat:
root@ip-xxx-xx-xx-xx:/var/log/apache2# netstat -ltnpa | grep -i --colour LISTEN
tcp 0 0 127.0.0.1:587 0.0.0.0:* LISTEN 1438/sendmail: MTA:
tcp 0 0 0.0.0.0:8080 0.0.0.0:* LISTEN 1437/java
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 1208/sshd
tcp 0 0 0.0.0.0:5432 0.0.0.0:* LISTEN 1350/postgres
tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN 1438/sendmail: MTA:
tcp 0 0 127.0.0.1:8005 0.0.0.0:* LISTEN 1437/java
tcp6 0 0 :::80 :::* LISTEN 1456/apache2
tcp6 0 0 :::22 :::* LISTEN 1208/sshd
tcp6 0 0 :::5432 :::* LISTEN 1350/postgres
tcp6 0 0 :::443 :::* LISTEN 1456/apache2
Problems Observed
Debug reveals that the Flask app sends well-formed requests to http://Public-IP:8080/geoserver/roar/wms?... but the call to urllib2.urlopen(req) throws exception ‘[Errno 110] Connection timed out’, which is reported with abort(404).
If I take the request string (with Public-IP) and paste it in the browser it works. Note this is http:// and not https:// - not sure if this is significant - see below for notes on using https://
If I try cURL from the command line using the Public-IP, I get timeout messages:
connect to Public-IP port 8080 failed: Connection timed out
* Failed to connect to Public-IP port 8080: Connection timed out
* Closing connection 0
If I try cURL from the command line using the localhost, e.g. to Tomcat7 on port 8080, it works…
$ curl -v localhost:8080/
* Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 8080 (#0)
> GET / HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/7.47.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Server: Apache-Coyote/1.1
< Accept-Ranges: bytes
< ETag: W/"1896-1493801565000"
< Last-Modified: Wed, 03 May 2017 08:52:45 GMT
< Content-Type: text/html
< Content-Length: 1896
< Date: Fri, 22 Jun 2018 09:39:46 GMT
<
<?xml version="1.0" encoding="ISO-8859-1"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<title>Apache Tomcat</title>
</head>
<body>
<h1>It works !</h1>
etc…….
If I try cURL from the command line using the Private-IP, e.g. to Tomcat7 on port 8080, it works…
$ curl -v Private-IP:8080/
* Trying Private-IP...
* Connected to Private-IP (Private-IP) port 8080 (#0)
> GET / HTTP/1.1
> Host: Private-IP:8080
> User-Agent: curl/7.47.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Server: Apache-Coyote/1.1
< Accept-Ranges: bytes
< ETag: W/"1896-1493801565000"
< Last-Modified: Wed, 03 May 2017 08:52:45 GMT
< Content-Type: text/html
< Content-Length: 1896
< Date: Fri, 22 Jun 2018 09:42:30 GMT
<
<?xml version="1.0" encoding="ISO-8859-1"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<title>Apache Tomcat</title>
</head>
<body>
<h1>It works !</h1>
etc…….
If I simply try connecting to port 8080 from the browser using http:// with either the domain name or the Public-IP, I get the Tomcat7 index.html ‘It works!’ response.
If I try the same using https://, I get the following from Mozilla…
Secure Connection Failed
An error occurred during a connection to Public-IP:8080. SSL received a record that
exceeded the maximum permissible length. Error code: SSL_ERROR_RX_RECORD_TOO_LONG
The page you are trying to view cannot be shown because the authenticity of the received data
could not be verified.
and from IE (suggested settings are turned on)…
This page can’t be displayed
Turn on TLS 1.0, TLS 1.1, and TLS 1.2 in Advanced settings and try connecting to
https://Public-IP:8080 again. If this error persists, it is possible that this site uses an
unsupported protocol or cipher suite such as RC4 (link for the details), which is not considered
secure. Please contact your site administrator.
and from Chrome…
This site can’t provide a secure connection
Public-IP sent an invalid response.
Try running Windows Network Diagnostics.
ERR_SSL_PROTOCOL_ERROR
If Flask app and GeoServer run on the same EC2 server, they should communicate between each other via localhost not Private-IP or Public-IP