I want to create an Openshift route to a RabbitMQ.
As far as I understand the documentation at https://docs.openshift.com/container-platform/3.11/architecture/networking/routes.html, the "TLS with SNI" support should be able to inspect the SNI header and then route the traffic to the appropriate service.
Unfortunately, I'm having problems both with edge termination and with reencrypt termination. In both cases, the HAProxy seems to inspect the traffic and throws an error because the content is no HTTP.
With a simple Java application which uses the AMQP Java library, I can see the following traffic:
javax.net.ssl|DEBUG|01|main|2021-02-26 13:35:47.001 CET|SSLSocketOutputRecord.java:331|WRITE: TLS12 application_data, length = 8
javax.net.ssl|DEBUG|01|main|2021-02-26 13:35:47.001 CET|SSLCipher.java:1770|Plaintext before ENCRYPTION (
0000: 41 4D 51 50 00 00 09 01 AMQP....
)
javax.net.ssl|DEBUG|0D|AMQP Connection 192.168.0.1:443|2021-02-26 13:35:47.044 CET|SSLSocketInputRecord.java:249|READ: TLSv1.2 application_data, length = 211
javax.net.ssl|DEBUG|0D|AMQP Connection 192.168.0.1:443|2021-02-26 13:35:47.045 CET|SSLCipher.java:1672|Plaintext after DECRYPTION (
0000: 48 54 54 50 2F 31 2E 30 20 34 30 30 20 42 61 64 HTTP/1.0 400 Bad
0010: 20 72 65 71 75 65 73 74 0D 0A 43 61 63 68 65 2D request..Cache-
0020: 43 6F 6E 74 72 6F 6C 3A 20 6E 6F 2D 63 61 63 68 Control: no-cach
0030: 65 0D 0A 43 6F 6E 6E 65 63 74 69 6F 6E 3A 20 63 e..Connection: c
0040: 6C 6F 73 65 0D 0A 43 6F 6E 74 65 6E 74 2D 54 79 lose..Content-Ty
0050: 70 65 3A 20 74 65 78 74 2F 68 74 6D 6C 0D 0A 0D pe: text/html...
0060: 0A 3C 68 74 6D 6C 3E 3C 62 6F 64 79 3E 3C 68 31 .<html><body><h1
0070: 3E 34 30 30 20 42 61 64 20 72 65 71 75 65 73 74 >400 Bad request
0080: 3C 2F 68 31 3E 0A 59 6F 75 72 20 62 72 6F 77 73 </h1>.Your brows
0090: 65 72 20 73 65 6E 74 20 61 6E 20 69 6E 76 61 6C er sent an inval
00A0: 69 64 20 72 65 71 75 65 73 74 2E 0A 3C 2F 62 6F id request..</bo
00B0: 64 79 3E 3C 2F 68 74 6D 6C 3E 0A dy></html>.
)
(The output is generated with
java -Djavax.net.debug=all -jar rabbitmqtest-1.0-SNAPSHOT-all.jar amqps://myroute:443 2> output.txt)
The traffic does not get routed to the RabbitMQ. If I open the router in the web browser, RabbitMQ receives a connection request (but of course cannot understand it, because it is HTTP traffic).
The Helm route template is:
{{- if .Values.route.enabled }}
apiVersion: route.openshift.io/v1
kind: Route
metadata:
name: rabbitmq
spec:
host: rabbitmq-{{ .Values.route.identifier}}.{{ .Values.route.host }}
port:
targetPort: amqp-ssl
tls:
termination: reencrypt
destinationCACertificate: {{ .Values.tls.ca_crt | quote }}
to:
kind: Service
weight: 100
name: rabbitmq
status:
ingress: []
{{- end }}
Is there any way to use the router as a raw TCP proxy? I would prefer not to use a node port, as I would have to manage SSL certs on the RabbitMQ (Currently I have installed a long-lived self signed cert).
In theory, using an Ingress Controller should work in this scenario, as you are using TLS with SNI. It may however require additional work on the infrastructure side (adding additional ports to the loadbalancer, firewalling). Using the "standard" port 443 will not work (as you noted), as the Router expects HTTP traffic on that port.
Alternatives are to use externalIP (non-cloud environments) or exposing services using loadbalancers (cloud environment).