Unable to connect to port 53589 on EC2 instance using Docker and Caddy server

Question

What I'm trying to do

Host a Taskwarrior Server on an AWS EC2 instance, and connect to it via a subdomain (e.g. task.mydomain.dev).

Taskwarrior server operates on port 53589.

Tech involved

• AWS EC2: the server (Ubuntu)
• Caddy Server: for creating a reverse proxy for each app on the EC2 instance
• Docker (docker-compose): for launching apps, including the Caddy Server and the Taskwarrior server
• Cloudflare: DNS hosting and SSL certificates

How I've tried to do this

I have:

• allowed incoming connections for ports 22, 80, 443 and 53589 in the instance's security policy
• given the EC2 instance an elastic IP
• setup the DNS records (task.mydomain.dev is CNAME'd to mydomain.dev, mydomain.dev has an A record pointing to the elastic IP)
• used Caddy server to setup a reverse proxy on port 53589 for task.mydomain.dev
• setup the Taskwarrior server as per instructions (i.e. certificates created; user and organisation created; taskrc file updated with cert, auth and server info; etc)

Config files

/opt/task/docker-compose.yml

version: '3.3'

services:
  taskd:
    image: connectical/taskd
    restart: always
    volumes:
      - /opt/task:/var/taskd
    ports:
      - 53589:53589

networks:
  default:
    external:
      name: caddy_net

/opt/caddy/docker-compose.yml

version: "3.4"

services:
  caddy:
    build:
      context: .
      dockerfile: Dockerfile
    container_name: caddy
    restart: always
    ports:
      - 80:80
      - 443:443
    volumes:
      - ./config:/config
      - ./data:/data
      - ./Caddyfile:/etc/caddy/Caddyfile

networks:
  default:
    external:
      name: caddy_net

/opt/caddy/Caddyfile:

task.mydomain.dev:53589 {
  reverse_proxy taskd:53589
  tls {
    dns cloudflare myCloudflareAPIkey
  }
}

What's actually happening

• I'm unable to connect to port 53589 on task.mydomain.dev
  • Running telnet task.mydomain.dev 53589 times out
• I'm unable to connect to port 53589 on mydomain.dev
  • Running telnet mydomain.dev 53589 times out
• I'm able to connect to port 53589 at 127.0.0.1 by ssh'ing into the EC2 instance
  • Runningtelnet 127.0.0.1 53589 from the EC2 instance successfully connects
• I'm able to connect to port 80 on task.mydomain.dev, but unable to sync with the Taskwarrior server
  • Running task sync init returns:

    c: 1 Received record packet of unknown type 72
    Syncing with task.mydomain.dev:80
    
    Cannot perform this action while handshake is in progress.
    Sync failed.  Could not connect to the Taskserver.
    

• I'm able to connect to port 443 on task.mydomain.dev, but unable to sync with the Taskwarrior server
  • Running task sync init returns:

    Syncing with task.mydomain.dev:443
    
    Malformed message
    Sync failed.  Could not connect to the Taskserver.
    

What I've tried to fix it

• Changing the Caddyfile's first line to:
  • task.mydomain.dev { and task.mydomain.dev:80 {, then connecting to port 80
    • Running task sync init returns:

      c: 1 Received record packet of unknown type 72
      Syncing with task.mydomain.dev:80
      
      Cannot perform this action while handshake is in progress.
      Sync failed.  Could not connect to the Taskserver.
      

  • task.mydomain.dev { and task.mydomain.dev:443 {, then connecting to port 443
    • Running task sync init returns:

      Syncing with task.mydomain.dev:443
      
      Malformed message
      Sync failed.  Could not connect to the Taskserver.
      

• Changing Caddyfile's second line to reverse_proxy 127.0.0.1:53589, reverse_proxy 0.0.0.0:53589 and reverse_proxy localhost:53589. Same errors occur.
• Removing the CNAME records for the subdomain. Same errors occur

Does anyone have any idea what's happening or could point me in the right direction?

Answer 1

If you are attempting to proxy HTTPS traffic on Cloudflare on a port not on the standard list, you will need to follow one of these options:

1. Set it up as a Cloudflare HTTPS Spectrum app on the required port 53589
2. Set up the record in the Cloudflare DNS tab as Grey cloud (in other words, it will only perform the DNS resolution - meaning you will need to manage the certificates on your side)
3. Change your service so that it listens on one of the standard HTTPS ports listed in the documentation in point (1)