Search code examples
rubynginxwebsocketlxc

How to serve websocket apps from inside lxc container?


Inside lxc container I'm running a faye app.

Gemfile:

source 'https://rubygems.org'
gem 'faye'
gem 'thin'

config.ru:

require 'faye'
Faye::WebSocket.load_adapter('thin')
bayeux = Faye::RackAdapter.new(:mount => '/faye', :timeout => 25)
run bayeux

Then

$ thin start

/etc/nginx/sites-available/domain.com:

server {
    server_name   domain.com;
    location /faye {
        proxy_pass http://localhost:3000;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
    }
}

On the host:

/etc/nginx/sites-available/domain.com:

server {
    server_name   domain.com;
    location / {
        proxy_pass   http://10.0.0.109:80;
        proxy_set_header   X-Forwarded-For   $proxy_add_x_forwarded_for;
        proxy_set_header   X-Real-IP   $remote_addr;
        proxy_set_header   Host   $http_host;
    }
}

Then, I try to connect to it from here (ws://domain.com/faye), but it fails. What am I doing wrong?

The app says:

Using rack adapter
Thin web server (v1.6.4 codename Gob Bluth)
Maximum connections set to 1024
Listening on 0.0.0.0:3000, CTRL+C to stop

Guest nginx access log:

10.0.0.1 - - [09/Dec/2015:11:03:21 +0200] "GET /faye?encoding=text HTTP/1.0" 400 11 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.86 Safari/537.36"

Host nginx access log:

11.111.111.111 - - [09/Dec/2015:11:03:21 +0200] "GET /faye?encoding=text HTTP/1.1" 400 21 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.86 Safari/537.36"

chrome's Developer tools's console:

WebSocket connection to 'ws://domain.com/faye?encoding=text' failed: Error during WebSocket handshake: Unexpected response code: 400


I tried to run the app suggested by Myst on the guest.

Gemfile:

source 'https://rubygems.org'
gem 'plezi'

app.rb:

#!/usr/bin/env ruby
# require the gems
require 'bundler'
Bundler.require(:default, ENV['ENV'].to_s.to_sym)
# handle requests
class MyController
    # Http
    def index
        Iodine.log request_data_string
    end
    # Websockets
    def on_message data
        write ERB::Util.html_escape(data)
    end
    def pre_connect
        puts Iodine.log(request_data_string)
        true
    end
    def on_open
        write 'Welcome!'
    end
    # formatting the request data
    protected
    def request_data_string
        out = String.new
        out << "Request headers:\n"
        out << (request.headers.to_a.map {|p| p.join ': '} .join "\n")
        out << "\n\nRequest cookies:\n"
        out << (request.cookies.to_a.map {|p| p.join ': '} .join "\n")
        out << "\n\nAll request data:\n"
        out << (request.to_a.map {|p| p.join ': '} .join "\n")
        out
    end
end
route '*', MyController
# # you can also set up logging to a file:
# Plezi.logger = Logger.new("filename.log")

Then

$ ruby app.rb

/etc/nginx/sites-available/domain.com:

server {
    server_name   domain.com;
    location / {
        proxy_pass http://localhost:3000;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
    }
}

With this when I connect to ws://domain.com/ the app says:

Iodine 0.1.19 is listening on port 3000
Plezi is feeling optimistic running version 0.12.21.

Press ^C to stop the server.
Request headers:
connection: upgrade
host: localhost:3000
x-forwarded-for: 11.111.111.111
pragma: no-cache
cache-control: no-cache
origin: http://www.websocket.org
sec-websocket-version: 13
user-agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.86 Safari/537.36
accept-encoding: gzip, deflate, sdch
accept-language: en-US,en;q=0.8
sec-websocket-key: Qs2LMnJ12SjclOxlrYKwlg==
sec-websocket-extensions: permessage-deflate; client_max_window_bits

Request cookies:


All request data:
io: #<Iodine::Http::Http1:0x007fd5a0006b08>
cookies: {}
params: {:encoding=>"text"}
method: GET
query: /?encoding=text
version: 1.1
time_recieved: 2015-12-09 11:24:16 +0200
connection: upgrade
host: localhost:3000
x-forwarded-for: 11.111.111.111
pragma: no-cache
cache-control: no-cache
origin: http://www.websocket.org
sec-websocket-version: 13
user-agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.86 Safari/537.36
accept-encoding: gzip, deflate, sdch
accept-language: en-US,en;q=0.8
sec-websocket-key: Qs2LMnJ12SjclOxlrYKwlg==
sec-websocket-extensions: permessage-deflate; client_max_window_bits
headers_complete: true
client_ip: 11.111.111.111
scheme: http
host_name: localhost
port: 3000
path:
original_path: /
query_params: encoding=text
host_settings: {:index_file=>"index.html", :assets_public=>"/assets", :public=>nil, :assets_public_regex=>/^\/assets\//i, :assets_public_length=>8, :assets_refuse_templates=>/(erb|coffee|scss|sass|\.\.\/)$/i}11.111.111.111 [2015-12-09 09:24:16 UTC] "GET / http/1.1" 200 1659 0.6ms

Guest nginx access log:

10.0.0.1 - - [09/Dec/2015:10:55:44 +0200] "GET /?encoding=text HTTP/1.0" 200 1526 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.86 Safari/537.36"

Host nginx access log:

11.111.111.111 - - [09/Dec/2015:10:55:44 +0200] "GET /?encoding=text HTTP/1.1" 200 1526 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.86 Safari/537.36"

chrome's Developer tools's console:

WebSocket connection to 'ws://domain.com/?encoding=text' failed: Error during WebSocket handshake: Unexpected response code: 200


Solution

  • Probably, the easier way to set it up is to proxy from host nginx directly to the app:

    /etc/nginx/sites-available/domain.com (host):

    server {
        server_name   domain.com;
        location / {
            proxy_pass   http://10.0.0.109:3000;
            proxy_http_version 1.1;
            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection 'upgrade';
        }
    }
    

    But if you want to have the same nginx configuration regardless whether the app is running on the host, or guest, you can proxy it with two nginx.

    /etc/nginx/sites-available/domain.com (host):

    server {
        server_name   domain.com;
        location / {
            proxy_pass   http://10.0.0.109:80;
            proxy_set_header   Host   $http_host;
            proxy_http_version 1.1;
            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection 'upgrade';
        }
    }
    

    Passing Host header here is vital. Or else guest nginx might let the wrong virtual host handle the request. That was what I couldn't realize for quite a while.

    /etc/nginx/sites-available/domain.com (guest):

    server {
        server_name   domain.com;
        location / {
            proxy_pass http://localhost:3000;
            proxy_http_version 1.1;
            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection 'upgrade';
        }
    }