Search code examples
varnishvarnish-vcl

How to resolve Varnish FetchError "Timed out reusing backend connection"

I am seeing frequent errors Varnish FetchError "Timed out reusing backend connection". Checked couple of blogs, but not find any resolution. Could you please help?

  • BereqHeader Accept-Encoding: gzip
  • BereqHeader X-Varnish: 37849780
  • VCL_call BACKEND_FETCH
  • VCL_return fetch
  • BackendOpen 36 NODEJS_2 xx.xx.xx.xx 9000 yy.yy.yy.yy 43309
  • Timestamp Bereq: 1605444526.456709 0.000102 0.000102
  • FetchError Timed out reusing backend connection
  • BackendClose 36 NODEJS_2
  • Timestamp Beresp: 1605444571.456893 45.000285 45.000183
  • Timestamp Error: 1605444571.456900 45.000292 0.000006
  • BerespProtocol HTTP/1.1
  • BerespStatus 503
  • BerespReason Backend fetch failed
  • BerespHeader Date: Sun, 15 Nov 2020 12:49:31 GMT
  • BerespHeader Server: Varnish
  • VCL_call BACKEND_ERROR
  • BerespHeader Content-Type: text/html; charset=utf-8
  • BerespHeader Retry-After: 5
  • VCL_return deliver
  • Storage malloc Transient
  • Length 285
  • BereqAcct 2940 185 3125 0 0 0
  • End
Solution

  • The Timestamp Beresp: 1605444571.456893 45.000285 45.000183 tag in your VSL output indicates that your backend took 45.000183 seconds to generate its response, which triggered the first_byte_timeout.

    In reality, your backend probably needed more than 45 seconds to generate the output, but Varnish just gave up after it hit the timeout.

    Here are your options:

    1. Increase the first_byte_timeout runtime parameter to a better number
    2. Examine why your backend it taking so long

    Although option 1 is theoretically viable, you really want to go for option 2, and figure out why it is taking so long for the backend to respond.