Unexplainable WCF Service Timeout

Question

Running an MVC web site that calls a WCF service.
Site and service on 2 different machines.
.NET framework 4; IIS 7.

Production site is encountering a timeout when it should not. I created the simplest samples I could (simple web site that calls service method that returns "hi mom" after a delay) and was able to reproduce the problem. Here is how it goes:

• Client web app has it's binding's SendTimeout set to 6 minutes.
• Server service does nothing but write a message to a file, sleep for 5 minutes, write another message.
• Messages written to file on server are correct; server trace shows no problems.
• Client never receives return value from server service method. It's like it just loses it's connection to the server.
• Client throws TimeoutException after 6 minutes (any timeout greater than that just delays the exception further). If I change the thread sleep time on the server to 4 minutes, everything works fine. This is repeatable, and there is nobody else messing with this test scenario.

So here is the pertinent config section of the client:

    <binding name="BasicHttpBinding_IService1"
             closeTimeout="00:6:00"
             openTimeout="00:6:00"
             receiveTimeout="00:6:00"
             sendTimeout="00:6:00"
             allowCookies="false"
             bypassProxyOnLocal="false"
             hostNameComparisonMode="StrongWildcard"
             maxBufferSize="2147483647"
             maxBufferPoolSize="2147483647"
             maxReceivedMessageSize="2147483647"
             messageEncoding="Text"
             textEncoding="utf-8"
             transferMode="Buffered"
             useDefaultWebProxy="true">

<httpRuntime executionTimeout="1200"
             maxRequestLength="10240" />

  <serviceBehaviors>
    <behavior>
      <serviceTimeouts transactionTimeout="00:6:00" />
    </behavior>
  </serviceBehaviors>

The same values are present on the server. Following include things I've tried:

• Added the serviceTimeouts element above.
• Ensured to call close on the client proxy class (even though that wouldn't be the issue in my case since the error happens every time).
• Created a channel explicitly, cast as IContextChannel and set OperationTimeout (as outlined here).
• Increased the FailureInterval property of the server's app pool in IIS.
• Ensured that all the timeout values of the binding element in the configs were all set to a value greater than the time taken by the service method.
• Created a ChannelFactory and set it's Endpoint.Binding.SendTimeout value.
• Ensured that Compilation debug="false" was set at client and server (read that timeouts [unsure which timeouts] not respected correctly unless this is set).
• Increased maxBufferSize, maxBufferPoolSize, and maxReceivedMessageSize even though this shouldn't have anything to do with the problem (the server simply returns "hi mom").
• Set SendTimeout explicitly in code on the client proxy (since this should be the relevant timeout).
• Called the service from a console app (same result).

What else can I try?

[Edit] Moved the service to the same machine as the web app and the problem goes away.

Answer 1

It was HTTP policy on firewall in production environment: max request time 5 minutes.