Search code examples
androidiis-7.5microsoft-sync-framework

MS Syncframework 4 - no response after 5 minutes of server side processing

Background:

We have set up a syncronization on an old production server with MS SyncFrameWork 4 between several MS SQL databases and android devices. The initial sync always took the longest but that time is not an issue since it only occurs the very first time a device is used.

Server details:

  • OS: Windows Web Server 2008 R2
  • Database: SQL Server 2008 R2
  • IIS: 7.5
  • SyncFramework 4

Issue:

If the database on the server is large - and the initial sync takes more than 5 minutes to process on the server side - the android client doesn't seem to get a response from the server and runs in a timeout.

Depending on the load of the server the initial sync process on the server side takes 3 to 10 minutes. If the initial sync succeeds there are no problems afterwards.

When checking the logs and traces we cannot find any error. The IIS working process seems to complete the task with no error even if it takes more than 5 minutes. But if 

    public Message ProcessRequestForMessage(Stream messageBody)

in class Microsoft.Synchronization.Services.SyncService needs more than 5 minutes to complete, the android client does not get a response. Even if it is within the specified socket timeout of 900 seconds.

On the Android device occurs a IOException: java.net.SocketTimeoutException: Read timed out Exception after 900 seconds.

Code on Android side for initialization of httpClient:

    HttpParams httpParameters = new BasicHttpParams();
    int timeout = 900 * 1000;
    HttpConnectionParams.setConnectionTimeout(httpParameters, 5000);
    HttpConnectionParams.setSoTimeout(httpParameters, timeout);
    DefaultHttpClient httpclient = new DefaultHttpClient(httpParameters);

Later use:

    HttpResponse response = httpclient.execute(request);

What we tried:

  • increasing socket timeout values on Android side
  • increasing IIS application pool worker process timeouts (although the default value of 20 minutes was never hit anyway)
  • increased ExecutionTimeout to 600 and even more in Web.config - taken from IIS Request Timeout on long ASP.NET operation
  • increased binding timeouts in Web.config: openTimeout="00:15:00" closeTimeout="00:15:00" sendTimeout="00:15:00" receiveTimeout="00:15:00"

So my questions are:

  • are there any other IIS process timeouts settings that silently prevent sending a response when hit? Where can we set them?
  • is there a known problem with androids httpclient to silently stop working after idling for 5 minutes?
  • any other idea what might interfere or where we can tweak our code?

Edit for clarity:

  • We use batching (filesize 1000 kb)
  • We changed default batch folder from C:\Windows\Temp to C:\andsync\batchdir which is working so far as indended
  • Issue also arises when the android device does not go into standby on Nexus 6p (Android 8) and Nexus 9 (Android 7.1.1) so I think we can dismiss the doze / standby Android feature mentioned by Selvin
  • In the SyncScope is only one table which contains one column of type 'image'. This table is always increasing in size and at around 30.000 entries (which accumulates to roughly 2,2 GB size) it starts showing the described behaviour.
Solution

  • The answer to my question:

    The code on both client and server side is OK. The culprit was the NAT 'TCP-Aging'-Setting in our LANCOM router. https://www.lancom-systems.com/docs/LCOS/menu-reference/topics/2_8_9_1.html

    It was configured to 300 seconds => 5 minutes ... So our router silently cut off the connection after idling for 5 minutes ...

    I understand it's not best practice to keep an idle connection alive for so long, but since this rarely occurs in our use case, it's ok for us.

    The answer to this post brought me on the right track: Netty websocket client does not read new frames from server after 5 minutes of being idle