Search code examples
c#wcfcachingappfabric

AppFabric Caching adding large object to a remote server


I'm using AppFabric caching for Windows Server 1.1. I'm trying to add an object of 80MB to the cache which is located on another server on the network. I get the following error:

ErrorCode:SubStatus:There is a temporary failure. Please retry later. (One or more specified cache servers are unavailable, which could be caused by busy network or servers. For on-premises cache clusters, also verify the following conditions. Ensure that security permission has been granted for this client account, and check that the AppFabric Caching Service is allowed through the firewall on all cache hosts. Also the MaxBufferSize on the server must be greater than or equal to the serialized object size sent from the client.): inner System.ServiceModel.CommunicationException: The socket was aborted because an asynchronous receive from the socket did not complete within the allotted timeout of 00:01:00. The time allotted to this operation may have been a portion of a longer timeout. ---> System.Net.Sockets.SocketException: An established connection was aborted by the software in your host machine at System.ServiceModel.Channels.SocketConnection.Write(Byte[] buffer, Int32 offset, Int32 size, Boolean immediate, TimeSpan timeout) --- End of inner exception stack trace --- at System.ServiceModel.Channels.SocketConnection.Write(Byte[] buffer, Int32 offset, Int32 size, Boolean immediate, TimeSpan timeout) at System.ServiceModel.Channels.SocketConnection.Write(Byte[] buffer, Int32 offset, Int32 size, Boolean immediate, TimeSpan timeout, BufferManager bufferManager) at System.ServiceModel.Channels.BufferedConnection.WriteNow(Byte[] buffer, Int32 offset, Int32 size, TimeSpan timeout, BufferManager bufferManager) at System.ServiceModel.Channels.BufferedConnection.Write(Byte[] buffer, Int32 offset, Int32 size, Boolean immediate, TimeSpan timeout, BufferManager bufferManager) at System.ServiceModel.Channels.FramingDuplexSessionChannel.OnSend(Message message, TimeSpan timeout) at System.ServiceModel.Channels.OutputChannel.Send(Message message, TimeSpan timeout) at Microsoft.ApplicationServer.Caching.WcfClientChannel.SendOnChannel(EndpointID endpoint, TimeSpan& timeout, WaitCallback callback, Object state, Boolean async, IDuplexSessionChannel channel, Message message)

I can add smaller objects fine to the remote server so it's obviously something to do with the size. I've also installed the Cumulative update package 3 for AppFabric. I've disabled all security. Any ideas?

Server Config File:

<configuration>
    <configSections>
        <section name="dataCache" type="Microsoft.ApplicationServer.Caching.DataCacheSection, Microsoft.ApplicationServer.Caching.Core, Version=1.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" />
    </configSections>
    <dataCache size="Small">
        <caches partitionCount="32">
            <cache consistency="StrongConsistency" name="default" minSecondaries="0">
                <policy>
                    <eviction type="Lru" />
                    <expiration defaultTTL="1440" isExpirable="true" />
                </policy>
            </cache>
        </caches>
        <hosts>
            <host replicationPort="22236" arbitrationPort="22235" clusterPort="22234"
                hostId="1216617116" size="6126" leadHost="true" account="test$"
                cacheHostName="AppFabricCachingService" name="test.com"
                cachePort="22233" />
        </hosts>
        <advancedProperties>
            <securityProperties mode="None" protectionLevel="None">
                <authorization>
                    <allow users="IIS AppPool\test" />
                </authorization>
            </securityProperties>
            <transportProperties maxBufferPoolSize="26843545600" maxBufferSize="838860800" receiveTimeout="40000" />
        </advancedProperties>
        <deploymentSettings>
            <deploymentMode value="RoutingClient" />
        </deploymentSettings>
    </dataCache>
</configuration>

Section from Web.config from Client which is trying to push to AppFabric:

<dataCacheClient requestTimeout="60000" channelOpenTimeout="12000" maxConnectionsToServer="1">
    <localCache isEnabled="false" sync="TimeoutBased" ttlValue="300" objectCount="10000"/>
    <clientNotification pollInterval="300" maxQueueLength="10000"/>        
    <securityProperties mode="None" protectionLevel="None" />
    <transportProperties connectionBufferSize="131072" maxBufferPoolSize="568435456" maxBufferSize="183886080" maxOutputDelay="2" channelInitializationTimeout="60000" receiveTimeout="60000" />  </dataCacheClient>

The connection is established in code as follows:

DataCacheServerEndpoint[] servers = new DataCacheServerEndpoint[1];
                servers[0] = new DataCacheServerEndpoint("remoteServerName", 22233);

                DataCacheFactoryConfiguration factoryConfig = new DataCacheFactoryConfiguration();
                factoryConfig.Servers = servers;

                //Pass configuration settings to cacheFactory constructor
                _factory = new DataCacheFactory(factoryConfig);

                _cache = _factory.GetCache("default");

And finally the code to add an item to the cache:

_cache.CreateRegion(region);                             
_cache.Put(key, value, new TimeSpan(0, timeToLive, 0), tag, region); 

Solution

  • The solution to this issue was to increase the requestTimeout in the client web.config and remove the receiveTimeout setting.

    The web.config now looks like:

    <dataCacheClient requestTimeout="600000" channelOpenTimeout="12000" maxConnectionsToServer="1">
        <localCache isEnabled="false" sync="TimeoutBased" ttlValue="300" objectCount="10000"/>
        <clientNotification pollInterval="300" maxQueueLength="10000"/>        
        <securityProperties mode="None" protectionLevel="None" />
        <transportProperties connectionBufferSize="131072" maxBufferPoolSize="568435456" maxBufferSize="183886080" maxOutputDelay="2" channelInitializationTimeout="60000" />  </dataCacheClient>