Publishing many items uses a lot of memory

Question

I have a problem similar to this, although hopefully I've narrowed it down a little.

I am currently testing the publishing part of my Tridion 2011 installation (Conent Manager and Content Delivery on separate servers). When I try to publish many pages at once (1000 in my case) the process dllhst3g.exe *32 starts to acquire a large amount of memory. This is understandable as there is a lot of work to do but it never gives it back. This is causing the content manager to run slowly and eventually crash.

I was getting out of memory exceptions being thrown by the Tridion Content Manager when the dllhst3g.exe *32 process had around 3.6GB of memory (I assume this is due to it being a 32 bit process). To prevent these I limited the SDL Tridion Content Manager COM+ Application to an arbitrary 500Mb per process which gets forcibly terminated 15 minutes after the limit is reached. This has stopped the out of memory exceptions for now but I still have performance issues and a potential to run out of memory if more than my servers 8GB is allocated in the 15 minute window. More about this here

I have ruled out the underlying PageTemplate code as the associated page template has no code associated. These pages are blank.

The issue also seems to be much more prominent when using the Core Service API. Code I am using is

using (var client = new Tridion2011CoreService.CoreServiceClient())
{
    foreach (var id in ids) // ids is a collection of 500 page ids
    {
        // publishing to staging and live
        var targets = new string[] { "tcm:0-7-65538", "tcm:0-8-65538" };

        var publishInstructionData = new PublishInstructionData();
        publishInstructionData.ResolveInstruction = new ResolveInstructionData();
        publishInstructionData.RenderInstruction = new RenderInstructionData();

        var readOptions = new ReadOptions();

        client.Publish(new string[] { id }, publishInstructionData, targets, PublishPriority.Normal, readOptions);
    }
}

(I realise I could send all the ids through with one call to publish but then I hit a message limit error and as far as I am aware the result of multiple calls and single calls is effectively the same)

Any ideas?

(let me know if I have missed out any details and I'll update the question accordingly)

Forgot to add. The Content Manager is installed on a virtualised Windows Server 2008 with 4 CPUs, 8GB RAM and 50GB disk space. The Content Delivery is installed on a separate server with the same specs.

Answer 1

After contacting SDL customer support I was told that the only solution is to recycle the dllhost process when it takes up a pre-determined amount of memory.

To do this for SDL Tridion Content Manager 2011 running on Windows Server 2008

1. Open Component Services (either search for it or Control Panel -> Administrative Tools)
2. Expand the tree Component Services -> Computers -> My Computer -> COM+ Applications
3. Right click SDL Tridion Content Manager and select Properties
4. Select the Pooling & Recycling tab
   • Set Pool Size to 1
   • Set Lifetime limit to 0
   • Set Memory Limit to 524288 (512 MB) or choose your own limit (I'd go for a value less than 1GB)
   • Set Call Limit to 0
   • Set Activation Limit to 0
5. If you can, restart your system. Otherwise, restart all the Tridion Services (Control Panel -> Administrative Tools -> Services and restart everything that starts with "Tridion")

Maybe the "real" fix is but a patch away...