"Win32Exception Not enough storage is available to process this command" error when run as a windows service

Sometimes I get the error

Win32Exception Not enough storage is available to process this command

after the application has been running for around a month or so.

This corresponds to the system error

ERROR_NOT_ENOUGH_MEMORY (8)

Usually, it's run as a Windows service, and changing the user it's logged on as still makes the error occur. However, if I run the application by double clicking on it, it works fine. The only way it can run as a Windows service again is if the server is restarted.

The error appears in the logs that the application writes to. The full error is:

System.ComponentModel.Win32Exception: Not enough storage is available to process this command
   at System.Windows.Forms.NativeWindow.WindowClass.RegisterClass()
   at System.Windows.Forms.NativeWindow.WindowClass.Create(String className, Int32 classStyle)
   at System.Windows.Forms.NativeWindow.CreateHandle(CreateParams cp)
   at System.Windows.Forms.Control.CreateHandle()
   at System.Windows.Forms.Application.MarshalingControl..ctor()
   at System.Windows.Forms.Application.ThreadContext.get_MarshalingControl()
   at System.Windows.Forms.WindowsFormsSynchronizationContext..ctor()
   at System.Windows.Forms.WindowsFormsSynchronizationContext.InstallIfNeeded()
   at System.Windows.Forms.Control..ctor(Boolean autoInstallSyncContext)
   at System.Windows.Forms.ScrollableControl..ctor()
   at System.Windows.Forms.ContainerControl..ctor()
   at System.Windows.Forms.Form..ctor()
   at LicensingModule.LicenseKeyValidator..ctor(String Name, String Path)
   at MIDICOMExporter.Program.Main(String[] args)

I've seen other answers say it's related to a memory leak on the server, but if the application can still run when double clicked on, does that still mean there may be a memory leak?

Also, this is running on Windows Server 2012.

Solution

Yes, the problem might be related to a memory leak on the server. Maybe the desktop heap is depleted. The desktop heap for "non-interactive" window sessions is different from the desktop heap for real (interactive) desktop sessions. This explains why double clicking the application is still possible.

Try finding the registry key \System\CurrentControlSet\Control\Session Manager\SubSystem and increasing the 3rd parameter (zzz) of SharedSection=xxx,yyy,zzz as described in MSKB 126962 (for yyy).

The third SharedSection value (512) is the size of the desktop heap for each desktop that is associated with a "non-interactive" window station.

If this fixes your problem, heap depletion, caused for instance by leaking resources, is likely to be the source of your problem. The size of the "non-interactive" desktop heap is by default much smaller than the standard heap.