I have a .NET Core app which uses up to ~150MB on Windows, but ~1-2GB on Linux (not VM size, but actual private bytes). It calls Azure HTTP APIs in a loop to retrieve stats.
I'd like to diagnose the problem on Linux, but I don't know the tools.
I have a dump via gdump
, and have been following https://codeblog.dotsandbrackets.com/net-core-memory-linux/ . In the case below, it was using just under a gig.
I can get to this point:
(lldb) sos DumpHeap -stat -min 10000
Statistics:
MT Count TotalSize Class Name
00007f88ebedf438 2 131120 Microsoft.EntityFrameworkCore.ChangeTracking.Internal.InternalEntityEntry[]
00007f88ebed5438 1 202080 System.Collections.Generic.Dictionary`2+Entry[[System.String, System.Private.CoreLib],[Microsoft.EntityFrameworkCore.ChangeTracking.Internal.InternalEntityEntry, Microsoft.EntityFrameworkCore]][]
00007f88ebec6328 1 202080 System.Collections.Generic.Dictionary`2+Entry[[System.Object, System.Private.CoreLib],[Microsoft.EntityFrameworkCore.ChangeTracking.Internal.InternalEntityEntry, Microsoft.EntityFrameworkCore]][]
00007f88e8c92660 4 432604 System.Byte[]
00007f88e8cdf228 4 865120 System.String
0000000001e2d2c0 36 6008920 Free
(without the -min
option, I get too much output to read/do anything with)
The ~600MB free is suspicious, but I don't know what it means, how to remedy, or any other SOS commands to try next. The SOS docs (https://github.com/dotnet/coreclr/blob/master/src/ToolBox/SOS/Strike/sosdocs.txt) look great, but my level of understanding isn't high enough to put together a diagnostic process here yet.
(Some context: I'm running on Kubernetes, have a RAM request and limit set of ~2GB, but this process keeps hitting it. As I say, on Windows it uses way less RAM. Weird.)
Edit: running without -min
does thankfully end with the biggest memory users it seems. So here's the end of it:
...
00007f88ebec5de0 4727 340344 Microsoft.EntityFrameworkCore.ChangeTracking.Internal.InternalClrEntityEntry
00007f88e8c91d00 6484 347528 System.Int32[]
00007f88e9e9e0b8 9434 377360 UNKNOWN
00007f88ea4222d8 4836 386880 Microsoft.Azure.Management.Monitor.Fluent.Models.MetricDefinition
00007f88ebe47f28 9672 464256 Microsoft.Azure.Management.Monitor.Fluent.Models.MetricAvailability
00007f88e9e9f110 9433 528248 UNKNOWN
00007f88e9e9e2f8 9434 528304 UNKNOWN
00007f88ebe42a78 4836 580320 Microsoft.Azure.Management.Monitor.Fluent.Models.MetricDefinitionImpl
00007f88ea423158 3837 1074360 azure2elasticstack.Program+<ProcessMetricAsync>d__10
00007f88e8c92660 1832 2315924 System.Byte[]
00007f88e8cdf228 44713 10688048 System.String
0000000001e2d2c0 22762 22762768 Free
Total 289009 objects
(lldb)
Repo here: https://github.com/kierenj/proto-azure2elasticstack/tree/master
The memory usage is way down and into a respectable range for .NET Core 2.1 (netcoreapp2.1).