In Perl, I generate a huge read-only data-structure once, then fork()
.
This is to take advantage of COW on RSS pages when forking. It works really well, but when a child process exits, it allocates all the RAM from itelf just prior dying.
Is there a way to avoid this useless allocation ?
Here is sample Perl code that shows the issue.
#! /usr/bin/perl
my $a = [];
# Allocate 100 MiB
for my $i (1 .. 100000) {
push @$a, "x" x 1024;
}
# Fork 10 other process
for my $j (1 .. 10) {
last unless fork();
}
# Sleep for a while to be able to see the RSS
sleep(5);
In the sample vmstat
output, we can see that it first allocates only 100MiB, then after the 1rst sleep it allocates the whole for a short while, and then releases all of it.
procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
r b swpd free buff cache si so bi bo in cs us sy id wa st
0 0 0 1329660 80596 86936 0 0 21 18 160 25 0 0 100 0 0
1 0 0 1328048 80596 86936 0 0 0 0 1013 44 0 0 100 0 0
0 0 0 1223888 80596 86936 0 0 0 0 1028 76 11 5 84 0 0
0 0 0 1223888 80596 86936 0 0 0 0 1010 40 0 0 100 0 0
0 0 0 1223888 80596 86936 0 0 0 0 1026 54 0 0 100 0 0
0 0 0 1223888 80596 86936 0 0 0 0 1006 39 0 0 100 0 0
13 0 0 741156 80596 86936 0 0 0 0 1012 66 13 58 28 0 0
0 0 0 1329288 80596 86936 0 0 0 0 1032 60 0 0 100 0 0
Note: it seems it isn't a Perl version specific issue. As I tested 5.8.8, 5.10.1 & 5.14.2 and they all do exhibit this behavior.
Update:
As @choroba asked in comments, I also tried to undef
the data-structure, but it seems that it triggers the memory-touching as the RAM is then allocated.
You can add the following snippet at the end of the first script.
# Unallocate $a
undef $a;
# Sleep for a while to be able to see the RSS
sleep(5);
Actually, as I found out myself, this behavior is a feature, and the answer lies in the Perl doc:
The
exit()
function does not always exit immediately. Likewise any object destructors that need to be called are called before the real exit. If this is a problem, you can callPOSIX::_exit($status)
to avoidEND
and destructor processing.
And indeed, adding it at the end of the original code sample does avoid the behavior.
# XXX - To be added just before ending the process
# Use POSIX::_exit($status) to end without allocating copy-on-write RAM
use POSIX;
POSIX::_exit(0);
Note: for this to work, the child has to exit also before the data-structure goes out of scope.