Search code examples
windowsperliounbuffered

Unbuffered IO in perl


I have a Perl application which writes logs to a file using open and print calls.

open (FH, "d:\\temp.txt");
print FH "Some log";
close (FH);

However during an abrupt shutdown of the machine, the logs are not persisted to the file. So after searching at several places, two options were suggested for doing unbuffered IO (i.e writing the text to the disk instead of maintaining it in cache and then flushing it):

  1. sysopen, syswrite
  2. $| = 1;

I have tried both these options and it just doesn't work. Any write that I do seconds before the abnormal shutdown gets lost.

Is there any way that I can almost deterministically accomplish unbuffered IO in Perl? I am running Windows 7 64-bit with Perl 5.8.3.

EDIT: I searched for how to have windows perform unbuffered IO and this is how it can be done! Call

  1. CreateFile with FILE_FLAG_NO_BUFFERING for the dwFlagsAndAttributes parameter. However, this has memory alignment issues to consider (i.e File Access buffers should be sector aligned; An application determine the sector size by calling GetDiskFreeSpace)
  2. Use WriteFile to write data to the file. This write would be unbuffered and instead of going to the cache, it straightaway goes to the disk.
  3. Finally, call FlushFileBuffers to flush the metadata associated with the files.

Could someone please assist with the Win32 APIs from Perl for these 3 calls.


Solution

  • How about this?

    use strict;
    use warnings;
    
    use IO::Handle     qw( );  # For autoflush.
    use Symbol         qw( gensym );
    use Win32API::File qw( CloseHandle CreateFile GetOsFHandle OsFHandleOpen GENERIC_WRITE OPEN_ALWAYS FILE_FLAG_WRITE_THROUGH );
    use Win32::API     qw( );
    
    use constant WIN32API_FILE_NULL => [];
    
    sub open_log_handle {
        my ($qfn) = @_;
    
        my $handle;
        if (!($handle = CreateFile(
            $qfn,
            GENERIC_WRITE,
            0,                        # Exclusive lock.
            WIN32API_FILE_NULL,       # No security descriptor.
            OPEN_ALWAYS,              # Create if doesn't exist.
            FILE_FLAG_WRITE_THROUGH,  # Flush writes immediately.
            WIN32API_FILE_NULL,       # No prototype.
        ))) {
            return undef;
        }
    
        my $fh = gensym();
        if (!OsFHandleOpen($fh, $handle, 'wa')) {
            my $e = $^E;
            CloseHandle($handle);
            $^E = $e;
            return undef;
        }
    
        $fh->autoflush(1);
    
        return $fh;
    }
    
    sub close_log_handle {
        my ($fh) = @_;
    
        my $handle = GetOsFHandle($fh)
            or return undef;
    
        if (!FlushFileBuffers($handle)) {
            my $e = $^E;
            close($fh);
            $^E = $e;
            return undef;
        }
    
        return close($fh);
    }
    
    my $FlushFileBuffers = Win32::API->new('kernel32.dll', 'FlushFileBuffers', 'N', 'N')
        or die $^E;
    
    sub FlushFileBuffers {
        my ($handle) = @_;
        return $FlushFileBuffers->Call($handle);
    }
    
    {
        my $fh = open_log_handle('log.txt')
            or die $^E;
    
        print($fh "log!\n")
            or die $^E;
    
        close_log_handle($fh)
            or die $^E;
    }