How do I write a huge bytearray to file progressively and avoid that dreaded memory error

Question

I'm working on an Air project that downloads huge (zip) files, then unzips them and finally deletes the original .zip file it downloaded.

Everything's running smoothly except when I want to write the unzipped file to disk.

Trouble is, I keep running into the issue of this little monster of an error.

Error: Error #1000: The system is out of memory.

Here's the part of the code that is giving me grief. This works perfectly when the file is small (tested on a 3 MB file) but as soon as I try one of my large files (458 MB), I get the error.

private function unzipFile()
    {
        var pulledFile:FZipFile;
        var index:int = 0;
        var chunkSize:int = 1000000;

        Console.log("Unzipping file...");
        var zip:FZip = new FZip();
        zip.addEventListener(Event.COMPLETE, onUnzipComplete);
        var fileStream:FileStream = new FileStream();

        zip.load(new URLRequest(urlToUnzip));

        function onUnzipComplete(e:Event)
        {
            pulledFile = zip.getFileAt(0);
            var file:File = File.documentsDirectory.resolvePath(pulledFile.filename);
            fileStream.openAsync(file, FileMode.WRITE);
            writeChunk();
        }

        function writeChunk()
        {
            var bytesToGet:int = pulledFile.content.bytesAvailable;
            if (bytesToGet > chunkSize)
                bytesToGet = chunkSize;

            Console.log("Writing chunk " + (int((pulledFile.content.length - pulledFile.content.bytesAvailable) / chunkSize) + 1) + " of " + (int(pulledFile.content.length / chunkSize) + 1));

            var fileData:ByteArray = new ByteArray();
            pulledFile.content.readBytes(fileData, 0, bytesToGet);

            fileStream.writeBytes(fileData, 0, fileData.length);

            if (index < pulledFile.content.bytesAvailable)
            {
                writeChunk();
                index += bytesToGet;
            }
            else
            {
                fileStream.close();
                Console.log("Unzipping complete");
            }
        }
    }

The code crashes specifically on the line

var bytesToGet:int = pulledFile.content.bytesAvailable;

How do I still progressively write content to a file without knowing how many bytes are available to me? If I don't have access to the bytesAvailable property, how do I know I'm done writing?

I'm using the FZip library for decompression.

Answer 1

This was tough, but I figured it out. The trick is to use the "serialize" method of the FZip and avoid touching the properties of the content ByteArrayaltogether.

Here's the final code that works.

private function unzipFile()
    {
        var pulledFile:FZipFile;

        Console.log("Decompressing file...");
        var zip:FZip = new FZip();
        zip.addEventListener(Event.COMPLETE, onUnzipComplete);

        var fileStream:FileStream = new FileStream();

        zip.load(new URLRequest(urlToUnzip));

        function onUnzipComplete(e:Event)
        {
            pulledFile = zip.getFileAt(0);
            var file:File = File.documentsDirectory.resolvePath("Easywatch/" + pulledFile.filename);
            fileStream.open(file, FileMode.WRITE);
            zip.serialize(fileStream, false);
            fileStream.close();

            Console.log("Decompression complete");
        }
    }