Search code examples
phpfile-upload

Can PHP load file data into $_POST?

I have a device that submits ~20,000 small 16-byte binary files (data from RFID tags) as multipart/form-data to a URL of my choice.

------WebKitFormBoundaryP1dTbAtca8AXmaAM
Content-Disposition: form-data; name="400000001"; filename="400000001.bin"
Content-Type: application/vnd.ply-badge

⚉⟗⢑☭⥢↺⡠⥔
------WebKitFormBoundaryP1dTbAtca8AXmaAM
Content-Disposition: form-data; name="400000002"; filename="400000002.bin"
Content-Type: application/vnd.ply-badge

⨧ⶈⰡⰑ⸈ⱬ⋪⣫

I want to insert this data into a SQL database. Right now the back end for the rest of this application is PHP, so I would like to do this in PHP as well. Right now I am getting

PHP Warning: Maximum number of allowable file uploads has been exceeded in Unknown on line 0

I'm aware I can raise the limit in the php.ini file, and if I do it works, but it writes all 20,000 files to disk. Since this is less than 400K of actual data (maybe a few MB once you include metadata) I would like to just load it into memory and skip the filesystem all together. Does PHP have an option to load values like this into $_POST rather than writing them to disk and using $_FILES? If not, is there some other mechanism that will let me parse this data without writing everything to disk?

Unfortunately this is on a Windows server, so in-memory filesystems are... complicated.

Solution

  • I guess this could work, if you do two things:

    a) prevent PHP from automatically parsing this request, and

    b) parsing the request body yourself afterwards.

    PHP automatically parses the request because of the Content-Type: multipart/form-data request header - so that would have to be manipulated, before the request reaches PHP.
    With Apache as the web server, this can be done using the RequestHeader directive. (With IIS or nginx, you'd have to figure out what equivalent they offer.)

    Would make most sense to set it to just text/plain, I think. That should prevent PHP from doing any parsing on its own, so $_POST and $_FILES won't be populated either.

    Now you can read the request body via php://input, and parse it yourself. Some hints on the latter can be found under Manually parse raw multipart/form-data data with PHP, there's also libraries out there on GitHub etc.

    Of course at this point you'd have to make sure that you choose a parser implementation, that does not create temporary files again, but provides you access to the data in form of an array or object.