I need to parse a huge xml feed containing games data and download all the games through the url node of each repeating item to my server.
I have no problem parsing the xml feed, I need to know the best way to download the files from their remote site to my server. Also bear in mind that the feed contains several thousands of items.
I solved it using the php file_get_contents()/file_put_contents() which is very easy to use and works perfectly. The only problem I have now is that if I try to grab around 50+ games at once, sometimes the server returns an error and I have to request less number say 25 games to be able to get them without problem. Anyway I can ask the server to get me huge amount say 500-1000 game without throwing an error?