In one of my applications, I need to download about 50,000 XML documents, most averaging around 5kb in size. I need to find the most efficient way bandwidth wise to transfer all of it over http, and the XML is gzipped. Which is the most efficient way to download it, using the least bandwidth. Downloading each XML file individually, downloading them in sets, or are they practically the same? I have no idea as I don't really know how GZIP would effect this.
Thanks
If you are using HTTP, the primary factor for perceived connection speed is the number of TCP connections established.
Serving it as one master file will help (for only one connection would be established). If the server, supports Accept-Ranges (in other words, the resumption of file downloads) feature, then you can use a 'smarter' client to download the master file in chunks over several connections, and regroup them.
But, if that cannot be done, use a HTTP client that supports HTTP 1.1 Keep-alive. That way, the connection would be re-used over several files.