Search code examples
javasocketsnetwork-programmingbandwidth

How do I implement client-side bandwidth throttling for FTP/HTTP?

I am tasked with writing a client-side data download system (on Linux) that uses FTP or HTTP to download terabyte-sized data from external partners to our local site. Our company's network admin tells me that I cannot exceed a certain bandwidth. What is the best way for me to implement such a system? Do existing libraries exist?

I am open to writing my own FTP and HTTP clients (in either C or Java on Linux) but would prefer to stay out of the kernel. I know that I can limit the rate at which my FTP/HTTP client calls a socket read(), but what happens if the server-side calls write() faster than my limit?

Solution

  • You could build another layer on top of an InputStream: In the read method, you can count the bytes so far. If the number of bytes/second exceed a certain limit, let the download thread sleep for a while. TCP's flow control does the rest.