Search code examples
performancedownloadgothrottlingrate

How to limit download speed with Go?


I'm currently developing a download server in Go. I need to limit the download speed of users to 100KB/s.

This was my code:

func serveFile(w http.ResponseWriter, r *http.Request) {
    fileID := r.URL.Query().Get("fileID")
    if len(fileID) != 0 {
        w.Header().Set("Content-Disposition", "attachment; filename=filename.txt")
        w.Header().Set("Content-Type", r.Header.Get("Content-Type"))
        w.Header().Set("Content-Length", r.Header.Get("Content-Length"))

        file, err := os.Open(fmt.Sprintf("../../bin/files/test.txt"))
        defer file.Close()
        if err != nil {
            http.NotFound(w, r)
            return
        }
        io.Copy(w, file)
    } else {
        io.WriteString(w, "Invalid request.")
    }
}

Then I found a package on github and my code became the following:

func serveFile(w http.ResponseWriter, r *http.Request) {
    fileID := r.URL.Query().Get("fileID")
    if len(fileID) != 0 {
        w.Header().Set("Content-Disposition", "attachment; filename=Wiki.png")
        w.Header().Set("Content-Type", r.Header.Get("Content-Type"))
        w.Header().Set("Content-Length", r.Header.Get("Content-Length"))

        file, err := os.Open(fmt.Sprintf("../../bin/files/test.txt"))
        defer file.Close()
        if err != nil {
            http.NotFound(w, r)
            return
        }
        bucket := ratelimit.NewBucketWithRate(100*1024, 100*1024)
        reader := bufio.NewReader(file)
        io.Copy(w, ratelimit.Reader(reader, bucket))
    } else {
        io.WriteString(w, "Invalid request.")
    }
}

But I'm getting this error:

Corrupted Content Error

The page you are trying to view cannot be shown because an error in the data transmission was detected.

Here's my code on the Go playground: http://play.golang.org/p/ulgXQl4eQO


Solution

  • I'm not seeing the error, but I did notice some issues with the code. For this:

    w.Header().Set("Content-Type", r.Header.Get("Content-Type"))
    

    You should use the mime package's:

    func TypeByExtension(ext string) string
    

    To determine the content type. (if you end up with the empty string default to application/octet-stream)

    For:

    w.Header().Set("Content-Length", r.Header.Get("Content-Length"))
    

    You need to get the content length from the file itself. By using the request content length, for a GET this basically ends up as a no-op, but for a POST you're sending back the wrong length, which might explain the error you're seeing. After you open the file, do this:

    fi, err := file.Stat()
    if err != nil {
        http.Error(w, err.Error(), 500)
        return
    }
    w.Header().Set("Content-Length", fmt.Sprint(fi.Size()))
    

    One final thing, when you open the file, if there's an error, you don't need to close the file handle. Do it like this instead:

    file, err := os.Open(...)
    if err != nil {
        http.NotFound(w, r)
        return
    }
    defer file.Close()