Streaming string data with F# Suave

Question

With Suave 2.4.0 supporting TransferEncoding.chunked and HttpOutput.writeChunk I have written the below code to stream out data over HTTP.

let sendStrings getStringsFromProducer : WebPart =
    Writers.setStatus HTTP_200 >=>
    TransferEncoding.chunked (fun conn -> socket {
        let refConn = ref conn

        for str in getStringsFromProducer do
            let! (_, conn) = (str |> stringToBytes |> HttpOutput.writeChunk) !refConn
            refConn := conn

        return! HttpOutput.writeChunk [||] !refConn
    }
)

While this works, I question the reliability of using ref and hoping there are better way out there to do the same in a more functional manner. Are there better way to do this? Assuming I cannot change getStringsFromProducer?

Answer 1

I think you cannot avoid all mutation in this case - writing chunks one by one is a fairly imperative operation and iterating over a lazy sequence also requires (mutable) iterator, so there is no way to avoid all mutation. I think your sendStrings function does a nice job at hiding the mutation from the consumer and provides a nice functional API.

You can avoid using ref cells and replace them with local mutable variable, which is a bit safer - because the mutable variable cannot escape the local scope:

TransferEncoding.chunked (fun conn -> socket {
    let mutable conn = conn
    for str in getStringsFromProducer do
        let! _, newConn = HttpOutput.writeChunk (stringToBytes str) conn
        conn <- newConn
    return! HttpOutput.writeChunk [||] conn
}

You could avoid the mutable conn variable by using recursion, but this requires you to work with IEnumerator<'T> rather than using a nice for loop to iterate over the sequence, so I think this is actually less nice than the version using a mutable variable:

TransferEncoding.chunked (fun conn -> socket {
    let en = getStringsFromProducer.GetEnumerator()
    let rec loop conn = socket {
      if en.MoveNext() then 
        let! _, conn = HttpOutput.writeChunk (stringToBytes en.Current) conn
        return! loop conn }
    do! loop conn
    return! HttpOutput.writeChunk [||] conn })