KTOR OOME while sending large file

Question

I'm trying to send a file which is saved locally (90mb) in this way from Ktor documentation.

val client = HttpClient(CIO)

val response: HttpResponse = client.post("http://localhost:8080/upload") {
    setBody(MultiPartFormDataContent(
        formData {
            append("description", "Ktor logo")
            append("image", File("ktor_logo.png").readBytes(), Headers.build {
                append(HttpHeaders.ContentType, "image/png")
                append(HttpHeaders.ContentDisposition, "filename=\"ktor_logo.png\"")
            })
        },
        boundary = "WebAppBoundary"
    )
    )
    onUpload { bytesSentTotal, contentLength ->
        println("Sent $bytesSentTotal bytes from $contentLength")
    }
}

Than got this error

FATAL EXCEPTION: DefaultDispatcher-worker-2
    PID: 8152
    java.lang.OutOfMemoryError: Failed to allocate a 4112 byte allocation with 1424 free bytes and 1424B until OOM, max allowed footprint 100663296, growth limit 100663296
        at java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:54)
        at java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:49)
        at java.nio.ByteBuffer.allocate(ByteBuffer.java:281)
        at io.ktor.utils.io.bits.DefaultAllocator.alloc-gFv-Zug(MemoryFactoryJvm.kt:41)
        at io.ktor.utils.io.core.DefaultBufferPool.produceInstance(BufferFactory.kt:58)
        at io.ktor.utils.io.core.DefaultBufferPool.produceInstance(BufferFactory.kt:51)
        at io.ktor.utils.io.pool.DefaultPool.borrow(DefaultPool.kt:51)
        at io.ktor.utils.io.core.internal.ChunkBuffer$Companion$Pool$1.borrow(ChunkBuffer.kt:133)
        at io.ktor.utils.io.core.internal.ChunkBuffer$Companion$Pool$1.borrow(ChunkBuffer.kt:128)
        at io.ktor.utils.io.core.Output.appendNewChunk(Output.kt:102)
        at io.ktor.utils.io.core.Output.prepareWriteHead(Output.kt:354)
        at io.ktor.utils.io.core.internal.UnsafeKt.prepareWriteHead(Unsafe.kt:57)
        at io.ktor.utils.io.ByteBufferChannel.readRemainingSuspend(ByteBufferChannel.kt:3689)
        at io.ktor.utils.io.ByteBufferChannel.access$readRemainingSuspend(ByteBufferChannel.kt:24)
        at io.ktor.utils.io.ByteBufferChannel$readRemainingSuspend$1.invokeSuspend(Unknown Source:16)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
        at kotlinx.coroutines.internal.LimitedDispatcher.run(LimitedDispatcher.kt:42)
        at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:95)
        at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:570)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:677)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:664)
        Suppressed: kotlinx.coroutines.DiagnosticCoroutineContextException: [StandaloneCoroutine{Cancelling}@29cc6a1, Dispatchers.IO]

As I understand that's because of readBytes().

Is there any way to send large files partially in form-data with Ktor and avoid this error.

UPD 1

Also tried with, same error

appendInput(
        "MY_KEY",
        Headers.build {
            headers.forEach { h ->
                this.append(h.first, h.second) // Content-Disposition and Content-Type here
            }
        },
        file.length()
    ){
        file.inputStream().asInput()
    }

UPD 2

Client creation part, take it into account:

HttpClient(OkHttp) {
    install(ContentNegotiation) {
        json(Json {
            prettyPrint = true
            isLenient = true
            ignoreUnknownKeys = true
        })
    }

    install(Resources)
    install(Logging) {
        logger = Logger.DEFAULT
        level = LogLevel.BODY
    }    

    engine {
        config {
            followRedirects(true)
            connectTimeout(100_000, TimeUnit.MILLISECONDS)
            readTimeout(100_000, TimeUnit.MILLISECONDS)
        }
    }
}

Answer 1

You can use the InputProvider to read a file by chunks:

formData {
    // ...
    val file = File("ktor_logo.png")
    append(
        "image",
        InputProvider(file.length()) { file.inputStream().asInput() },
        Headers.build {
            append(HttpHeaders.ContentType, "image/png")
            append(HttpHeaders.ContentDisposition, "filename=\"ktor_logo.png\"")
        }
    )
},