What is the difference between limitedParallelism vs a fixed thread pool dispatcher?

Question

I am trying to use Kotlin coroutines to perform multiple HTTP calls concurrently, rather than one at a time, but I would like to avoid making all of the calls concurrently, to avoid rate limiting by the external API.

If I simply launch a coroutine for each request, they all are sent near instantly. So I looked into the limitedParallelism function, which sounds very close to what I need, and some stack overflow answers suggest is the recommended solution. Older answers to the same question suggested using newFixedThreadPoolContext.

The documentation for that function mentioned limitedParallelism as a preferred alternative "if you do not need a separate thread pool":

If you do not need a separate thread-pool, but only have to limit effective parallelism of the dispatcher, it is recommended to use CoroutineDispatcher.limitedParallelism instead.

However, when I write my code to use limitedParallelism, it does not reduce the number of concurrent calls, compared to newFixedThreadPoolContext which does.

In the example below, I replace my network calls with Thread.sleep, which does not change the behavior.

// method 1
val fixedThreadPoolContext = newFixedThreadPoolContext(2)

// method 2
val limitedParallelismContext = Dispatchers.IO.limitedParallelism(2)

runBlocking {
  val jobs = (1..1000).map {
    // swap out the dispatcher here
    launch(limitedParallelismContext) {
      println("started $it")
      Thread.sleep(1000)
      println("    finished $it")
    }
  }
  jobs.joinAll()
}

The behavior for fixedThreadPoolContext is as expected, no more than 2 of the coroutines runs at a time, and the total time to finish is several minutes (1000 times one second each, divided by two at a time, roughly 500 seconds).

However, for limitedParallelismContext, all "started #" lines print immediately, and one second later, all "finished #" lines print and the program completes in just over 1 total second.

Why does limitedParallelism not have the same effect as using a separate thread pool? What does it accomplish?

Answer 1

I modified your code slightly so that every coroutine takes 200ms to complete and it prints the time when it is completed. Then I pasted it to play.kotlinlang.org to check:

/**
 * You can edit, run, and share this code.
 * play.kotlinlang.org
 */
import kotlinx.coroutines.*

fun main() {
    // method 1
    val fixedThreadPoolContext = newFixedThreadPoolContext(2, "Pool")

    // method 2
    val limitedParallelismContext = Dispatchers.IO.limitedParallelism(2)

    runBlocking {
      val jobs = (1..10).map {
        // swap out the dispatcher here
        launch(limitedParallelismContext) {
          println("it at ${System.currentTimeMillis()}")
          Thread.sleep(200)
        }
      }
      jobs.joinAll()
    }
}

And there using kotlin 1.6.21 the result is as expected:

it at 1652887163155
it at 1652887163157
it at 1652887163358
it at 1652887163358
it at 1652887163559
it at 1652887163559
it at 1652887163759
it at 1652887163759
it at 1652887163959
it at 1652887163959

Only 2 coroutines are executed at a time.