Search code examples
scalanested-loops

Scala - Alternative to nested for loops while writing out


To introduce my situation, I'm trying to generate multiple log files, one for every hour, that will have a random number of timestamps (lines). While writing the code, I decided to write-out and name the filenames based on the hour (log0 ... log23). This is so I can test a Spark Streaming job that relies on logs being separated by the hour. However, couldn't figure out any other way to do this other than having nested for loops.

In the Scala spirit of avoiding nested for loops and to making code easier to read, I'm looking to see if there there is a way to rewrite the following sample code with identical functionality:

import scala.reflect.io.File

val hours = 24
val max_iterations = 100
val string_builder = scala.collection.mutable.StringBuilder.newBuilder
val rand = scala.util.Random

for (hour <- 0 until hours) {
    for (iter <- 1 to rand.nextInt(max_iterations)) {
        string_builder.append(s"{datetime=$hour:$minute:$second}\n")
    }
    File(s"log$hour.txt").createFile(false).writeAll(string_builder.toString)
    string_builder.clear
}

Edit: Just for clarification, this differs from a standard multiple file write out, as the hours need to match the file name.


Solution

  • A simple solution would be to use for-comprehension:

    for {
      hour <- 0 until hours
      iter <- 1 to rand.nextInt(max_iterations)
    } yield {
      File(s"log$hour.txt").appendAll(s"{datetime=$hour:${iter%60}:00}\n")
    }
    

    It has a downside of recreating the File handler over and over, so performance might be an issue, but if this code is just used to create some test data once, this shouldn't be a concern.

    An alternative would be to call foreach on the sequence of hours (and then of iters) directly:

    (0 until hours).foreach(hour => {
      val f = File(s"log$hour.txt")
      val lines = (1 to rand.nextInt(max_iterations)).map(iter => s"{datetime=$hour:${iter%60}:00}\n")
      f.writeAll(lines: _*)
    })