Search code examples
scalatestingscalatestscalacheckproperty-based-testing

Sharing elements between generated objects in ScalaCheck using nested forAll


Started coding in Scala fairly recently and I tried to write some property based test-cases. Here, I am trying to generate raw data which mimics the system I am testing. The goal is to first generate base elements (ctrl and idz), then use those values to generate two classes (A1 and B1) and finally check their properties. I first tried the following -

import org.scalatest._
import prop._
import scala.collection.immutable._
import org.scalacheck.{Gen, Arbitrary}

case class A(
    controller: String,
    id: Double,
    x: Double
)

case class B(
    controller: String,
    id: Double,
    y: Double
)

object BaseGenerators {
    val ctrl = Gen.const("ABC")
    val idz = Arbitrary.arbitrary[Double]
}

trait Generators {
    val obj = BaseGenerators

    val A1 = for {
        controller <- obj.ctrl
        id <- obj.idz
        x <- Arbitrary.arbitrary[Double]
    } yield A(controller, id, x)

    val B1 = for {
        controller <- obj.ctrl
        id <- obj.idz
        y <- Arbitrary.arbitrary[Double]
    } yield B(controller, id, y)

}

class Something extends PropSpec with PropertyChecks with Matchers with Generators{

    property("Controllers are equal") {
        forAll(A1, B1) {
            (a:A,b:B) => 
                a.controller should be (b.controller)
        }
    }

    property("IDs are equal") {
        forAll(A1, B1) {
            (a:A,b:B) => 
                a.id should be (b.id)
        }
    }

}

Running sbt test in terminal gave me the following -

[info] Something:
[info] - Controllers are equal
[info] - IDs are equal *** FAILED ***
[info]   TestFailedException was thrown during property evaluation.
[info]     Message: 1.1794559135007427E-271 was not equal to 7.871712821709093E212
[info]     Location: (testnew.scala:52)
[info]     Occurred when passed generated values (
[info]       arg0 = A(ABC,1.1794559135007427E-271,-1.6982696700585273E-23),
[info]       arg1 = B(ABC,7.871712821709093E212,-8.820696498155311E234)
[info]     )

Now it's easy to see why the second property failed. Because every time I yield A1 and B1 I'm yielding a different value for id and not for ctrl because it is a constant. The following is my second approach wherein, I create nested for-yield to try and accomplish my goal -

case class Popo(
    controller: String,
    id: Double,
    someA: Gen[A],
    someB: Gen[B]
)

trait Generators {
    val obj = for {
        ctrl <- Gen.alphaStr
        idz <- Arbitrary.arbitrary[Double]
        val someA = for {
            x <- Arbitrary.arbitrary[Double]
        } yield A(ctrl, idz, someA)
        val someB = for {
            y <- Arbitrary.arbitrary[Double]
        } yield B(ctrl, idz, y)
    } yield Popo(ctrl, idz, x, someB)
}

class Something extends PropSpec with PropertyChecks with Matchers with Generators{

    property("Controllers are equal") {
        forAll(obj) {
            (x: Popo) => 
            forAll(x.someA, x.someB) {
                (a:A,b:B) => 
                    a.controller should be (b.controller)
            }
        }
    }

    property("IDs are equal") {
        forAll(obj) {
            (x: Popo) =>
            forAll(x.someA, x.someB) {
                (a:A,b:B) => 
                    a.id should be (b.id)
            }
        }
    }
}

Running sbt test in the second approach tells me that all tests pass.

[info] Something:
[info] - Controllers are equal
[info] - IDs are equal
[info] ScalaTest
[info] Run completed in 335 milliseconds.
[info] Total number of tests run: 2
[info] Suites: completed 1, aborted 0
[info] Tests: succeeded 2, failed 0, canceled 0, ignored 0, pending 0
[info] All tests passed.

Is there a better/alternative way to reproduce my desired results? Nesting forAll seems rather clunky to me. If I were to have R -> S -> ... V -> W in my dependency graph for objects sharing elements then I'll have to create as many nested forAll.


Solution

  • I'm going to give an answer in just Scalacheck. I know Scalatest is popular, but I find its inclusion in a question about Scalacheck to be distracting, especially when there's no reason the example couldn't be written without it.

    It seems you want to test A and B, but they share information. One way to represent that dependency would be the Popo class you wrote. It contains both the shared information, and generated values of A and B. Another option is generating the shared values between A and B in a class.

    The simplest solution is generating A and B in pairs (a tuple of two). Unfortunately, there are some tricks to make it work. You will need to use case keyword in the forAll property. You can't give evidence for an implicit value for Arbitrary tuples, so you have to specify the generator for the tuples explicitly in the forAll.

    import org.scalacheck.Gen
    import org.scalacheck.Arbitrary
    import org.scalacheck.Prop
    import org.scalacheck.Prop.AnyOperators
    import org.scalacheck.Properties
    
    case class A(
      controller: String,
      id: Double,
      x: Double
    )
    
    case class B(
      controller: String,
      id: Double,
      y: Double
    )
    
    object BaseGenerators {
      val ctrl = Gen.const("ABC")
      val idz = Arbitrary.arbitrary[Double]
    }
    
    object Generators {
      val obj = BaseGenerators
    
      val genAB: Gen[(A,B)] = for {
        controller <- obj.ctrl
        id <- obj.idz
        x <- Arbitrary.arbitrary[Double]
        y <- Arbitrary.arbitrary[Double]
        val a = A(controller, id, x)
        val b = B(controller, id, y)
      } yield (a, b)                                         // !
    }
    
    class Something extends Properties("Something") {
    
      property("Controllers and IDs are equal") = {
        Prop.forAll(Generators.genAB) { case (a: A, b: B) => // !
          a.controller ?= b.controller && a.id ?= b.id
        }
      }
    }
    

    With respect to your broader question about having objects that share information, you could represent it by writing your generators with function arguments. However, it would still require nested forAll generators.

    object Generators {
      val obj = BaseGenerators
    
      val genA = for {
        controller <- obj.ctrl
        id <- obj.idz
        x <- Arbitrary.arbitrary[Double]
      } yield A(controller, id, x)
    
      def genB(a: A) = for {                                 // !
        y <- Arbitrary.arbitrary[Double]
      } yield B(a.controller, a.id, y)
    }
    
    class Something extends Properties("Something") {
    
      implicit val arbA: Arbitrary[A] = Arbitrary {
        Generators.genA
      }
    
      property("Controllers and IDs are equal") = {
        Prop.forAll { a: A =>                                // !
          Prop.forAll(Generators.genB(a)) { b: B =>          // !
            (a.controller ?= b.controller) && (a.id ?= b.id)
          }
        }
      }
    }