Setting testFrameworks for a specific SBT configuration

Question

I'm trying to integrate ScalaMeter into a separate configuration of our build. I want to run all performance tests in a separate configuration, since they are naturally slow. In Build.scala I have:

  lazy val ItTest = config("it").extend(Test)

  lazy val PerfTest = config("perf").extend(Test)

  val testSettings = Seq(
    testOptions in Test := Seq(Tests.Filter(x => !itFilter(x))),
    testOptions in ItTest := Seq(Tests.Filter(x => itFilter(x))),
    testFrameworks in PerfTest := Seq(new TestFramework("org.scalameter.ScalaMeterFramework")),
    logBuffered in PerfTest := false,
    // testOptions in PerfTest := Seq(Tests.Filter(perfFilter)),
    // needed thanks to http://stackoverflow.com/questions/7898273/how-to-get-logging-working-in-scala-unit-tests-with-testng-slf4s-and-logback
    parallelExecution in Test := false,
    parallelExecution in ItTest := false,
    parallelExecution in PerfTest := false)

 lazy val project1 = project.configs(ItTest, PerfTest).settings(testSettings: _*) // etc

testFrameworks are getting set as expected:

> show *:testFrameworks
[info] project1/*:testFrameworks
[info]  List(TestFramework(WrappedArray(org.scalacheck.ScalaCheckFramework)), TestFramework(WrappedArray(org.specs2.runner.Specs2Framework, org.specs2.runner.SpecsFramework)), TestFramework(WrappedArray(org.specs.runner.SpecsFramework)), TestFramework(WrappedArray(org.scalatest.tools.Framework, org.scalatest.tools.ScalaTestFramework)), TestFramework(WrappedArray(com.novocode.junit.JUnitFramework)))
[info] project2/*:testFrameworks
[info]  List(TestFramework(WrappedArray(org.scalacheck.ScalaCheckFramework)), TestFramework(WrappedArray(org.specs2.runner.Specs2Framework, org.specs2.runner.SpecsFramework)), TestFramework(WrappedArray(org.specs.runner.SpecsFramework)), TestFramework(WrappedArray(org.scalatest.tools.Framework, org.scalatest.tools.ScalaTestFramework)), TestFramework(WrappedArray(com.novocode.junit.JUnitFramework)))
[info] all/*:testFrameworks
[info]  List(TestFramework(WrappedArray(org.scalacheck.ScalaCheckFramework)), TestFramework(WrappedArray(org.specs2.runner.Specs2Framework, org.specs2.runner.SpecsFramework)), TestFramework(WrappedArray(org.specs.runner.SpecsFramework)), TestFramework(WrappedArray(org.scalatest.tools.Framework, org.scalatest.tools.ScalaTestFramework)), TestFramework(WrappedArray(com.novocode.junit.JUnitFramework)))
> show perf:testFrameworks
[info] project1/perf:testFrameworks
[info]  List(TestFramework(WrappedArray(org.scalameter.ScalaMeterFramework)))
[info] project2/perf:testFrameworks
[info]  List(TestFramework(WrappedArray(org.scalameter.ScalaMeterFramework)))
[info] all/perf:testFrameworks
[info]  List(TestFramework(WrappedArray(org.scalameter.ScalaMeterFramework)))

Howewer, perf:test runs ScalaTest tests, and doesn't run ScalaMeter ones (instead of vice versa, as expected). How can I fix this problem?

This happens with SBT version 0.13.1 as well as 0.13.0.

Answer 1

I was missing the

inConfig(PerfTest)(Defaults.testTasks)

line from http://www.scala-sbt.org/release/docs/Detailed-Topics/Testing.html#additional-test-configurations-with-shared-sources.