From the book Java Concurrency in Practice , Chapter 12.1 Testing for correctness, specifically in sub-section 12.1.3 Testing safety(where the author wants to set up test cases for testing data-race safety of a Bounded Buffer class)
To ensure that your test actually tests what you think it does, it is important that the checksums themselves not be guessable by the compiler. It would be a bad idea to use consecutive integers as your test data because then the result would always be the same, and smart compiler could conceivably just precompute it.
To avoid this problem, test data should be generated randomly,but many otherwise effective tests are compromised by a poor choice of random number generator(RNG). Random number generation can create couplings between classes and timing artifacts because most random number generator classes are thread safe and therefore introduce additional synchronization.Giving each thread its own RNG allows a non-thread-safe RNG to be used.
I do not understand the point made by the author against using Random number generators for generating the test inputs. Specifically the line Random number generation can create couplings between classes and timing artifacts is not clear to me.
Random number generation can create couplings between classes and timing artifacts is not clear to me.
This is more clear by taking into account the next sentence:
because most random number generator classes are thread safe and therefore introduce additional synchronization
It's the memory synchronization that may change the timing of your program. If you look into Random
, you can see that it uses an AtomicInteger
under the covers so using it will cause read and write memory barriers as part of the generation of the test data which may change how the other threads see data and the timing of your application overall.
Which classes and timing artifacts is he referring to here?
Any class that uses threads and relies on memory synchronization may be affected. Basically all threads and classes that they call.
What kind of couplings the RNG can create?
As @Bill the Lizard commented on, the book is saying that by using a RNG, the timing of the program then is relying on or affected by the RNG synchronization.
The real lesson here is that the test data you inject into program should not change the timing of your program if possible. It is often difficult and can be impossible but the goal is to simulate the application behavior (timing, input, output, ...) as much as possible in the test.
In terms of a solution, you could use another simple random algorithm that was not synchronized. You could also generate a class that stored 10000 random numbers (or however many you need) beforehand and then handed them out without synchronization. But by using a class in your tests that did memory synchronization, you are changing the timing of your program.