if local variables cant be tested then what other ways can the variable values be checked

Question

This is going to be a long question, driven by the thirst of wanting to know how something is working against the conventional methodologies.

I came across a very interesting application codeacademy, that was actually testing the local variables within a main method. Here is a screenshot of the page, that got me into thinking how this can be possible. found some similar questions in stackoverflow.

Is it possible to obtain variables in main method from junit test?

How to use BCEL or ASM to get the value of a local declared variable in JAVA?

I am not satisfied by knowing that it cant be done, what i want to know, is that, is there a way, like the java compiler api or some other, that can point me into knowing how such an application made is possible.

Answer 1

Testing local variables in implementation-validation (like unit testing, or QA automated testing) is generally bad practice.

Local variables depend on the particular implementation, and particular implementation should be hidden behind reasonably abstract API, to allow the developers to replace the implementation in the future - if they get some better idea - without affecting consumers of the result (as long, as the API is so good, that it doesn't need any change).

With some very complex implementations/algorithms a developer indeed may be interested to verify particular intermediate/inner results of the complex algorithm, to make the development of the implementation itself easier. At that point it makes sense to create internal inner-API providing reasonably abstract intermediate results, even if they are hard-bonded to particular algorithm, and test that inner-API in unit tests. But in case of the algorithm replacement you have to accept change of the inner-API, and all the inner unit tests. There should be still reasonable high-level abstract API which is unaffected by the inner changes.

A need to have tests on the level of local variables should indicate some deeper problem of the code base.

---

Your particular use-case of Java tutorial is quite unrelated to real Java code development, as the purpose of that lecture is completely different.

As was shown by the simple myNumber = 21 + 21; test, the validation of lecture is purely compare-text based, using probably some regex on the source code inputted by the student. Not even checking the resulting bytecode, as that one is identical with myNumber = 42;.

If you are working on some lecturing system, using some kind of custom virtual machine and debug interface may work, but in case of simple lectures even text-compare solution may be enough.

Usually when the student is advanced enough to solve some task, you can start to use input/output from/to stdin/stdout to create automated tests to validate student solution with set of known input/output tests, like some sites like https://www.hackerrank.com/ do, or various programming contests. At that point you don't need access to anything, nor local variables, nor API unit testing, you just redirect stdin to feed solution with desired input, and catch stdout to compare it with designed output.

---

But lecture validation is like completely unrelated to unit testing.

And too much stringent testing for expected result may be even counterproductive in the lecturing process!

Imagine you task the student to write code which will output sum of squares from 1 to N, and you will accept only:

    int sum = 0;
    for (int i = 1; i <= N; ++i) {
        sum += i * i;
    }

(verified on bytecode level, in a way that names of variables and ++ suffix/prefix doesn't matter)

Now some student will try to submit this:

    int sum = 0, square = 0, sq_delta = -1;
    for (int i = 1; i <= N; ++i) {
        sum += (square += (sq_delta += 2));
    }

And it will fail... (even if his solution would be absolutely superior around 1985 to the variant with actual multiplication) ... sad story. :)