Search code examples
karatespock

While running Karate(1.0.1) tests from Spock, System property that was set in mock ends up undefined in karate.properties['message']


In karate version 0.9.5 I was able to use System.setProperty('message', message) during a mock invocation. Then that property was available inside a feature using karate.properties['message']. I have upgraded to version 1.0.1 and now result of karate.properties['message'] results in undefined

Spock Test code

@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
class ApiTestRunnerSpec extends Specification {

    @LocalServerPort
    private int port

    @SpringBean
    MessageLogger messageLogger = Mock()

    def "setup"() {
        System.out.println("Running on port: " + port)
        System.setProperty("server.port", "" + port)
    }

    def "Run Mock ApiTest"() {
        given:
        System.setProperty('foo', 'bar')

        when:
        Results results = Runner.path("classpath:").tags("~@ignore").parallel(5)

        then:
        results != null
        1 * messageLogger.logMessage(_ as String) >> { String message ->
            assert message != null
            System.setProperty("message", message)
        }
    }
}

Controller

@RestController
public class MessageController {

  @Autowired private MessageLogger messageLogger;

  @GetMapping("/message")
  public String message() {
    String message = "Important Message";

    messageLogger.logMessage(message);

    return message;
  }
}

MessageLogger

@Component
public class MessageLogger {

  public void logMessage(String message) {
    System.out.println(message);
  }
}

karate-config.js

function fn() {
  karate.configure('connectTimeout', 10000);
  karate.configure('readTimeout', 10000);
  karate.configure('ssl', true);

  var config = {
    localUrl: 'http://localhost:' + java.lang.System.getProperty('server.port'),
  };
  print('localUrl::::::::::', config.localUrl);
  return config;
}

Feature

@mockMessage
@parallel=true
Feature: Test Message

  Background:
    * url localUrl

  Scenario: GET

  Given path '/message'
  When method get
  Then status 200

  * print 'foo value ' + karate.properties['foo']
  * print 'message value ' + karate.properties['message']

0.9.5

2021-04-28 15:07:51.819 (...) [print] **foo value bar**
2021-04-28 15:07:51.826 (...) [print] **message value Important Message**

1.0.1

2021-04-28 14:36:58.566 (...) [print] **foo value bar** 
2021-04-28 14:36:58.580 (...) [print] **message value undefined** 

Link to project on github


Solution

  • I cloned your project and noticed a few outdated things (Groovy, Spock and GMaven+ versions). Upgrading them did not change the outcome, I can still reproduce your problem.

    A also noticed that in your two branches the POM differs in more than just the Karate version number, also the dependencies differ. If I use the ones from the 1.0.1 branch, tests do not work under 0.9.5 anymore. So I forked your project and sent you two pull requests for each branch with a dependency setup working identically for both Karate versions. Now the branches really just differ in the Karate version number:

    https://github.com/kriegaex/spock-karate-example/compare/karate-0.9.5...kriegaex:karate-1.0.1

    BTW, for some reason I had to compile your code running JDK 11, JDK 16 did not work. GMaven+ complained about Java 16 groovy class files (bytecode version 60.0), even though GMaven+ should have used target level 11. No idea what this is about. Anyway, on Java 11 I can reproduce your problem. As the Spock version is identical for both branches, I guess the problem is within Karate itself. I recommend to open an issue there, linking to your GitHub project (after you have accepted my PRs). Spock definitely sets the system property, I have added more log output into the stubbing closure order to verify that. Maybe this is an issue concerning how and when Karate communicates with Spock.


    Update: Peter Thomas suggested in his answer to store the value to be transferred to the feature in a Java object and access that one from the feature after the Spock test has set it. I guess, he means something like this:

    https://github.com/kriegaex/spock-karate-example/commit/ca88e3da

    @SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
    class ApiTestRunnerSpec extends Specification {
    
        @LocalServerPort
        private int port
    
        @SpringBean
        MessageLogger messageLogger = Mock() {
            1 * logMessage(_ as String) >> { String message ->
                assert message != null
                MessageHolder.INSTANCE.message = message
            }
        }
    
        def "setup"() {
            System.out.println("Running on port: " + port)
            System.setProperty("server.port", "" + port)
        }
    
        def "Run Mock ApiTest"() {
            given:
            Results results = Runner
              .path("classpath:")
              .systemProperty("foo", "bar")
              .tags("~@ignore")
              .parallel(5)
    
            expect:
            results
        }
        
        static class MessageHolder {
            public static final MessageHolder INSTANCE = new MessageHolder()
            private String message
    
            private MessageHolder() {}
    
            String getMessage() {
                return message
            }
    
            void setMessage(String message) {
                this.message = message
            }
        }
    }
    
    @mockMessage
    @parallel=true
    Feature: Test Message
    
      Background:
        * url localUrl
    
      Scenario: GET
    
      Given path '/message'
      When method get
      Then status 200
    
      * print 'foo value ' + karate.properties['foo']
      * def getMessage =
        """
        function() {
          var MessageHolder = Java.type('com.example.spock.karate.ApiTestRunnerSpec.MessageHolder');
          return MessageHolder.INSTANCE.getMessage();
        }
        """
      * def message = call getMessage {}
      * print 'message value ' + message
    

    Update 2: This is the implementation of Peter's second idea to simply access Java system properties via JS. So I simplified the working, but unnecessarily complicated version with the message holder singleton, eliminating it again:

    https://github.com/kriegaex/spock-karate-example/commit/e235dd71

    Now it simply looks like this (similar to the original Spock specification, only refactored to be a bit less verbose):

    @SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
    class ApiTestRunnerSpec extends Specification {
    
        @LocalServerPort
        private int port
    
        @SpringBean
        MessageLogger messageLogger = Mock() {
            1 * logMessage(_ as String) >> { String message ->
                assert message != null
                System.setProperty('message', message)
            }
        }
    
        def "setup"() {
            System.out.println("Running on port: " + port)
            System.setProperty("server.port", "" + port)
        }
    
        def "Run Mock ApiTest"() {
            expect:
            Runner.path("classpath:").systemProperty("foo", "bar").tags("~@ignore").parallel(5)
        }
    }
    

    The only important change is in the Karate feature:

    @mockMessage
    @parallel=true
    Feature: Test Message
    
      Background:
        * url localUrl
    
      Scenario: GET
    
      Given path '/message'
      When method get
      Then status 200
    
      * print 'foo value ' + karate.properties['foo']
      * def getMessage = function() { return Java.type('java.lang.System').getProperty('message'); }
      * print 'message value ' + getMessage()