How can I set a minimum Spark module version in Foundry?

Question

I’m trying to set jobSpecOverrides in transforms-python/build.gradle in order to specify spark module of previous version to use:

            jobSpecOverrides {
                minimumModuleVersionOverride {
                version = "1.579.0"
                expiresAfter = "2022-12-30T08:00Z"
                }
            }

I’m running into the following error:

Caused by:
 org.gradle.internal.metaobject.AbstractDynamicObject$CustomMessageMissingMethodException:
  Could not find method jobSpecOverrides() for arguments
   [build_anieopudv6499qpbc9cij4zl7$_run_closure1$_closure4@5f06c6c8]
    on object of type org.gradle.api.internal.initialization.DefaultScriptHandler.

Can someone help me understand this error?

Answer 1

This override looks correct. A Jobspec override needs to be in the transforms-python/build.gradle at the end of the file. Make sure your repository is up to date and the override is at the bottom of the file.

Below I attached an example override - transforms-python/build.gradle file.

/ DO NOT MODIFY THIS FILE
buildscript {
    repositories {
        maven {
            credentials {
                username ''
                password transformsBearerToken
            }
            authentication {
                basic(BasicAuthentication)
            }
            url project.transformsMavenProxyRepoUri
        }
    }

    dependencies {
        classpath "com.palantir.transforms.python:lang-python-gradle-plugin:${transformsLangPythonPluginVersion}"
    }
}


jobSpecOverrides {
    minimumModuleVersionOverride {
        version = "1.376.0" // there must be a version that's runnable on the stack that's greater than or equal to this. also must be higher than the transforms-defined one.
        expiresAfter = "2021-11-09T08:00Z" // must be at most 7 days after the time CI runs.
    }
}