Pass variable from CodeBuildStep to stack in Stage

Question

I have developed a basic cdk pipeline. My Intended target is to create a pipeline where each pipeline will do

• extract artifact version from pom
• a build of the lambda (output is zip file)
• deploy artifact to codeArtifact
• update version in pom (haven't done it yet)
• deploy zip file to lambda function with a new version (artifact version in pom)

I am using aws-cdk-lib 2.99.1. I have developed a basic cdk pipeline as follows

• CodeBuildStep
  • synth
    • export LAMBDA_SNAPSHOT_VERSION_ID
    • mvn install -Passembly-zip
    • mvn deploy:deploy-file to snapshot repository
    • cd ${CODEBUILD_SRC_DIR}/cdk
    • cdk synth -c LAMBDA_SNAPSHOT_VERSION_ID=${LAMBDA_SNAPSHOT_VERSION_ID}
• stage
  • dev-deploy (LambdaPipelineStage extends Stage)
    • LambdaStack
      • inputs are functionName, env, version (LAMBDA_SNAPSHOT_VERSION_ID)

LambdaStack will create a function and a version with description as LAMBDA_SNAPSHOT_VERSION_ID

I can do a cdk deploy without issue. However, when I deployed this change, pipeline will do a infinite loop (Source -> Build-> UpdatePipeline). If I remove code of passing version value, then it will be working fine and will execute the function as follows. Source -> Build-> UpdatePipeline -> Assets -> lambda-dev-deploy (prepare-> deploy)

CodeBuildStep synthStep = new CodeBuildStep(
    codeBuildName,
    CodeBuildStepProps
    .builder()
    .projectName(codeBuildName)
    .cache(codebuildCache)
    .input(
        CodePipelineSource.codeCommit(
            codeCommitRepository,
            "master",
            CodeCommitSourceOptions
            .builder()
            .eventRole(codePipelineEventRole)
            .actionName("source-change")
            .build()
        )
    )
    .partialBuildSpec(getPartialBuildSpec())
    .installCommands(getInstallCommands())
    .commands(getBuildCommands())
    .primaryOutputDirectory("${CODEBUILD_SRC_DIR}/cdk/cdk.out")
    .buildEnvironment(buildEnvironment)
    .actionRole(codePipelineRole)
    .role(codeBuildRole)
    .rolePolicyStatements(List.of(policyStatement))
    .build()
);

String codePipeline = "cdk-codepipeline-" + repoName;
CodePipeline pipeline = new CodePipeline(
    this,
    codePipeline,
    CodePipelineProps.builder()
    .pipelineName(codePipeline)
    .selfMutation(Boolean.TRUE)
    .role(codePipelineRole)
    .synth(synthStep)
    .crossAccountKeys(Boolean.TRUE)
    .artifactBucket(getArtifactBucket())
    .build()
);

String functionName = "dev-" + repoName;

pipeline.addStage(
    new LambdaPipelineStage(
        this,
        repoName + "-dev-deploy",
        StageProps
        .builder()
        .stageName(repoName + "-dev-deploy")
        .build(),
        functionName,
        Constants.DEVELOPMENT_ENV

    )
);

public class LambdaPipelineStage extends Stage {
    public LambdaPipelineStage(final Construct scope,
        final String id,
        final StageProps props,
        final String functionName,
        final String environment
    ) {
        super(scope, id, props);

        LambdaStack stack = new LambdaStack(
            this,
            functionName,
            StackProps
            .builder()
            .stackName(functionName)
            .build(),
            functionName,
            environment,
            getValueFromContext(scope, Constants.LAMBDA_SNAPSHOT_VERSION_ID)
        );

    }

    private String getValueFromContext(final Construct scope, final String variableKey) {
        return (String) scope.getNode().tryGetContext(variableKey);
    }
}

public class LambdaStack extends Stack {

    public LambdaStack(final Construct scope,
        final String id,
        final StackProps props,
        final String functionName,
        final String environment,
        final String version) {
        super(scope, id, props);

        initStack(functionName, environment, version);
    }

    private void initStack(final String functionName,
        final String environment,
        final String version) {

        String artifactId = Constants.LAMBDA_ARTIFACT_ID_VALUE;
        String handler = Constants.LAMBDA_HANDLER;

        Code code = Code.fromAsset(
            "../lambda/target/" + artifactId + "-" + version + ".zip"
        );

        IRole lambdaRole = getLambdaRole();
        IFunction lambdaFunction = Function.Builder.create(this, functionName)
            .functionName(functionName)
            .runtime(Constants.LAMBDA_RUNTIME)
            .code(code)
            .handler(handler)
            .role(lambdaRole)
            .environment(getEnvironmentVariables(functionName, environment))
            .memorySize(Constants.LAMBDA_MEMORY_SIZE)
            .architecture(Constants.LAMBDA_ARCHITECTURE)
            .ephemeralStorageSize(Size.mebibytes(Constants.LAMBDA_MEMORY_SIZE))
            .timeout(Duration.seconds(Constants.LAMBDA_TIMEOUT_SECONDS))
            .layers(getLayers(functionName))
            .build();


        // Create a Lambda version using the Maven version

        Version lambdaVersion = new Version(
            this,
            functionName + "-version",
            VersionProps.builder()
            .lambda(lambdaFunction)
            .description(version)
            .build()
        );

    }
}

Answer 1

After several debugging and multiple pipeline failed executions, I found out the issue. I am setting up several environment variables to the build environment. previously I was using Map.of(). But when we created it like this, in compilation, order of the entries are randomized. Therefore cdk synth will identify that there is a change to the pipeline. I changed it to TreeMap. Now it is working fine.

Note: better to have cdk diff in synth step to identify what are the changes to the pipeline

    @NotNull
private static Map<String, BuildEnvironmentVariable> getCodeBuildUserDetails() {
    return Map.of(
            "AWS_ACCESS_KEY_ID", getSecretEnvVariable("XXXX"),
            "AWS_SECRET_ACCESS_KEY", getSecretEnvVariable("XXXXX"),
            "AWS_DEFAULT_REGION", getPlainTextEnvVariable("XXXXX")
    );
}

Solution

@NotNull
private static Map<String, BuildEnvironmentVariable> getCodeBuildUserDetails(
        final PipelineConfiguration config) {

    Map<String, BuildEnvironmentVariable> variableMap = new TreeMap<>();
    variableMap.put(
            "AWS_ACCESS_KEY_ID",getSecretEnvVariable("XXXXX")
    );
    variableMap.put(
            "AWS_SECRET_ACCESS_KEY",getSecretEnvVariable("XXXXX")
    );
    variableMap.put(
            "AWS_DEFAULT_REGION",getPlainTextEnvVariable("XXXXX")
    );
    return variableMap;
}