How to copy pipeline logs in Jenkins2 and upload it to amazon-S3?

Question

I'm creating a pipeline which creates a CloudFormation Stack, deploy my code to the instance and destroy the CloudFormation Stack at the end (Whether the pipeline fails or successfully completes).

But before destroying the stack I want to copy the logs of Jenkins and save them in amazon-S3. I'm able to successfully create the stack and delete it. But not sure how to copy logs to amazon-s3.

I'm using Jenkins2

Answer 1

1. Assume you have a script with cli/boto3 commands to push required logs to S3 in the ec2 box(Jenkins box) - Create it using init-data of ec2
2. Create a lambda custom resource in CFT which will do nothing for "Create/Update stack" operations. However, this should execute the push script during "Delete stack" operation (Refer : http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/template-custom-resources-lambda.html)
3. The script execution can be triggerred from lambda using Systems Manager Run Command (Refer : https://stackoverflow.com/a/40135899/4840338)

This way the stack gets created with a custom lambda resource which will upload the logs by the executing the script in the instance upon deletion of the stack.