Is there any way to handle the changes of aws layer to all the associated aws lambdas in serverless if they both are in different stack?

Question

I am using Serverless Framework and AWS cloud services for my project. I have created many services with AWS lambda and created a layer to serve common purposes of those services. I am maintaining the layer in a separate stack and included this layer to all the services using cloudformation syntax. The problem is everytime I am updating the layer and deploying it to AWS, I need to deploy all the services once again to reflect those changes into the associated services. Is there any way to mitigate this issue so that once I deploy my layer, all the associated services will also be updated with latest layer changes? I don't need to update those services manually everytime I deploy a layer. Hope this make sense to you. I am adding serverless.yml file of layer and one of my service to make it more clear to you. Looking forward to hear from you. Thanks in advance.

serverless.yml file for layer

service: t5-globals

frameworkVersion: '2'

provider:
  name: aws
  runtime: nodejs14.x
  lambdaHashingVersion: 20201221
  environment: 
    NODE_PATH: "./:/opt/node_modules"

layers:
  t5Globals:
    path: nodejs
    compatibleRuntimes: 
      - nodejs14.x

resources:
  Outputs:
    T5GlobalsLayerExport:
      Value:
        Ref: T5GlobalsLambdaLayer
      Export:
        Name: T5GlobalsLambdaLayer

plugins:
  - serverless-offline

serverless.yml file of one service

service: XXXXX
projectDir: XXXXXX
frameworkVersion: '2'
provider: XXXXXX
plugins: XXXXXX
resources: XXXXXX
functions:
  XXXXXX:
    name: XXXXXXX
    handler: XXXXXXX
    layers:
      - ${cf:t5-globals-dev.T5GlobalsLayerExport}
    events:
      - http: XXXXX

Answer 1

When you implement CI/CD, this thing must be automated, we normally use a trigger that trap the git change event of say CodeCommit and execute a lambda function.

The lambda function then scans the files for changes and creates new layer, and update all the lambda functions that uses this layer to use latest version of the layer.

Sharing the code written in python, can change and use as per your needs.

    import base64
    import datetime
    from urllib.parse import unquote_plus
    from botocore.exceptions import ClientError

    s3 = boto3.resource('s3')
    region = os.environ['region']
    lambdaclient = boto3.client('lambda', region_name=region)

    def layerUpdateExistingLambdaFunctions(extensionLayer) : 
            functionslist = []
            nextmarker = None
            while True:
                    if nextmarker is not None:
                            list_function_response = lambdaclient.list_functions(
                                    FunctionVersion='ALL',
                                    MaxItems=10,
                                    Marker = nextmarker
                            )
                    else:
                            list_function_response = lambdaclient.list_functions(
                                    FunctionVersion='ALL',
                                    MaxItems=10
                            )        
                    if 'Functions' in list_function_response.keys():
                            for function in list_function_response['Functions']:
                                    functionName = function['FunctionName']
                            
                                    layersUsed = []
                                    usingExtensionLayer = False                
                                    if 'Layers' in function.keys():
                                            layers = function['Layers']
                                            for layer in layers:
                                                    layerArn = layer['Arn'][:layer['Arn'].rfind(':')]
                                                    layersUsed.append(layerArn)
                                                    if extensionLayer.find(layerArn) >= 0:
                                                            print(f'Function {functionName} using extension layer')
                                                            usingExtensionLayer = True
                                                            functionslist.append(functionName)

                                    if usingExtensionLayer is True:
                                            extensionLayerArn = extensionLayer[:extensionLayer.rfind(':')]
                                            print(f'Existing function {functionName} using {extensionLayerArn}, needs to be updated')
                                            newLayers = []
                                            for layerUsed in layersUsed:
                                                    newLayers.append(lambdaclient.list_layer_versions(
                                                            CompatibleRuntime = 'python3.7',
                                                            LayerName=layerUsed
                                                    )['LayerVersions'][0]['LayerVersionArn'])                                                
                                            lambdaclient.update_function_configuration(
                                                    FunctionName = functionName,
                                                    Layers=newLayers
                                            )
                                            print(f'Function {functionName} updated with latest layer versions')
                            
                    if 'NextMarker' in list_function_response.keys():
                            nextmarker = list_function_response['NextMarker']
                    else:
                            break
            return functionslist