Terraform: Deploying the same web-app for multiple clients

Question

I am currently using Terraform to deploy a PHP app to AWS.
This PHP app is deployed as a Service using AWS ECS.

I have multiple clients using this app, and each client receives their own copy of the system with their own configuration as their own service - a white label if you will.

Now, having done a bit of research on Terraform I've modularized my code and created the following file structure:

+---my-application
    |   shared.tf
    |   iam_policies.tf
    |   iam_roles.tf
    |   variables.tf
    |   web-apps.tf
    |   
    +---modules
    |   \---role
    |   |       main.tf
    |   |       outputs.tf
    |   |       variables.tf
    |   |       
    |   \---webapp
    |           main.tf
    |           variables.tf
    |           
    +---templates
            web_definition.tpl.json

My problem lies in the web-apps.tf file which I use as the "glue" for all of the webapp modules:

module "client_bob" {
  source = "modules/webapp"
  ...
}

module "client_alice" {
  source = "modules/webapp"
  ...
}

... Over 30 more client module blocks ...

Needless to say that this is not a good setup.
It is not scalable and also creates huge .tfstate files.

Once when attempting to use Consul as a backend I got an error message saying I've reached the size limit allowed for Consul KV value.

What is the correct way to approach this situations?

I've looked at all of the questions in the Similar Questions section when I wrote this one and all of them revolve around the idea of using multiple .tfstate files, but I don't quite understand how this would solve my problem, any help would be greatly appreciated!

Answer 1

I did similar projects with terragrunt, take a look.
It was born to answer your requests.

the oss website is https://github.com/gruntwork-io/terragrunt

Terragrunt is a thin wrapper for Terraform that provides extra tools for working with multiple Terraform modules. https://www.gruntwork.io

In your case, you can easily manage with different tfstate files for each client.

I also recommend to manage the iam roles, policies or any other resources for each client as well, do not mix them.

For example, the structure will become to

(I guess you will manage different environments for each client, right?)

└── bob
    ├── prod
    │   ├── app
    │   │   └── terraform.tfvars
    ├── nonprod
        ├── app
            └── terraform.tfvars
└── alice
    ├── prod
    │   ├── app
    │   │   └── terraform.tfvars
    ├── nonprod
        ├── app
            └── terraform.tfvars

...

Later, after you master the command terraform apply-all, then it makes the deployment simpler and easier.

Quick start

https://github.com/gruntwork-io/terragrunt-infrastructure-modules-example

https://github.com/gruntwork-io/terragrunt-infrastructure-live-example