Local state cannot be unlocked by another process on terraform

Question

My terraform remote states and lockers are configured on s3 and dynamodb under aws account, On gitlab runner some plan task has been crashed and on the next execution plan the following error pops up:

Error: Error locking state: Error acquiring the state lock: ConditionalCheckFailedException:
The conditional request failed

Lock Info:
  ID:        <some-hash>
  Path:      remote-terrform-states/app/terraform.tfstate
  Operation: OperationTypePlan
  Who:       root@runner-abc-project-123-concurrent-0
  Version:   0.14.10
  Created:   2022-01-01 00:00:00 +0000 UTC
  Info:  some really nice info

While trying to unlock this locker in order to perform additional execution plan again - I get the following error:

  terraform force-unlock <some-hash-abc-123>

  #output:
  Local state cannot be unlocked by another process

How do we release this terraform locker?

Answer 1

According to reference of terraform command: force-unlock

Manually unlock the state for the defined configuration.

This will not modify your infrastructure. This command removes the lock on the state for the current configuration. The behavior of this lock is dependent on the backend being used. Local state files cannot be unlocked by another process.

Explanation: apparently the execution plan is processing the plan output file locally and being apply on the second phase of terraform steps, like the following example:

phase 1: terraform plan -out execution-plan.out

phase 2: terraform apply -input=false execution-plan.out

Make sure that filename is same in phase 1 and 2

However - if phase 1 is being terminated or accidentally crashing, the locker will be assigned to the local state file and therefore must be removed on the dynamodb itself and not with the terraform force-unlock command.

Solution: Locate this specific item under the dynamodb terraform lockers table and explicitly remove the locked item, you can do either with aws console or through the api. For example:

aws dynamodb delete-item \
    --table-name terraform-locker-bucket \
    --key file://key.json

Contents of key.json:

{
 "LockID": "remote-terrform-states/app/terraform.tfstate",
 "Info": {
   "ID":"<some-hash>",
   "Operation":"OperationTypePlan",
   "Who":"root@runner-abc-project-123-concurrent-0",
   "Version":"0.14.10",
   "Created":"2022-01-01 00:00:00 +0000 UTC",
   "Info":"some really nice info"
   }
 }