I'm creating GCP VM instances via Terraform, with startup and shutdown scripts supplied via metadata. I can tell the startup script works, but the shutdown script does not run when terraform apply requires the instance to be destroyed.
I'd like to check that the shutdown-script was actually copied to the server, and also be able check the content of both startup and shutdown scripts if I need to debug the templates.
On AWS, they're copied to /var/lib/cloud/instance but under the same location on my GCP VMs, I don't see my scripts there. Are there other locations I should search?
Compute Engine Shutdown scripts are not guaranteed to run.
Compute Engine executes shutdown scripts only on a best-effort basis. In rare cases, Compute Engine cannot guarantee that the shutdown script will complete.
You can manually execute a shutdown script to verify/debug execution:
google_metadata_script_runner shutdown
Startup and Shutdown scripts are stored in metadata. The metadata might indicate that the startup script is located at a URL (e.g. Cloud Storage). The metadata is copied to a local file for execution. The filename is not documented.
Tip: Connect via the instance's serial port and you can watch the shutdown GCEMetadataScripts event start the shutdown script and see the result while it executes.