Environment
Hi I am using buildkite on my aws mac1.metal instances. These agents acts as CI agents in our org. Instances now launch buildkite-agent when the instance is booted (done vis plist in /Library/LauchDaemon). Here is my plist
Issue
The agent trys to run the following command using python subprocess
subprocess.CalledProcessError: Command '['/usr/bin/codesign', '-v', '--sign', 'XXXXXXXXXXXXXXXXXXXXXXXXX', '--entitlements', 'bazel-out/darwin-opt/bin/MYORG/iOS/WidgetExtension/WidgetExtension_entitlements.entitlements', '--force', 'bazel-out/darwin-opt/bin/MYORG/iOS/WidgetExtension/WidgetExtension_archive-root/WidgetExtension.appex']
resulting this error:
ERROR:
--
| bazel-out/darwin-opt/bin/MYORG/iOS/WidgetExtension/WidgetExtension_archive-root/WidgetExtension.appex: errSecInternalComponent
Interestingly running the same job with agent that I launch locally. (ssh into the machine, then run buildkite-agent start) then it worked flawlessly. So I am wondering if there are some kind of permission difference between launching buildkite-agent as a daemon via launchd and starting it locally. (similar to this issue I opened before where bk agent failed a job as launchd daemon but succeed if launch locally)
Things I have tried to fix/Debugging
errSecInternalComponent seems like a common issue with codesign so I tried locking and unlocking the keychain with security unlock-keychain -p password path/to/key
people who use other CI system/tool also encounter the same issue but largely solve when setting SessionCreate in their plist. . Tried but still failing
VNC into the mac1.metal ec2 instance and gave /usr/local/bin/buildkite-agent FDA - still failed the job
VNC in and open KeyChain access and give /usr/local/bin/buildkite-agent access to the private key it is having issue signing - still failed
everything here
Jenkins - Xcode build works codesign fails creating a temp key in post checkout hook but still failed
Wondering if any codesign/security/buildkite wizard 🧙♀️ 🧙♂️ out there knows the difference between launchd buildkite-agent and agent launch locally? I feel like there is a big gotcha I am missing. Perhaps an attribute missing in my plist.
I ran into this problem before, you need to run buildkite agent as LaunchAgent and configure auto login, instead of run as LaunchDaemon.