First, a little background: I'm investigating why my company's MacOS/X application (which by all accounts appears to be correctly signed; it runs fine under MacOS/X 10.11.x and 10.12.x; Gatekeeper is fine with it on all MacOS versions; "spctl --assess", and "codesign -vvvv" all say it satisfies its requirement on all OS versions) nevertheless won't launch under OS/X 10.10.x -- under 10.10.x when I try to launch it, I get a Crash Report where dyld complains that some of the libraries aren't signed correctly:
Dyld Error Message:
Library not loaded: @executable_path/../Frameworks/libcrypto.1.0.0.dylib
Referenced from: /Applications/MyApplication v123/MyApplication.app/Contents/MacOS/MyApplication
Reason: no suitable image found. Did find:
/Applications/MyApplication v123/MyApplication.app/Contents/MacOS/../Frameworks/libcrypto.1.0.0.dylib: code signature invalid for '/Applications/MyApplication v123/MyApplication.app/Contents/MacOS/../Frameworks/libcrypto.1.0.0.dylib'
While investigating that problem, I noticed that the libraries in the .app/Contents/Framework -- which are all signed using the exact same codesign command, via the build/package script on our OS/X build machine running OS/X 10.12 -- have differing kinds of hashes computed for them.
That is, if I look at how one of the non-Qt .dylib files was signed, I see it has only a sha256 hash recorded in it:
sierrabuild-polaris:MyApp v123 autobuild$ codesign -vvvd ./MyApp.app/Contents/Frameworks/libsndfile.1.dylib
Executable=/Applications/MyApp v123/MyApp.app/Contents/Frameworks/libsndfile.1.dylib
Identifier=libsndfile.1
Format=Mach-O thin (x86_64)
CodeDirectory v=20200 size=4140 flags=0x0(none) hashes=125+2 location=embedded
Hash type=sha256 size=32
CandidateCDHash sha256=b4256e9bf0fac567bb8ac86f56964c066b93d069
Hash choices=sha256 <----------------------------- ONLY 256!?
CDHash=b4256e9bf0fac567bb8ac86f56964c066b93d069
Signature size=8846
Authority=Developer ID Application: MyCompany
Authority=Developer ID Certification Authority
Authority=Apple Root CA
Timestamp=Jan 24, 2017, 1:39:58 AM
Info.plist=not bound
TeamIdentifier=5XD27G7646
Sealed Resources=none
Internal requirements count=1 size=172
... but if I look at how any of the captive Qt frameworks was signed, OTOH, I see it has both sha1 and sha256 hashes included:
sierrabuild-polaris:MyApp v123 autobuild$ codesign -vvvd ./MyApp.app/Contents/Frameworks/QtCore.framework/Versions/5/QtCore
Executable=/Applications/MyApp v123/MyApp.app/Contents/Frameworks/QtCore.framework/Versions/5/QtCore
Identifier=org.qt-project.QtCore
Format=bundle with Mach-O thin (x86_64)
CodeDirectory v=20200 size=42549 flags=0x0(none) hashes=1324+3 location=embedded
Hash type=sha256 size=32
CandidateCDHash sha1=09b5854f83091228f1baaad1455e7a30d6500c95
CandidateCDHash sha256=6dfdc74da06618e1b406a6e5fd0794fe43701def
Hash choices=sha1,sha256 <------------- BOTH sha1 and sha256, yay!
CDHash=6dfdc74da06618e1b406a6e5fd0794fe43701def
Signature size=8896
Authority=Developer ID Application: MyCompany
Authority=Developer ID Certification Authority
Authority=Apple Root CA
Timestamp=Jan 24, 2017, 1:39:57 AM
Info.plist entries=8
TeamIdentifier=5XD27G7646
Sealed Resources version=2 rules=13 files=1
Internal requirements count=1 size=184
Given that dyld's error when trying to run my app under Yosemite always refers to one of the libraries that only has a sha256 hash, my working theory is that OS/X 10.10.x's dyld is ancient enough that it doesn't know about SHA-256 hashes, and that is why it is erroring out when it tries to load a captive shared library that is signed only with an SHA-256 hash.
My question (assuming I'm not completely barking up the wrong tree here) is: how does codesign decide when to stamp a file with sha256 hash alone, vs adding both an sha1 and an sha256 hash? And how can I force codesign to always include both hashes, so that my app can launch under 10.10.x again (like it used to before we upgraded our build machine to OSX/Sierra)?
For the record, here is how I'm invoking codesign in my build script -- the invocation arguments are exactly the same for all libraries (both the Qt framework libraries that end up with sha1,sha256 and the non-Qt libraries that end up with only sha256), e.g.:
codesign -f -v -s "Developer ID Application: MyCompanyName" "./Frameworks/libcrypto.1.0.0.dylib"
codesign -f -v -s "Developer ID Application: MyCompanyName" "./Frameworks/QtCore.framework/Versions/5/QtCore"
After a lot of googling around, this answer and this answer led me to the solution.
The problem was that several of the third-party shared libraries included inside my app were being compiled using just their default build settings (e.g. "./configure; make"), and since they were being compiled under OS/X 10.12, naturally they were compiled with only 10.12-compatibility in mind.
In order to get them to compile in such a way that the resulting .dylib files would be appropriate for earlier OS/X versions as well, I added these lines to the top of my build script:
export LDFLAGS="-mmacosx-version-min=10.9"
export CFLAGS="-mmacosx-version-min=10.9"
export CXXFLAGS="-mmacosx-version-min=10.9"
... and that did the trick for all of the libraries (libssh2, libsndfile, libogg, libflac, libvorbis, etc) except for the libssl -- for that one I had to hand-modify the Configure file and insert the -mmacosx-version-min argument into the compiler's command-line arguments that way.
With that change, codesign now applies both SHA-1 and SHA-256 hashes to all of the .dylib files, and the resulting .app now runs as expected under 10.10.x.