Search code examples
c#.netmacos.net-coreuniversal-binary

How do you create a self-contained .NET core app for macOS that's x86_x64 / arm64 universal, without Visual Studio?

I'm trying to create a macOS .NET core application that's self contained and is a universal x86_64 / arm64 app. I don't have Visual Studio for Mac installed, and given Microsoft is retiring it, I'm trying to work out a method for doing this that doesn't rely on it.

I closely read through this question: Can I create a .net6 app for macOS with both x64 and arm64 support? ...but its accepted answer doesn't work, and seems to only apply to those using Visual Studio for Mac, which I am not. I need a solution that works specifically for using .NET core's command line tools. (edit) Further, that question is not about creating a self-contained release.

Normally with .NET core app, you would create a ready-to-distribute build using:

dotnet publish -c Release -r osx-x64 --self-contained

or

dotnet publish -c Release -r osx-arm64 --self-contained

This will create a folder in your build directory with the self-contained app. It won't be assembled into a application bundle, but it will contain the .NET core runtime including all of the dlls necessary and be self contained.

Unfortunately, it will also only be built for one architecture, either x86_64 or arm64, depending on the -r argument passed to dotnet publish.

I can assemble the application bundle myself by copying all of these files into the correct places. And I can create universal binaries of all the executables and dylibs by taking a release created for x86_64 and using lipo to combine it with a release created for arm64.

However, all of that still isn't enough because all of the .NET core runtime DLLs copied by dotnet publish are still platform specific. Trying to run the app on the architecture that was not the one you used to build the app will result in this error:

Failed to load System.Private.CoreLib.dll (error code 0x8007000B)
Path: /path/to/app/Contents/MacOS/System.Private.CoreLib.dll
Error message: An attempt was made to load a program with an incorrect format.
 (0x8007000B)
Failed to create CoreCLR, HRESULT: 0x8007000B

I'm now totally stuck. There's no documentation from Microsoft about how to do this. It's not even clear if it's possible. It ought to be, assuming there are runtime DLLs that are platform independent and there's some way to tell dotnet publish to use those instead of platform-specific ones.

Can anyone help?

Edit: One possible option is to bundle both the x86_64 and arm64 release builds into a single application bundle and then having the bundle's executable be a shell script that conditionally launches one depending on the system architecture. Any shared resources could be handled by symlinks. But I'm wondering if there's any better options, like perhaps using a platform independent (framework-dependent?) runtime, if it exists.

Edit 2: I've discovered that several of the runtime DLLs produced by dotnet publish --self-contained are the same between arm64 and x86_64. So another improvement that can be made to the above method is to only have one copy of each of those DLLs and use symlinks between the arm64 and x86_64 runtimes.

Really the big question at this point is whether it's possible to have platform-independent / framework-dependent (to use Microsoft's terminology) / arm64 + x86_64 universal runtime DLLs.

Solution

  • After more research, I've developed a method of creating a universal application that seems to be as space efficient as possible, and avoids using PublishSingleFile, which is apparently important for games developed using MonoGame, and may be important in other use cases as well.

    It doesn't seem possible to make a truly self-contained .NET release that uses a universal version of the .NET runtime, because no such runtime exists. Some of the DLLs included in a self-contained release have non-managed code (i.e. native code), and therefore those DLLs are specific to a platform and CPU architecture.

    However, this only accounts for roughly half of the DLLs in the .NET runtime. The other half are only managed code, and therefore can run on either x86_64 or arm64 macOS. We can take advantage of this to create a universal application that contains both x86_64 and arm64 runtimes with as little redundancy as possible.

    First, create both an x86_64 and arm64 release of your application with:

    dotnet publish -c Release -r osx-x64 -p:PublishReadyToRun=false \
    -p:TieredCompilation=false -p:PublishTrimmed=false --self-contained
dotnet publish -c Release -r osx-arm64 -p:PublishReadyToRun=false \
    -p:TieredCompilation=false -p:PublishTrimmed=false --self-contained

    I recommend having PublishReadyToRun, TieredCompilation, and PublishTrimmed set to false, but you can change those as you require.

    Then create an application bundle structured like this:

    Contents
├── MacOS
│   ├── osx-arm64
│   │   └── [ your self-contained arm64 release goes here ]
│   ├── osx-x64   
│   │   └── [ your self-contained x86_64 release goes here ]
│   ├── shared
│   │   └── [ all common files go here ]
│   └── $EXECUTABLE
├── Resources
│   ├── icon.icns
│   └── [ any other resources go here ]
└── Info.plist

    Where $EXECUTABLE is the name of your .NET executable produced by dotnet publish.

    The executable file in the macOS directory should contain a shell script that looks like this:

    #!/bin/sh

DIR=$(dirname "$0")
ARM64=$(sysctl -ni hw.optional.arm64)

if [[ "$ARM64" == 1 ]]; then
    exec "$DIR/osx-arm64/$EXECUTABLE"
else
    exec "$DIR/osx-x64/$EXECUTABLE"
fi

    This will launch the release of the correct architecture when the user launches the application. Note that macOS (possibly due to a bug) will always launch this shell script in an x86_64 process on arm64 systems that have Rosetta 2 installed, regardless of the value of the LSArchitecturePriority key in Info.plist. So we need to use sysctl in order to truly figure out if we're running on an Apple Silicon mac, because uname will indicate the system is x86_64 when running under Rosetta 2.

    The last step is to copy any DLLs that only contain managed code (and are therefore compatible with both x86_64 and arm64 systems) from the osx-arm64 / osx-x64 directories into shared, and then replace those DLLs in osx-arm64 and osx-x64 with symlinks to the same DLL in shared. The symlinks must use relative paths. You can tell which DLLs only contain managed code because they will be identical between the osx-arm64 and osx-x64 directories.

    If your project requires any external dylibs, make sure they're universal arm64 / x86_64 binaries, and copy them into shared as well. Then create symlinks to them in osx-arm64 and osx-x64, like you did with the DLLs.

    This process would be quite difficult and annoying to do by hand, so I've written a bash script that will do it automatically. You can run this script from anywhere inside your .NET project directory and will automatically generate x86_64, arm64, and universal applications. Be sure to modify the variables at the top of the script as necessary, and add in lines to copy in any needed dylibs and other resources into the application bundle as needed. (See the comments in the script for where to make these changes.)

    #!/bin/bash

# Edit these variables as appropriate:
EXECUTABLE="YourExecutableHere"
APP_BUNDLE="Your App Here.app"
PLIST_FILE="path/to/Info.plist"
ICNS_FILE="path/to/icon.icns"

find_closest_csproj_directory() {
    local current_dir=$(pwd)
    while [ "$current_dir" != "/" ]; do
        if [ -e "$current_dir"/*.csproj ]; then
            echo "$current_dir"
            return
        fi
        current_dir=$(dirname "$current_dir")
    done
    return 1
}

# This script is intended to run from the same directory as your csproj file:
PROJECTDIR=$(find_closest_csproj_directory)
if [ $? -eq 1 ]; then
    echo "No .csproj file found in the current directory or its parent" \
         "directories."
    exit 1
fi
pushd $PROJECTDIR > /dev/null

rm -rf "bin/app"
rm -rf bin/Release/net7.0/osx-x64
rm -rf bin/Release/net7.0/osx-arm64

echo Creating release builds

dotnet publish -c Release -r osx-x64 -p:PublishReadyToRun=false \
    -p:TieredCompilation=false -p:PublishTrimmed=false --self-contained
dotnet publish -c Release -r osx-arm64 -p:PublishReadyToRun=false \
    -p:TieredCompilation=false -p:PublishTrimmed=false --self-contained

echo Creating apps

mkdir -p "bin/app/x86_64/$APP_BUNDLE/Contents/MacOS"
mkdir -p "bin/app/x86_64/$APP_BUNDLE/Contents/Resources"
cp $PLIST_FILE "bin/app/x86_64/$APP_BUNDLE/Contents/"
cp $ICNS_FILE "bin/app/x86_64/$APP_BUNDLE/Contents/Resources"
cp -a bin/Release/net7.0/osx-x64/publish/* \
    "bin/app/x86_64/$APP_BUNDLE/Contents/MacOS"

mkdir -p "bin/app/arm64/$APP_BUNDLE/Contents/MacOS"
mkdir -p "bin/app/arm64/$APP_BUNDLE/Contents/Resources"
cp $PLIST_FILE "bin/app/arm64/$APP_BUNDLE/Contents/"
cp $ICNS_FILE "bin/app/arm64/$APP_BUNDLE/Contents/Resources"
cp -a bin/Release/net7.0/osx-arm64/publish/* \
    "bin/app/arm64/$APP_BUNDLE/Contents/MacOS"

echo Combining into universal app

rm -rf "bin/app/universal"

mkdir -p "bin/app/universal/$APP_BUNDLE/Contents/MacOS/osx-arm64"
mkdir -p "bin/app/universal/$APP_BUNDLE/Contents/MacOS/osx-x64"
mkdir -p "bin/app/universal/$APP_BUNDLE/Contents/MacOS/shared"
mkdir -p "bin/app/universal/$APP_BUNDLE/Contents/Resources"

# This app bundle's executable is a shell script will launch either the arm64 or
# x64 executable in osx-arm64 or osx-x64 depending on the system architecture.
# Note that do to a quirk (or bug) in macOS, this script will always be run in
# an x86_64 process on arm64 systems that have Rosetta 2 installed, regardless
# of the LSArchitecturePriority key in Info.plist, hence the use of sysctl
# instead of uname.
cat <<EOF > "bin/app/universal/$APP_BUNDLE/Contents/MacOS/$EXECUTABLE"
#!/bin/sh

DIR=\$(dirname "\$0")
ARM64=\$(sysctl -ni hw.optional.arm64)

if [[ "\$ARM64" == 1 ]]; then
    exec "\$DIR/osx-arm64/$EXECUTABLE"
else
    exec "\$DIR/osx-x64/$EXECUTABLE"
fi
EOF
chmod a+x "bin/app/universal/$APP_BUNDLE/Contents/MacOS/$EXECUTABLE"

cp $PLIST_FILE "bin/app/universal/$APP_BUNDLE/Contents/"
cp $ICNS_FILE "bin/app/universal/$APP_BUNDLE/Contents/Resources"
cp -a bin/Release/net7.0/osx-arm64/publish/* \
    "bin/app/universal/$APP_BUNDLE/Contents/MacOS/osx-arm64"
cp -a bin/Release/net7.0/osx-x64/publish/* \
    "bin/app/universal/$APP_BUNDLE/Contents/MacOS/osx-x64"

# Edit this script at this point to do the following as appropriate for your
# project:
# 
# 1. Copy any other shared application data (such as MonoGame Content directory)
# into either bin/app/universal/$APP_BUNDLE/Contents/MacOS/shared or
# bin/app/universal/$APP_BUNDLE/Contents/Resources as appropriate.
# e.g.:
# cp -a Content "bin/app/universal/$APP_BUNDLE/Contents/Resources/Content"
# 
# 2. Copy any external dylibs or other supporting files into:
# bin/app/universal/$APP_BUNDLE/Contents/MacOS/shared
# e.g.:
# cp macOS/libs/libMyLibrary.dylib \
#     bin/app/universal/$APP_BUNDLE/Contents/MacOS/shared
# 
# 3. Create symlinks pointing to these external dylibs or other supporting files
# contained within bin/app/universal/$APP_BUNDLE/Contents/MacOS/shared using
# a path relative to bin/app/universal/$APP_BUNDLE/Contents/MacOS/osx-* in both
# the osx-arm64 and osx-x64 folders.
# e.g.:
# ln -s ../shared/libMyLibrary.dylib \
#    "bin/app/universal/$APP_BUNDLE/Contents/MacOS/osx-arm64/libMyLibrary.dylib"
# ln -s ../shared/libMyLibrary.dylib \
#    "bin/app/universal/$APP_BUNDLE/Contents/MacOS/osx-x64/libMyLibrary.dylib"

pushd "bin/app/universal/$APP_BUNDLE/Contents/MacOS/osx-x64" > /dev/null

# Now we remove any redundant .NET runtime DLLs that do not contain unmanaged
# code and replace them with symlinks. We can tell which those are because they
# will be identical between the osx-arm64 and osx-x64 directories:
for file in ./*.dll; do
    if diff -q $file ../osx-arm64/$file > /dev/null; then
        # Files are the same
        echo Combining $file
        cp $file ../shared
        rm $file
        rm ../osx-arm64/$file
        ln -s ../shared/$file $file
        ln -s ../shared/$file ../osx-arm64/$file
    fi
done

popd > /dev/null
popd > /dev/null