Google Service Account's Private Key problems with .NET client tool used to publish Android app to Google Play Console

Question

I am creating an open source tool called google-play-publisher developed as a .NET 7 REST API which delegates on a Google API .NET Client.

The idea is to run it as a container (i.e: service) in my GitLab CI/CD pipeline in order to upload Android apps (i.e: aab files) to Google Play Developer, as it exposes a simple endpoint that I can invoke with curl like this:

curl -XPOST --data-binary @your_file.aab http://localhost:5000/api/apps/<your_package_name>/tracks/<track_name>

to make the continuous delivery convenient.

I have all necessary to make it work, including a Google Service Account with proper roles and credentials, which is known by my Google Play Console.

I download the keys as a JSON format, and it has a format like this (fake values):

{
  "type": "service_account",
  "project_id": "pc-api-...",
  "private_key_id": "5342dce..",
  "private_key": "-----BEGIN PRIVATE KEY-----\nbunchofcharacters\nandmorecharactes=\n-----END PRIVATE KEY-----\n",
  "client_email": "publisher@pc-api-foo-iam.gserviceaccount.com",
  "client_id": "123456",
  "auth_uri": "https://accounts.google.com/o/oauth2/auth",
  "token_uri": "https://oauth2.googleapis.com/token",
  "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
  "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/publisher@pc-api-foo-iam.gserviceaccount.com"
}

NOTE: The -----BEGIN PRIVATE KEY----- and -----END PRIVATE KEY-----\n are very necessary. The Google API .NET client library fails if removed.

From that, I just need 6 things:

1. type
2. project_id
3. private_key_id
4. private_key
5. client_email
6. client_id

in order to be authorized by the Google Publisher v3 API and upload AAB for a specific package into a specific track.

The application needs only those 6 settings to be injected when starting, either by configuring them at appsettings.json or as environment variables injeted when spinning up the container. Pretty standard so far.

My appsettings.json would look like this (with the proper values from the credentials json)

  "ServiceAccount": {
    "PrivateKey": "***",
    "ClientEmail": "***",
    "ProjectId": "***",
    "Type": "***",
    "PrivateKeyId": "***",
    "ClientId": "***"
  },

When I configure this appsettings.json with the proper values, everything works well.

But I don't want to use appsettings.json for secrets. I need to use environment variables so that I can use the tool with docker and inject the variables when spinning up the container/service.

But when running the tool as a docker container or as a service inside GitLab CI/CD pipeline, I have some problems.

PROBLEM 1 - Docker run doesn't like the private key format

When I run the tool (version 0.1.8) as a docker container like this

docker run \
  --name google-play-publisher \
  -p 5000:80 \
  -e ServiceAccount__PrivateKey="-----BEGIN PRIVATE KEY-----\nbunchofcharacters\nandmorecharactes=\n-----END PRIVATE KEY-----\n" \
  -e ServiceAccount__ClientEmail="publisher@pc-api-foo-iam.gserviceaccount.com" \
  -e ServiceAccount__ProjectId="pc-api-..."\
  -e ServiceAccount__Type="service_account" \
  -e ServiceAccount__PrivateKeyId="5342dce.." \
  -e ServiceAccount__ClientId="123456" \
  registry.gitlab.com/roundev/devops/google-play-publisher:0.1.8

and then I POST an aab file for a package

curl -XPOST --data-binary @my_file.aab http://localhost:5000/api/apps/com.foo.app/tracks/production

The authentication fails

System.ArgumentException: PKCS8 data must be contained within '-----BEGIN PRIVATE KEY-----' and '-----END PRIVATE KEY-----'. (Parameter 'pkcs8PrivateKey')

because docker run doesn't seem to like passing special environment variables with \n characters, etc.

PROBLEM 2 - GitLab CI/CD Variables doesn't like that format either

It cannot mask it, which is a problem, and then something doesn't work but I cannot see yet the error (no error shown, the pipeline passes but the AAB is not uploaded, so I need to figure out a way to read GitLab CI/CD service logs, which is out of scope for this)

SOLUTION ATTEMPT 1 - Convert private key value to base64 and decode it

I thought of encoding the private key in base64 (not sure if there is another alternative) in order to gain the following advantages:

• GitLab CI/CD variable could be masked
• Docker run would not behave strange when passing base64 string as environment variable

But this shows a different problem. Using an online base 64 encoder and decoding it in the code to generate google auth credentials doesn't work. The \n characters seem to mess up things.

I have tried different variants, but I always get some invalid_token problems later on when using Google Publisher API.

// Functions used for encoding and decoding
string EncodeBase64(string text)
{
    var textBytes = Encoding.UTF8.GetBytes(text);
    var result = Convert.ToBase64String(textBytes);
    return result;
}

string DecodeBase64(string base64Text)
{
    var base64EncodedBytes = Convert.FromBase64String(base64Text);
    var result = Encoding.UTF8.GetString(base64EncodedBytes);
    return result;
}

I haven't yet found a good solution that allows me to:

• Make my tool work with docker run by passing private key as an environment variable which doesn't cause problems, so that it could work the same as when running it with appsetting.json
• Configure the environment variable in GitLab CI/CD as a variable in a secure way (so that it can be masked).

There must be a good solution using some kind of encoding/decoding private key. But how?

Answer 1

I've resolved it with Hex approach.

Basically the solutions is to grab the Json value for the private_key as it is, and transform it to Hex, for example using this online tool https://codebeautify.org/string-hex-converter so the private key that looked like this "-----BEGIN PRIVATE KEY-----\nfoo...\nbar\n-----END PRIVATE KEY-----\n" now looks like this 2d2d2d2d2d424...b45592d2d2d2d2d5c6e

This string has the following characteristics:

• It can be protected as an environment variable in GitLab CI/CD
• It can be passed as a docker environment variable
• It can be easily read and transformed into the original private_key with this:

private class JsonCredentials
{
    public string Type { get; init; } = string.Empty;
    public string PrivateKeyId { get; init; } = string.Empty;
    public string PrivateKeyHex { get; init; } = string.Empty;
    public string PrivateKey => ToJsonValue(PrivateKeyHex);
    public string ClientEmail { get; init; } = string.Empty;
    public string ProjectId { get; init; } = string.Empty;
    public string ClientId { get; init; } = string.Empty;

    private string ToJsonValue(string hexadecimalValue)
    {
        var hexadecimalValueBytes = Convert.FromHexString(hexadecimalValue);
        var result = Encoding.UTF8.GetString(hexadecimalValueBytes);
        var unescapedResult = Regex.Unescape(result);
        return unescapedResult;
    }
}

See my open source repo for more details: https://gitlab.com/roundev/devops/google-play-publisher and to give it a try.