Search code examples
jsonencryptionmulesoftmule4

Encrypting json file for GCP connection thgough mulesoft

This may be a two part question. I am working on a Mulesoft flow pulling the data from salesforce to BigQuery. As part of best practice, I understand that I should encrypt the passwords and store it in secure.yaml file and reference it in this way ${security::}. However for GCP it expects a securekey.json file in connector. enter image description here

So what is the best way to encrypt json file? secure-properties-tool.jar only expects .yaml or .properties file to encrypt and does not work with json files.

I tried to put the information in a yaml with encrypted values and write a flow to convert yaml file to json. If I use this separate flow to generate runtime json file, not sure how to access the secret in a transform message. enter image description here

I tried to write this separate yaml to json flow without encryption. I was hoping, the subsequent flow in the application picks up this new json file, and connect to GCP. However it failed. The prior flow created the file but since the application was in a deployed state, maybe it didnt matter that the json file got created. The connector was not able to find the file. When I re triggered the application with that created file the bigquery connector worked. So not sure what is the correct way to get this to work.

Additional details: I am using Mule 4 application. BigQuery connector version: enter image description here xml code for connector

<?xml version="1.0" encoding="UTF-8"?>

<mule xmlns:file="http://www.mulesoft.org/schema/mule/file" xmlns:bigquery="http://www.mulesoft.org/schema/mule/bigquery"
    xmlns:http="http://www.mulesoft.org/schema/mule/http"
    xmlns:tls="http://www.mulesoft.org/schema/mule/tls" xmlns:kafka="http://www.mulesoft.org/schema/mule/kafka" xmlns:salesforce="http://www.mulesoft.org/schema/mule/salesforce" xmlns="http://www.mulesoft.org/schema/mule/core" xmlns:doc="http://www.mulesoft.org/schema/mule/documentation" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd
http://www.mulesoft.org/schema/mule/salesforce http://www.mulesoft.org/schema/mule/salesforce/current/mule-salesforce.xsd
http://www.mulesoft.org/schema/mule/kafka http://www.mulesoft.org/schema/mule/kafka/current/mule-kafka.xsd
http://www.mulesoft.org/schema/mule/tls http://www.mulesoft.org/schema/mule/tls/current/mule-tls.xsd
http://www.mulesoft.org/schema/mule/http http://www.mulesoft.org/schema/mule/http/current/mule-http.xsd
http://www.mulesoft.org/schema/mule/bigquery http://www.mulesoft.org/schema/mule/bigquery/current/mule-bigquery.xsd
http://www.mulesoft.org/schema/mule/file http://www.mulesoft.org/schema/mule/file/current/mule-file.xsd">
    <salesforce:sfdc-config name="Salesforce_Config" doc:name="Salesforce Config" doc:id="63e49cc6-1939-4390-a7ff-5b7fbf3fd8a8" >
        <salesforce:basic-connection username="simba@santa.com" password="126531" securityToken="askdjhbkatfekw" url="https://santacorp.my.salesforce.com/services/Soap/u/59.0"/>
    </salesforce:sfdc-config>
    <salesforce:sfdc-config name="sandbox" doc:name="Salesforce Config" doc:id="fd9cec1d-ae93-47cb-9268-29511623e974" >
        <salesforce:basic-connection username="santa@banta.com" password="12368512" securityToken="weurygwiqeyrgw" url="https://santacorp--dev.ramen.com/services/Soap/u/59.0"/>
    </salesforce:sfdc-config>
    <http:listener-config name="HTTP_Listener_config" doc:name="HTTP Listener config" doc:id="4e55c859-34b9-4c81-b94d-b1ceef77fcd0" basePath="/api" >
        <http:listener-connection host="0.0.0.0" port="8083" />
    </http:listener-config>
    <bigquery:config name="Google_bigquery_Config" doc:name="Google_bigquery Config" doc:id="9b28d71f-70f8-4206-9d59-da3b16d9acad" >
        <bigquery:connection projectId="eng-sbx" jsonServiceAccountKeyFile="test.json"/>
    </bigquery:config>
    <file:config name="File_Config" doc:name="File Config" doc:id="aa04a618-11b6-4a1b-94b8-0ba4c0e2d873" >
        <file:connection />
    </file:config>

The json file should have contents like so

{
    "type": "service_account",
    "project_id": "sandbox-control",
    "private_key_id": "123y1t23uy12r3uy12tf3",
    "private_key": "-----BEGIN PRIVATE KEY-----********-----END PRIVATE KEY-----\n",
    "client_email": "email.iam.gserviceaccount.com",
    "client_id": "12371862531",
    "auth_uri": "https://accounts.google.com/o/oauth2/auth",
    "token_uri": "https://oauth2.googleapis.com/token",
    "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
    "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/something.iam.gserviceaccount.com"
  }
Solution

  • From the limited information in the question it seems that you are using a third party connector (Google BigQuery Connector 2.2.0 from Great Software Laboratory) not the Google BigQuery connector developed by MuleSoft which currently at version 1.1.5. Don't get confused by the naming and versioning, these are two completely different connectors from two different companies.

    It is unlikely that trying to write the JSON file dynamically in the application will work. The file is used at configuration time when the application is started to configure the connector. A flow is executed at execution time after the application has started. Usually a connector would already have read the file at execution time. Even if it worked the file would be in clear text, defeating the intention of encrypting it.

    One option is to migrate to the MuleSoft Google BigQuery connector that uses configurations that don't depend on a file so you should be able to encrypt information and use the property placeholders directly. You'll need to configure either JWT or OAuth2 authentications.