How to load CSV data in CloudSQL using Cloud Functions in Node.js?

Question

I am receiving data in CSV file in a specific bucket. I have to load that data in corresponding CloudSQL table. I have following queries on that:

1. Do I have an option of calling any api or gcloud command or CURL command in my cloud function code which takes CSV as an input and performs the inserts in DB?
2. If the above option is not possible then is there any performance optimised way available for preparing queries and firing those in CloudSQL as the data would be huge and I suppose it would take a lot of time to load this data.

Answer 1

The documentation on importing a CSV into CloudSQL contains this sample of a curl call:

ACCESS_TOKEN="$(gcloud auth application-default print-access-token)"
curl --header "Authorization: Bearer ${ACCESS_TOKEN}" \
     --header 'Content-Type: application/json' \
     --data '{"importContext":
                  {"fileType": "CSV",
                   "uri": "gs://<BUCKET_NAME>/<PATH_TO_DUMP_FILE>",
                   "database": "<DATABASE_NAME>",
                   "csvImportOptions":
                        {"table": "<TABLE_NAME>"}}}' \
   -X POST \
   https://www.googleapis.com/sql/v1beta4/projects/[PROJECT-ID]/instances/[INSTANCE_NAME]/import

Some important things to note:

• curl is just a way to perform a HTTP call from the command line. In Cloud Functions you'd perform the equivalent HTTP call from your code, for example with fetch() in Node.js.
• the CSV file to be imported has to be in a Google Cloud Storage bucket.
• Cloud Functions run a Google environment under the so-called application default credentials, so the ACCESS_TOKEN might not be needed. If they are needed after all, have a look here: https://www.npmjs.com/package/google-auth-library#application-default-credentials