Save CVS from the web to Apify Dataset

Question

I am trying to get some CVS data from google sheet and store it into an Apify dataset.

const Apify = require('apify');
const request = require('request-promise');

Apify.main(async () => {

  var URL = "https://docs.google.com/spreadsheets/d/1-auXklWqHQ-jj6AXymMPa7FLtP1eYGJGF3rprxuWitk/gviz/tq?tqx=out:csv";

  const html = await request(URL);
  console.log('My output:');
  console.log(html);

  await Apify.setValue('OUTPUT', html);

  const namedDataset = await Apify.openDataset();

  await namedDataset.pushData(html);
});

Here is error message:

2020-01-01T16:43:21.501Z My output:
2020-01-01T16:43:21.510Z "city","country"
2020-01-01T16:43:21.512Z "Berlin ","Germany"
2020-01-01T16:43:21.513Z "Los Angeles","United States"
2020-01-01T16:43:21.514Z "Melbourne","Australia"
2020-01-01T16:43:21.516Z "Sydney","Australia"
2020-01-01T16:43:21.517Z "London","United Kingdom"
2020-01-01T16:43:21.519Z "New York City","United States"
2020-01-01T16:43:21.614Z ERROR: The function passed to Apify.main() threw an exception: (error details: type=invalid-parameter)
2020-01-01T16:43:21.616Z   ApifyClientError: Parameter "data" of type Array | Object must be provided
2020-01-01T16:43:21.617Z     at exports.checkParamOrThrow (/usr/src/app/node_modules/apify-client/build/utils.js:222:15)
2020-01-01T16:43:21.619Z     at Dataset.pushData (/usr/src/app/node_modules/apify/build/dataset.js:222:34)
2020-01-01T16:43:21.620Z     at Apify.main (/usr/src/app/main.js:16:22)
2020-01-01T16:43:21.621Z     at process._tickCallback (internal/process/next_tick.js:68:7)

Answer 1

A more elegant solution would be using our Google Sheets actor.

const Apify = require('apify');

Apify.main(async () => {
  const spreadsheetId = '1-auXklWqHQ-jj6AXymMPa7FLtP1eYGJGF3rprxuWitk';
  const sheetsActorInput = {
     mode: 'read',
     spreadsheetId,
  };
  const data = await Apify.call('lukaskrivka/google-sheets', sheetsActorInput);

  const namedDataset = await Apify.openDataset('my-dataset');
  await namedDataset.pushData(data);
});

The only disadvantage (also an advantage is some sense) is that you need to authorize in your first run but that is really simple.