Search code examples
google-drive-apigoogle-bigquerygoogle-cloud-dataflow

Permissioning Dataflow to read a BigQuery table that is pointing to Drive?

BigQuery can read from Google Drive as a federated source. See here. I want to be able to read a table in BigQuery into my Dataflow pipeline that is pointing to a Drive document.

Hooking up BigQuery to the file in Drive works perfectly fine:

enter image description here

But, when I then try to read that table into my Dataflow pipeline I (understandably) get the following error:

No suitable credentials found to access Google Drive. Contact the table owner for assistance.

[..]
PCollection<TableRow> results = pipeline.apply("whatever",
BigQueryIO.Read.fromQuery("SELECT * from [CPT_7414_PLAYGROUND.google_drive_test]"))
.apply(ParDo.of(new DoFn<TableRow, TableRow>() {
[..]

How do I permission Dataflow to be able to read from a table in BigQuery that is pointing to Drive?

Solution

  • Dataflow does not currently support reading from a federated table backed by Drive, but this is coming soon.