Search code examples
google-bigquerydatasetkaggle

How to use Kaggle datasets in BigQuery?

Is it possible to query Kaggle data directly from BigQuery, cloud to cloud?

Downloading from Kaggle and creating a table in BigQuery is one way, as recorded in this GitHub repository.

Is there a way to run a query in BigQuery

  • using a possibly existing Kaggle public project in BigQuery, and thus
    ↳ selecting FROM `kaggle-datasets-project.dataset.table`?

  • or, alternatively, connecting to the Kaggle dataset in BigQuery?

Solution

  • It is currently not possible to query a public kaggle dataset directly in the BigQuery UI console. You'll have to download the dataset and store it as a native dataset to be able to query it directly.

    Alternatively, Kaggle data can be stored in external sources like Drive or Google Cloud Storage and can be queried directly on the BQ console.

    So to summarize, You can query a BQ dataset on kaggle kernel, however public kaggle datasets cannot be queried directly on BQ console.