Search code examples
command-lineterminalgoogle-bigquerybq

Export Google BigQuery datasets in a single project as list in bq command-line?


I've looked at this resource, but it's not quite what I need. This question is what I want to accomplish, but I want to run it in the BQ terminal.

For instance, in the past I've exported table information as a .json in bq command-line as so:

bq show --schema --format=prettyjson Dataset.TableView > /home/directory/Dataset.TableView.json

This gives a prettyjson of Table information of a specified dataset in a set project. I would like to just have a .csv (or any type of list) of all dataset names in the project. But I can't figure out how to change that command-line appropriately to output what I want.


Solution

  • In order to further contribute to the community, as an alternative to @DanielZagales answer, using the bq command line. According to the documentation, you can use the bq ls to list all the datasets in a project. Such as follows,

    bq ls -a --format=pretty --project_id your-project-id

    The flag -a is short for --all, which guarantees that all the datasets will be included in the list. The flag --format=pretty will output the list as a table format, you can use other formatting such as described here. Furthermore, you can also filter the datasets which match an expression with --filter labels.key:value or set the maximum number of results with --max_results or -n.

    Note: you can also list all the tables within a dataset, such as described here.