Search code examples
google-cloud-platformgoogle-bigquerygoogle-python-api

Google Big Query view limitation on Python API


I am trying to create a view using the Python API for Google Big Query. I have the free trial and all the authentication setup. Whenever I create a view using the API only 25 rows are generated in the view, however when I create a view from the big query website all the rows are shown (3006). Is there a limit on rows when using API or might there be a problem in my code. This is the query that I am using (same was used in API and web):

query = "SELECT DISTINCT author, subreddit FROM `%s` WHERE subreddit = 'The_Donald'" %(TABLE+DATES[7])

Solution

  • I've just created a view with the Python Client Libraries using the following code:

    from google.cloud import bigquery
    client = bigquery.Client()
    project = 'bigquery-samples'
    source_dataset_id = 'reddit'
    source_table_id = 'full'
    shared_dataset_ref = client.dataset('my_dataset')
    view_ref = shared_dataset_ref.table('my_shared_view')
    view = bigquery.Table(view_ref)
    sql_template = (
        'SELECT DISTINCT author,subreddit_id FROM `{}.{}.{}` WHERE subreddit_id LIKE "%t5%"')
    view.view_query = sql_template.format(
        project, source_dataset_id, source_table_id)
    view = client.create_table(view)  # API request
    
    print('Successfully created view at {}'.format(view.full_table_id))
    

    And my view has 1359016 rows, verified by doing a:

    SELECT COUNT(*) FROM `my_dataset.my_shared_view` 
    

    Hope it helps.