Search code examples
google-bigquerygdelt

GDELT: count occurence of specific themes


I am trying to count how often the term "BITCOIN" occurs in the Themes column of the GDELT database, and then group the counts by date. Here is what I have so far:

SELECT DATE, SPLIT(RTRIM(Themes,';'),';') themes 

FROM `gdelt-bq.gdeltv2.gkg_partitioned` WHERE _PARTITIONTIME >= "2020-11-01 00:00:00" AND _PARTITIONTIME < "2020-11-07 00:00:00" 
#and (Themes like "%BITCOIN%")
#or (AllNames like "%bitcoin%" or AllNames like "%BITCOIN%")

and length(Themes) > 1
) select count(theme) cnt from nested, UNNEST(themes) as theme WHERE theme like "%BITCOIN%"

group by DATE

Is this the correct approach? Thanks!


Solution

  • Below is for BigQuery Standard SQL

    #standardsql
    select date(_partitiontime) date, count(theme) occurences
    from `gdelt-bq.gdeltv2.gkg_partitioned`, unnest(split(themes,';')) as theme 
    where _partitiontime >= "2020-11-01 00:00:00" and _partitiontime < "2020-11-07 00:00:00" 
    and lower(theme) like "%bitcoin%"
    group by date
    -- order by date  
    

    with output

    enter image description here