Search code examples
google-bigquerygoogle-cloud-platformstackdrivergoogle-cloud-logging

Why do BigQuery audit logs that use the V2 format, have column names with "_v1_"?


I switched on audit logs via the console using the new V2 format and created a sink to export them to back to BigQuery for analysis:

enter image description here

The tables that are exported to BigQuery all have "v1" in the column names, even though I selected V2 format:

enter image description here

Then, when I try to query the table, because the column names exceed 128 chars, it throws an error:

enter image description here

Why are the audit logs being exported using the v1 naming schema, and how do I get around the column names being over the 128 character limit?


Solution

  • how do I get around the column names being over the 128 character limit?

    I believe the problem is not in referencing long named column path but rather in output column's name
    So, to resolve the issue within Legacy SQL - you should provide alias that is compliant with name cnvention.
    Or just use Standard SQL - in this case the alias is by default the name of the leaf field (in this case totalBilledBytes)

    #legacySQL  
    SELECT  
      protopayload_google_cloud_audit_auditlog.
        servicedata_google_cloud_bigquery_logging_v1_auditdata.
        jobCompletedEvent.
        job.
        jobStatistics.
        totalBilledBytes AS totalBilledBytes   
    FROM [yourTable]
    

    or

    #standardSQL   
    SELECT  
      protopayload_google_cloud_audit_auditlog.
        servicedata_google_cloud_bigquery_logging_v1_auditdata.
        jobCompletedEvent.
        job.
        jobStatistics.
        totalBilledBytes    
    FROM `yourTable`