Search code examples
google-bigqueryapache-beamdataflow

Apache Beam - Reading Big Query as Generic Records (Avro format)


I am trying to read BigQuery data on Dataflow and I would like to read it as Generic Record type instead of TableRows. Is there any way to directly load as Generic records without needing to convert from TableRows to Generic Records?


Solution

  • SchemaAndRecord already contains GenericRecord. So can you do something like this?

    BigQueryIO.read(
            (SerializableFunction<SchemaAndRecord, GenericRecord>) input -> input.getRecord());