Search code examples
prestotrino

RowType support in Presto

Question for those who knows Presto API for plugins.

I implement BigQuery plugin. BigQuery supports struct type, which could be represented as RowType class in Presto.

RowType creates RowBlockBuilder in RowType::createBlockBuilder, which has RowBlockBuilder::appendStructure method, which requires to accept only instance of AbstractSingleRowBlock class.

This means that in my implementation of Presto's RecordCursor BigQueryRecordCursor::getObject method I had to return something that is AbstractSingleRowBlock for field which has type RowType.

But AbstractSingleRowBlock has package private abstract method, which prevents me from implementing this class. The only child SingleRowBlock has package private constructor, and there are no factories or builders that could build an instance for me.

How to implement struct support in BigQueryRecordCursor::getObject? (reminding: BigQueryRecordCursor is a child of RecordCursor).

Solution

  • You need to assemble the block for the row by calling beginBlockEntry, appending the values for each column via Type.writeXXX with the column's type and then closeEntry. Here's some pseudo-code.

    BlockBuilder builder = type.createBlockBuilder(..);

builder = builder.beginBlockEntry();
for each column {
    ...

    columnType.writeXXX(builder, ...);
}
builder.closeEntry();

return (Block) type.getObject(builder, 0);

    However, I suggest you use the columnar APIs, instead (i.e., ConnectorPageSource and friends). Take a look at how the Elasticsearch connector implements it:

    https://github.com/prestosql/presto/blob/master/presto-elasticsearch/src/main/java/io/prestosql/elasticsearch/ElasticsearchPageSourceProvider.java https://github.com/prestosql/presto/blob/master/presto-elasticsearch/src/main/java/io/prestosql/elasticsearch/ElasticsearchPageSource.java

    Here's how it handles Row types:

    https://github.com/prestosql/presto/blob/master/presto-elasticsearch/src/main/java/io/prestosql/elasticsearch/decoders/RowDecoder.java

    Also, I suggest you join #dev channel on the Presto Community Slack, where all the Presto developers hang out.