Search code examples
apache-flinkpyflink

PyFlink - Issue with UNNEST: query uses an unsupported SQL feature?

I am trying to flatten an array using UNNEST function in the Table API.

Am I doing something wrong or it is not a supported function ? This page suggests it though: https://ci.apache.org/projects/flink/flink-docs-master/dev/table/sql/queries.html

Thanks !

Code

Python udf

@udf(input_types=DataTypes.STRING(), result_type=DataTypes.ARRAY(DataTypes.STRING()))
def explode(s):
    return 3*[s]

t_env.register_function("explode", explode)

Processing

tab = t_env.from_path('mySource').select("id, explode(dummy) as dummy_list")
t_env.register_table("temp_table", tab)
t_env.sql_query("SELECT t.item as dummy_item FROM UNNEST(select dummy_list from temp_table) AS t(item)").insert_into("mySink")

Execution

t_env.execute("dummy_unnest")

Error

TableException: Cannot generate a valid execution plan for the given query: 

LogicalProject(dummy_item=[$0])
  Uncollect
    LogicalProject(EXPR$0=[org$apache$flink$table$functions$python$PythonScalarFunction$908596b4671476ee325743dba92ed6c7($1)])
      LogicalTableScan(table=[[default_catalog, default_database, mySource]])

This exception indicates that the query uses an unsupported SQL feature.
Please check the documentation for the set of currently supported SQL features.
Solution

  • I think you can change the query to

    select id, dummy_item from temp_table CROSS JOIN UNNEST(dummy_list) AS t (dummy_item)