Search code examples
sqlhivehiveqlanalytics

hive convert array<map<string, string>> to string


I have a column in hive table which type is array<map<string, string>>, I am struggling how to convert this column into string using hql?

I found post here Convert Map<string,string> to just string in hive to convert map<string, string> to string. However, I still failed to convert array<map<string, string>> to string.


Solution

  • Building off of the original answer you linked to, you can first explode the array into the individual maps using posexplode to maintain a positition column. Then you can use the method from the original post, but additionally group by the position column to convert each map to a string. Then you collect your maps into the final string. Here’s an example:

    with test_data as (
    select stack(2,
                 1, array(map('m1key1', 'm1val1', 'm1key2', 'm1val2'), map('m2key1', 'm2val1', 'm2key2', 'm2val2')),
                 2, array(map('m1key1', 'm1val1', 'm1key2', 'm1val2'), map('m2key1', 'm2val1', 'm2key2', 'm2val2'))
                ) as (id, arr_col)
    ),
    map_data as (
    select id, arr_col as original_arr, m.pos as map_num, m.val as map_col
      from test_data d
           lateral view posexplode(arr_col) m as pos, val
    ),
    map_strs as (
    select id, original_arr, map_num,
           concat('{',concat_ws(',',collect_set(concat(m.key,':', m.val))),'}') map_str
      from map_data d
           lateral view explode(map_col) m as key, val
     group by id, original_arr, map_num
     )
     select id, original_arr, concat('[', concat_ws(',', collect_set(map_str)), ']') as arr_str
     from map_strs
     group by id, original_arr;