We are testing Apache Impala and have noticed that using GROUP BY and LIKE together works very slowly -- separate queries work much faster. Here are two examples:
# 1.37s 1.08s 1.35s
SELECT * FROM hive.default.pcopy1B where
(lower("by") like '%part%' and lower("by") like '%and%' and lower("by") like '%the%')
or (lower(title) like '%part%' and lower(title) like '%and%' and lower(title) like '%the%')
or (lower(url) like '%part%' and lower(url) like '%and%' and lower(url) like '%the%')
or (lower(text) like '%part%' and lower(text) like '%and%' and lower(text) like '%the%')
limit 100;
# 156.64s 155.63s
select "by", type, ranking, count(*) from pcopy where
(lower("by") like '%part%' and lower("by") like '%and%' and lower("by") like '%the%')
or (lower(title) like '%part%' and lower(title) like '%and%' and lower(title) like '%the%')
or (lower(url) like '%part%' and lower(url) like '%and%' and lower(url) like '%the%')
or (lower(text) like '%part%' and lower(text) like '%and%' and lower(text) like '%the%')
group by "by", type, ranking
order by 4 desc limit 10;
Why does this issue occur, and are there any workarounds?
There is a basic difference between two queries.
1st Query
Main points:
WHERE
clause, it is marked as completed and 100 records will be returned.2nd Query
Main points:
GROUP BY
clause.WHERE
clauseGROUP BY
clause.ORDER BY
clause.So the queries provided by you might look similar but they are totally different and solves all together different purpose.