Search code examples
phpmysqlsqlsqlperformance

optimise mysql query with LIKE operator for 10k records


Note:

So as I figured, the real problem is because of the IN clause I'm using for tagids. Changing the portion of query for text search didn't help much. Any idea how to improve the query?

The query takes too long while running on the server. Here Partha S is a search item entered by user. The table contacts contains personal information , tags contains category name and id; and contacts2tags table contains contactid and tagid with values similar to id in contacts and tags respectively.

    SELECT *
    FROM
    (
    SELECT *,
     IF
    (
     first_name LIKE 'Partha S'
    OR last_name LIKE 'Partha S'
    OR phone_number LIKE 'Partha S'
    OR mobile_number LIKE 'Partha S'
    OR email_address LIKE 'Partha S'
    OR address LIKE 'Partha S'
    OR organization LIKE 'Partha S'
    OR other LIKE 'Partha S'
    OR sector LIKE 'Partha S'
    OR designation LIKE 'Partha S'
    OR concat ( first_name,  ' ',  last_name ) LIKE 'Partha S'
    OR concat ( last_name,  ' ',  first_name ) LIKE 'Partha S',
     1,
     0 )
     as exact,
     IF
    (
    (
     first_name LIKE '%Partha%'
    OR last_name LIKE '%Partha%'
    OR phone_number LIKE '%Partha%'
    OR mobile_number LIKE '%Partha%'
    OR email_address LIKE '%Partha%'
    OR address LIKE '%Partha%'
    OR organization LIKE '%Partha%'
    OR other LIKE '%Partha%'
    OR sector LIKE '%Partha%'
    OR designation LIKE '%Partha%' )
    AND
    (
     first_name LIKE '%S%'
    OR last_name LIKE '%S%'
    OR phone_number LIKE '%S%'
    OR mobile_number LIKE '%S%'
    OR email_address LIKE '%S%'
    OR address LIKE '%S%'
    OR organization LIKE '%S%'
    OR other LIKE '%S%'
    OR sector LIKE '%S%'
    OR designation LIKE '%S%' )
    ,
     1,
     0 )
     as normal
    FROM contacts
    WHERE id in
    (
    SELECT DISTINCT contacts.id
    from contacts INNER
    JOIN contacts2tags ON contacts.id = contacts2tags.contactid
    WHERE ( tagid in ( 178 ) ) )
     )
     d
    WHERE exact = 1
    OR normal = 1
    ORDER BY exact desc,
     last_name asc LIMIT 0,
     20

UPDATE: As per the suggestions, I removed the LIKE operator for exact search, and used MATCH(..) AGAINST(..) instead of LIKE in the latter case. While the first change did improve the performance a little, but using MATCH() AGAINST() didn't change the execution time surprisingly. Here's the updated query. PS I tried using both MATCH(all cols) AGAINST(search item) and MATCH(single cols) AGAINST (search item) combined with OR. Please suggest. thanks

     SELECT *
    FROM
    (
    SELECT *,
     IF
    (
         first_name ='Partha S'
       OR last_name ='Partha S'
       OR phone_number ='Partha S'
       OR mobile_number ='Partha S'
       OR email_address = 'Partha S'
       OR address ='Partha S'
       OR organization ='Partha S'
       OR other ='Partha S'
       OR sector ='Partha S'
       OR designation ='Partha S'
       OR concat ( first_name,  ' ',  last_name ) ='Partha S'
       OR concat ( last_name,  ' ',  first_name ) ='Partha S',
       1,
       0 )
      as exact,
       IF
      ( match(first_name,last_name,phone_number,mobile_number,email_address,  address,organization,other,sector,designation) against( 'Partha')                 
    OR  match(first_name,last_name,phone_number,mobile_number,email_address,address,organization,other,sector,designation) against( 'S')


    ,
    1,
    0 )
     as normal
    FROM contacts
    WHERE id in
    (
    SELECT DISTINCT contacts.id
    from contacts INNER
    JOIN contacts2tags ON contacts.id = contacts2tags.contactid
    WHERE ( tagid in ( 178 ) ) )
     )
     d
    WHERE exact = 1
    OR normal = 1
    ORDER BY exact desc,
     last_name asc LIMIT 0,
      20

Solution

  • One optimization is that in the exact case, you don't need to use LIKE (you should only use it with the wildcard - %).

    Another thing that you can do to make the things faster is adding an INDEX to the fileds you're going to be searching in.

    Also, only if you're using MyISSAM as your storage engine (for that table) you can use full text search like this

    SELECT * FROM normal WHERE MATCH(title, body) AGAINST ('Queried_string')

    first_name LIKE '%S%'
    OR last_name LIKE '%S%'
    OR phone_number LIKE '%S%'
    OR mobile_number LIKE '%S%'
    OR email_address LIKE '%S%'
    OR address LIKE '%S%'
    OR organization LIKE '%S%'
    OR other LIKE '%S%'
    OR sector LIKE '%S%'
    OR designation LIKE '%S%' )
    

    seems to be bringing very little value to the whole process.

    Hope this helps.