Search code examples
sqlsearchfull-text-searchcontainsfreetext

Optimizing an SQL search using a column for keywords and a variable as the text to be searched


Here's my quandary. I have a variable that contains a paragraph of text, and I have a column full of keywords. I want to search each of the keywords contained in the column against the entirety of the text contained within my variable.

The only way that I know how to do this is to select each row of the column one at a time, and then use the LIKE operator with wildcards on each side to see if the keyword from the column is found anywhere in the text within the variable. Every way that I try and do this ends up seeming GROSSLY inefficient.

Another way of looking at this is, if I could reverse the order of an operator like FREETEXT to do something like FREETEXT(@input, keywords), that would be awesome. But I just can't seem to figure out a way to do it simply and efficiently like that.

Any tips on how to do this as efficiently as possible? I am very grateful for the help!

Edit:

Here is my stored procedure, for reference:

GO
    @input varchar(1000),
    @debug varchar(25) output

AS
BEGIN

    SELECT TOP 1 @debug = kw.keyword
    FROM (SELECT @input input) bigstring
    INNER JOIN table1 kw 
    on bigstring.input LIKE '%' + kw.keyword + '%'

END

Solution

  • Here are two ways depending on what you looking to do (assuming SQL 2005+ from the FREETEXT keyword). The first select after the sample data returns the index of the keyword (filter out zeros if you don't want keywords that aren't found). The second just checks for the existance

    Declare @keywords  as table (keyword varchar(50))
    INSERT INTO @keywords 
    VALUES ('quandary'),
           ('variable'),
           ('paragraph'),
           ('Narwhal')
    
    
    DECLARE @input as varchar(max)
    SET @input = 'Heres my quandary. I have a variable that contains a paragraph of text, and I have a column full of keywords. I want to search each of the keywords contained in the column against the entirety of the text contained within my variable'
    
    
    SELECT keyword, CHARINDEX(keyword, @input , 0)
    FROM @keywords
    
    SELECT kw.keyword
    FROM 
    (SELECT @input input) bigstring
    INNER JOIN @keywords kw 
    on bigstring.input like '%' + kw.keyword + '%'
    
    
    
    (4 row(s) affected)
    keyword                                            
    ----------------------- --------------------
    quandary                10
    variable                29
    paragraph               54
    Narwhal                 0
    
    (4 row(s) affected)
    
    keyword
    -----------------------
    quandary
    variable
    paragraph
    
    (3 row(s) affected)
    

    I wouldn't be surprised if there was a CROSS APPLY solution as well

    Update Getting only the first keyword out as an out param

    Data

    CREATE TABLE table1 (keyword varchar(50))
    INSERT INTO table1 
    VALUES ('quandary'),
           ('variable'),
           ('paragraph'),
           ('Narwhal')
    
    GO
    

    Proc

    CREATE  proc testKeyword
            @input varchar(1000),
            @debug varchar(25) output
    
        AS
        BEGIN
    
            SELECT TOP 1 @debug = kw.keyword
            FROM (SELECT @input input) bigstring
            INNER JOIN table1 kw 
            on bigstring.input LIKE '%' + kw.keyword + '%'
    
        END
    

    Test

    DECLARE @debug varchar(25)
    EXEC testKeyword 'Heres my quandary. I have a variable that contains a paragraph of text, and I have a column full of keywords. I want to search each of the keywords contained in the column against the entirety of the text contained within my variable',
               @debug out 
    
    SELECT @debug 
    
    outputs 
    
    
    -------------------------
    quandary
    
    (1 row(s) affected)