Search code examples
ruby-on-railssearchfull-text-searchthinking-sphinxsemantic-analysis

Rails Searching (Full text vs Semantic) for Food Products


I'm trying to build a search interface for a grocery store in Rails. I have several grocery store products stored in my database, but I'm struggling to find a good way to search through them.

So far, I've used Thinking Sphinx to do full text searching of the products. This works well for some items, such as 'eggs', but has several problems for other searches, such as 'lime'.

For example, when I search for 'lime', I get the following returned items:

  • Diet Sierra Mist Lemon Lime Fridge Mate Soda
  • G2 Lemon Lime Low Calorie Electrolyte Beverage
  • Deer Park Berry Lime Sparkling Water
  • Ocean Spray Cocktail With Lime Cranberry Juice
  • Vintage Seltzer Water Limn Lime
  • Coke Diet With Lime Soda Contour Bottle
  • Coke Diet With Lime Soda Fridge Pack
  • Diet Sierra Mist Lemon Lime Fridge Mate Soda
  • Refreshe Soda Diet Lemon Lime Fridge Pack
  • Refreshe Soda Lemon Lime
  • Refreshe Soda Lemon Lime Fridge Pack
  • Sierra Mist Natural Lemon Lime Soda
  • ... etc

All I really wanted was the actual fruit, which are listed as these products in my database:

  • Limes Large
  • Organic Limes
  • Limes Key Prepacked

How can I make my search more intelligent?

It's worth noting that I do have a lot of categorical data on each product. I essentially have an entire "food tree", where each product is a root node of the tree.

For the product 'Limes Large', for example, I have the following categories:

Fruits & Vegetables > Fresh Fruit > Citrus > Limes Large

How can I better use these categories to improve my search?

In my research, I am starting to believe I need to use semantic searching instead of full text searching. I came across the Picky gem for Ruby, which looks promising, but I'm not sure if I have the right approach.

Can semantic searching help me perform a better search? Is Picky a good fit for data categorized in this way? Any other insights? Any guidance would be really appreciated. Thanks in advance.


Solution

  • As for the 'categories' you might want to think about using a tagging system also. There are a number of tagging gems out there. That way what is being searched on is roped off from any other data structures or 'trees' you have created.

    In order to make it easy you could have it so that the tags that are automatically put on an item at creation are based on its location in the tree and or the items name. This makes it so could be easily changed if the need comes up, but at the same time have the ease of the tree system, while still having the tag system's power of narrow searches.

    A good place to start would be bellow.

    https://github.com/mbleigh/acts-as-taggable-on#readme

    That way you can have something like Diet Sierra Mist Lemon Lime Fridge Mate Soda not tagged with Lemon so it wouldn't come up, but you could still tag something like Sunny D tagged with "Orange Juice." All while not making it to hard on your backend user by having default tags!

    As for actually search it might just be worth it to look into using a google search system, while paying for no ads. You could use the tags as your keywords to narrow down the results. I believe this also helps with SEO while at the same time putting the search in google's hands, which means support is basically guaranteed.