Search code examples
pythonsearchtrie

Using a Trie to search through tokenized sentences for phrases


I am looking for help on a portion of a program I am trying to develop. I recently implemented a naive version of a text searching program that stored tokenized sentences in a list, and searched for phrases using if statements. This process worked but took an extremely long time to complete. I found the Trie data structure and from my research it is capable of substantially cutting down my run-time. I was wondering if anyone has an idea about using a Trie to search for phrases in a list of tokenized sentences. And after matching, returning the entire sentence with the sought after key phrase in it. By the way all the text is in string format.


Solution

  • I use a library for python that allows you to create and search through Tries, if you wanted to just import the data structure here's a link to where you can download it. https://pypi.org/project/marisa-trie/