Search code examples
jqueryperformancefilteringlarge-data

Efficient jQuery live search with large (70,000+ item) data set


I'm creating a new form for a page on my WordPress site, and on one of the inputs on this form I'd like to add a live search feature using jQuery that lists possible options based on what the user has typed so far. The data set I'm filtering is a very large JSON object stored in my theme that contains over 70,000 key/value pairs. Because I'm currently looping through the entire object on each keyup, the live search I've implemented is very slow and often freezes.

What's the most efficient way to perform a live search on large data sets using jQuery?


Solution

  • jQuery is for DOM manipulation, and will not be useful here.

    JavaScript can solve this problem, however, as the data you serve increases, it will also become increasingly apparent that it should be accessible through a server either in cached memory or a database.

    The size of your JSON can be problematic to page load speeds, and for certain devices with small memory availability, may mean that the page does not load at all.

    That aside, what you need to do is build a lookup structure in the keys. This design pattern is called a "trie". Without any sort of example code to go on, implementing here for the sake of example would be a waste of time.

    I suggest you look into the linked wikipedia article for some education on the general subject and then look for a library on github which provides some sort of trie functionality for json searching; unless, of course, if you feel confident enough to implement one yourself.