Search code examples
javadatabasepostgresqlresultset

Check if query result has changed, in order to not re-query for everything?


I am developing a java application that loads certain things from a database, such as client records and product info. When the user navigates to say the 'products' tab, I query for products in the database and update a table with that information. I am wondering if there is a way to see if the query results have changed since the last check, in order to avoid querying and loading all info from the database, and instead just load updates. Is there a way to do this, or perhaps just load changes only from a query into my table list? My goal is to make the program run faster when switching between tabs.


Solution

  • I am wondering if there is a way to see if the query results have changed since the last check

    Stated differently, you want a way to automatically answer the question “is this the same result?” without retrieving the entire result.

    The general approach to this problem would be to come up with some fast-to-query proxy for the entire state of the result set, and query that instead.

    Once you have determined a stable fast computation for the entire result set, you can compute that any time the relevant data changes; and only poll that stored proxy to see whether the data has changed.


    For example, you could say that “the SHA-256 hash of fields lorem, ipsum, and dolor” is your proxy. You can now:

    • Implement that computation inside the database as a function, maybe products_hash.

    • Create a latest_products_hash table, that stores created timestamp and products_hash that was computed at that time.

    • In your application, retrieve the most recent record from latest_products_hash and keep it for reference.

    • In the database, have a scheduled job, or a trigger on some event you decide makes sense, that will compute and store the products_hash in latest_products_hash automatically without any action from the application.

    • To determine whether there have been updates yet, the application will query the latest_products_hash table again and compare its most recent record with the one the application stored for reference.

    • Only if the latest_products_hash most-recent value is different, then query the products table and get the full result set.

    That way, the application is polling a much faster query (the most-recent record in latest_products_hash) frequently, and avoiding the full products query until it knows the result set will be new.