Search code examples
varnishvarnish-vcl

How to extract value from URL and check cache to load data in varnish

I have a scenario where my URL will be either contains a comma delimiter with value or without. i.e. /api/parameters/XXXXXXXXXX?tables=x0 or tables=x0;x1;x2.

now based on this URL I want to check in the varnish that, if URL contains multiple values as tables then separate that out and pass each table name in seperate URL (/api/parameters/XXXXXXXXXX?tables=x0, /api/parameters/XXXXXXXXXX?tables=x1, /api/parameters/XXXXXXXXXX?tables=x2) either to cache if miss then backend server.

then based on the response of this need to combine the result and return it to the client. my question here is:

  1. How to segregate the value from the URL and pass a modified URL to varnish cache or backend.
  2. after returning the result I want to return it as a combined JSON object in a sequence of which it was originally requested with a comma delimiter(i.e. x0 result;x1 result;x2 result).
Solution

  • It is possible to turn a single request into multiple subrequests in Varnish. Unfortunately this cannot be done with the open source version, only with the Enterprise version.

    vmod_http

    https://docs.varnish-software.com/varnish-cache-plus/vmods/http/ describes how you can perform HTTP calls from within Varnish using vmod_http.

    By sending HTTP requests to other URLs through Varnish, you can get multiple objects out of the cache and aggregate them into a single response

    No looping

    The fact that Varnish doesn't have loops makes matters a bit more complicated. You'll have so set an upper limit to the amount of values the tables querystring parameter has and you'll have to check the values using individual if-statements.

    Returning the combined JSON output

    Once you have fetched the results from the various URLs, you can create a JSON string and return it via return(synth(200,req.http.json)). Where req.http.json contains the JSON string.

    This will create a synthetic response.

    In Varnish Enterprise it is also possible to cache synthetic output. See https://docs.varnish-software.com/varnish-cache-plus/vmods/synthbackend/ to learn more about vmod_synthbackend.

    Varnish Enterprise disclaimer

    The solution I suggested in my answer uses Varnish Enterprise, the commercial version of Varnish. It extends Varnish capabilities with additional VMODs and features, which you can read about here. One easy way to try it out without upfront licensing payments, if you’re interested, is to spin up an instance on cloud infrastructure: