Search code examples
cachingvarnishvarnish-vcl

Malloc and file varnish storage together improves performance?

Need your suggestions with respect to varnish memory storage

Currently we run the varnish community version 6.0.1 with the following arguments

varnishd -F -j unix,user=nobody -a :6081 -T localhost:6082 -f /etc/varnish/default.vcl -s file,/opt/varnishdata/cache,500G
  • Would reducing the allocated cache size of 500G help here? Since we know the fact that cache size wont go above 30G (including head room)

(Object size ranges from few byte to at max 1MB, all being json type)

  • We are aware that file storage backend isnt the best one hence wanted to know if malloc would help here? Or can we use both in some ratio considering that hot pages are stored in memory?

(it is not possible to keep everything in memory at the moment considering cache size since max RAM is 32G and memory swapping can be costly IMO)

Other topics

  • At the moment we use xkey to inavlidate cache based on tags, would ykey benefit in any sense?

  • Whenever there is rise in session herd / session closed, we see issues with varnish (response time and backend fetch failed). Any specific thing that we should check?

varnish session issue

Any other suggestions or improvements are welcome

Thanks

Solution

  • Avoid using the file stevedore

    The file stevedore is notoriously bad when it comes to disk fragmentation. We would recommend to avoid using it. However, if it's an absolute must, you can combine memory & file storage.

    By setting beresp.storage to the right stevedore, you can decide on a per backend response basis which storage to use.

    Unless you name your stevedores via the varnishd -s parameter, Varnish will use its own naming scheme: s0 for the first stevedore that is defined, s1 for the next, etc ...

    Horizontally scaling memory-only Varnish servers

    If you want to have more cache storage than the available memory in your server, you can create a 2-tier architecture that scales horizontally:

    • At the top level you have edge servers that serve hot objects from memory
    • The edge servers use a shard director to distribute cache misses to the lower tier
    • The lower tier of Varnish servers will store all the content, even the long tail objects
    • Thanks to sharding objects will be consistently fetched from one tier 2 Varnish server which facilitates horizontal scalability

    See https://varnish-cache.org/docs/6.0/reference/vmod_generated.html#sharding for more information about the sharding director.

    Use the Massive Storage Engine

    The most reliable solution for your storage problem is to leverage Varnish Software's Massive Storage Engine.

    It's a proprietary stevedore that combines disk storage and memory. It was built from the ground up to bypass the typical limitations of the file stevedore.

    See https://docs.varnish-software.com/varnish-cache-plus/features/mse/ more more information on MSE.

    It's not an open source component, but it's insanely powerful and powers a lot of private CDNs of telcos, huge OTT video streaming platforms and many other web platforms.

    Benefits:

    • Can store petabytes of data
    • Has a custom page caching mechanism to store objects in memory
    • Highly configurable at every level
    • Built-in anti-fragmentation algorithms
    • Dynamic storage memory sizing based on the total memory footprint of Varnish

    Xkey vs Ykey

    xkey is an open source VMOD that allows the creation of secondary keys. The secondary keys can be used to invalidate multiple objects at once.

    While relatively powerful, functionality is quite limited.

    ykey is proprietary VMOD that is developed by Varnish Software. It's only available on Varnish Enterprise subscriptions. It's feature set and API is a lot broader.

    See https://docs.varnish-software.com/varnish-cache-plus/vmods/ykey/ for more information on ykey.

    ykey was developed because xkey didn't fully support Massive Storage Engine or at least didn't perform well on large volumes of data.

    Session issue

    Currently I cannot comment on your Varnish session issues because there's not enough information/context.

    I suggest you open up a separate question on StackOverflow and provide more details.

    Please attach the complete output of varnishstat to the question and if possible a complete varnishlog transaction that shows the backend fetch failed issue.

    That might allow me to figure out what's going on. As long as you tag the question with #varnish I'll be able to monitor it.

    Conclusion

    A lot of your challenges can be tackled by using Varnish Enterprise instead of using the open source version.

    If you want to try out Varnish Enterprise in the cloud without having to commit to an upfront license cost, you can have a look at https://www.varnish-software.com/developers/downloads/#cloud-images .

    It explains how to run it in the cloud and pay the license fee on a per-hour basis. On AWS there's even a Developer Edition.