I'm working on an API gateway-ish server, which supports user and groups.
I have an API endpoint something like the following.
/authorization/users/1?resource=users
basically, it's asking "Can this user 1 have access to 'users'?".
I would like to cache "/authorization/users/1?resource=users" in Varnish.
A permission can be set user level or group level. Each users belongs to at least one group.
User level cache invalidation is easy since I just need to send a PURGE request to a single URL.
When it comes to groups, it's complicated. A group can have over 50000 users. How do I invalidate those users?
Looking at https://www.varnish-software.com/blog/advanced-cache-invalidation-strategies, using X-Article-ID might be a good solution. My concern is that..how does it work with a large # of objects? Is there going to be a huge CPU usage? How fast can it handle 50000 objects?
Are there any better ways?
Using varnish ban will put the request you want to ban into the varnish ban list. each request is checked if it is in the ban list.
if the object is in the varnish cache with a timestamp that is older than an the item in the ban list. Then the item will be removed from the cache and a new fresh copy will be requested from the backend.
On top of this varnish also uses a process called the "ban lurker" this removes the items in the ban list pro-actively from the varnish cache. How fast this is done can be configured, for more information about this please check https://www.varnish-software.com/blog/ban-lurker
Personaly i did not have any issues with cpu and memory usage when using this type of varnish bans. But this all depends on how often an item is added to the ban list and how advanced the regex is you are using to ban the pages.