varnish 4 grace time depending on backend health

Question

I'm confused about where to set the grace time in varnish 4. I've seen example VCL's setting the grace time in vcl_recv

sub vcl_recv {
    ...
    set req.http.grace = 60m;
    ...
}

others set it in vcl_hit

sub vcl_hit {
    ...
    set obj.grace = 60m;
    ...
}

and in the documentation (https://www.varnish-cache.org/docs/4.0/users-guide/vcl-grace.html) they set in vcl_backend_response

sub vcl_backend_response {
    ...
    set beresp.grace = 60m;
    ...
}

I've also seen examples where the grace time was set both in vcl_recv and vcl_backend_response.

Can anyone please explain why one would want to set it in one particular place?

In practice I want to set the grace time depending whether the backend is deemed healthy or not; intuitively I'd set it before varnish contacts the backend, so I'd set the grace time in vcl_recv and decide whether to server from cache in vcl_hit like this:

sub vcl_recv {
    ...
    if (std.healthy(req.backend_hint)) {
        set req.http.grace = 2m;
    } else {
        set req.http.grace = 60m;
    }
    ...
}

sub vcl_hit {
    if (obj.ttl >= 0s) {
        # A standard hit, deliver from cache
        return (deliver);
    }
    elsif (obj.ttl + obj.grace > 0s) {
        # page expired, serve from cache in the meantime
        return (deliver);
    } else {
        return (fetch);
    }
}

Is this the correct approach?

Answer 1

This is the result of my research sofar:

1. setting req.http.grace in vcl_recv doesn't help as this just defines a new header entry but otherwise varnish will ignore it
2. setting obj.grace in vcl_hit doesn't work as obj is read only in varnish 4
3. the only place the grace time can be set so that varnish recognizes it is in the subroutine vcl_backend_response in beresp.grace
4. because the grace time can only be set in vcl_backend_response setting different grace times depending on the backend health is not possible there as this function is only called when varnish fetches data from the backend after a cache miss, which is too late for what I want. I would need to set the grace time before the backend is contacted.

My solution for having different grace times depending on the backend health is this:

1. I'm setting the grace time to the maximum of the 2 grace times: "normal grace time" and "grace time when backend is sick"
2. whether to serve from cache or contact the backend is decided in vcl_hit; that's where I can emulate a 2nd grace time

Here is how my vcl_hit looks like

sub vcl_hit {
        if (obj.ttl >= 0s) {
                # A standard hit, deliver from cache
                return (deliver);
        }
        elsif (std.healthy(req.backend_hint)) {
                if (obj.ttl + 30m > 0s) {
                        # page expired within a limited grace time and backend
                        # is healthy: deliver from cache while cache is updated
                        # asynchronous
                        return (deliver);
                } else {
                        # page expired too long ago - fetch from backend
                        return (fetch);
                }
        }
        else {
                if (obj.ttl + obj.grace > 0s) {
                        # backend is not healthy - provide the page from cache
                        # during full grace time set in vcl_backend_response
                        return (deliver);
                } else {
                        # page expired for the full grace time and backend is
                        # considered unhealthy - try to contact the backend
                        # anyway
                        return (fetch);
                }
        }
}

Here I "defined" a 2nd grace time of 30 minutes by using the conditional

            if (obj.ttl + 30m > 0s) {

In vcl_backend_response I just set the maximum grace time of 6 hours:

sub vcl_backend_response {
        # define full grace time here
        set beresp.grace = 6h;
        ...
}