Using gRPC, how should I handle partial results between my server and client?

This is more of an API design question, but I'm curious what other people think of this scenario:

Say I have a server, clients will send me a list of items they'd like to retrieve data on and I return a map of item:data.

Naturally first I will check my cache, the query my DB. Let's pretend the DB query is taking too long (for whatever reason) so I terminate with enough time to return partial results (found in the cache) to the client.

There are two important thing I want the client to know:

Here are your partial results (if you choose to use them).
This was the reason why I couldn't resolve the rest of your queries (err.ConnDeadlineExceeded)

Now currently I plan to only return the partial results and a nil error. The reason for this is most of my clients first check that err != nil, and only if that's true will they actually bother to look at the results.

But this feels wrong because I'm dropping information about the DB timeout. Any suggest solutions?

Solution

As you've identified, it's really up to how you want the client to react to an API response:

results, err := app.Api(ctx, queryMessage)

A partial results indicator would either reside in results or in a non-nil err.

Take the example of ldapsearch - if you provide a size limit - the API will return up to that amount of results but return a non-zero return-code (sizeLimitExceeded). Clients need to know to check for that specific code and if they want to process the partial results that were sent.

A nil error seems the more Go-way, so you would need to include that a sizelimit was hit in the results message. One suggestion is to define an proto enum:

enum extraStatusCode {
    UNKNOWN         = 0;  // not set/used
    SIZELIMIT       = 1;
    TIMELIMIT       = 2;
}

and include this in your results proto message:

message Result {
    repeated Record records = 1;  // results (full or partial)

    // ...

    extraStatusCode = 15; // indicate if partial results 
}