Service Fabric Reliable Collections - Are key filters applied in deterministic order for CreateEnumerableAsync?

Question

I'm attempting to implement cursor-based pagination for a reliable dictionary. I know that IReliableDictionary keys must implement IComparable, and that IReliableDictionary this method for enumerating dictionary entries:

IAsyncEnumerable<KeyValuePair<TKey,TValue>>> 
CreateEnumerableAsync (
    ITransaction txn, 
    Func<TKey,bool> filter,
    EnumerationMode enumerationMode);

When EnumerationMode.Ordered is used, I assume we enumerate the key-value pairs according to the key's IComparable implementation.

Can we also assume the filter parameter is applied to each key in order of the key's IComparable implementation? Maybe another way to ask - are keys laid out in memory and/or enumerated in order of their IComparable implementation? If so, is this behavior documented, or should it be considered an implementation detail that may change?

Answer 1

I performed an experiment using the Voting Web sample, and the keys do appear to be filtered in order of their IComparable implementation, with some caveats:

• It appears the filter can be run multiple times over the same key
• The filter can be applied to items which have been recently deleted

VotingData.Controllers.VoteDataController has the following get and put operations to list and add voting categories:

[HttpPut("{name}")]
public async Task<IActionResult> Put(string name)
{
    IReliableDictionary<string, int> votesDictionary = await this.stateManager.GetOrAddAsync<IReliableDictionary<string, int>>("counts");

    using (ITransaction tx = this.stateManager.CreateTransaction())
    {
        await votesDictionary.AddOrUpdateAsync(tx, name, 1, (key, oldvalue) => oldvalue + 1);
        await tx.CommitAsync();
    }

    return new OkResult();
}

I modified Get to enumerate votesDictionary in order, and applied a filter which builds a list of keys seen by the filter:

[HttpGet]
public async Task<IActionResult> Get()
{
     CancellationToken ct = new CancellationToken();
     IReliableDictionary<string, int> votesDictionary = await this.stateManager.GetOrAddAsync<IReliableDictionary<string, int>>("counts");            

     var filteredKeys = new List<string>();

     using (ITransaction tx = this.stateManager.CreateTransaction())
     {
        IAsyncEnumerable<KeyValuePair<string, int>> list = await votesDictionary.CreateEnumerableAsync(tx,                                                                                                        key =>
                                                                                                     {
                                                                                                           lock (this.locker)
                                                                                                           {
                                                                                                               filteredKeys.Add(key);
                                                                                                               return true;
                                                                                                           }
                                                                                                       },
                                                                                                       EnumerationMode.Ordered);
        IAsyncEnumerator<KeyValuePair<string, int>> enumerator = list.GetAsyncEnumerator();
        List<KeyValuePair<string, int>> result = new List<KeyValuePair<string, int>>();

        while (await enumerator.MoveNextAsync(ct))
        {
            result.Add(enumerator.Current);
        }
         return this.Json(result);
    }
}

I added keys to the dictionary in random alphabetical order and refreshed the page to run the Get query. On each refresh, the filteredKeys collection contained my entries in sorted alphabetical order. As mentioned above, the collection sometimes contained duplicate entries for certain strings. When I removed items from the collection and refreshed the page, I found deleted keys were still added to filteredKeys, although these elements were not returned in the result enumeration.