Search code examples
c#azure-cosmosdb-sqlapi

How to use "ToFeedIterator" in Linq?


I am using a DbContext to query a Cosmos container. I have a scenario where I would get a comma-separated string with N ids. I want to query them efficiently. The majority of examples suggests using container.GetItemQueryIterator but in my case, I can use query.ToFeedIterator() (unless there is a different way, I am not aware of it).

To do that, I am trying to use .ToFeedIterator but I am getting the following error:

"ToFeedIterator is only supported on Cosmos LINQ query operations (Parameter 'linqQuery')" error and not sure how to resolve

My goal is to query Cosmos as efficiently as I can when I receive a huge amount of data.

References:

Questions:

  • Is this the correct way to use FeedIterator?
  • How can I pass a partition key in my query?
  • Let's say these strings come up with 1000 ids (or more), is this the most efficient way? OR is there a better way to do that?

Code:

DbContext.cs

public class CosmosDbContext: DbContext
{
   
    public CosmosDbContext(DbContextOptions<CosmosDbContext> options) : base(options) { }

    #region DbSets

    public DbSet<User> User { get; set; }

    #endregion

    protected override void OnModelCreating(ModelBuilder modelBuilder)
    {
        #region user
        modelBuilder.Entity<User>()
            .ToContainer(nameof(User).ToLower())
            .HasNoDiscriminator()
            .HasPartitionKey(user => user.userId)
            .HasKey(user => user.id);

        modelBuilder.Entity<User>()
            .Property(user => user.id)
            .ValueGeneratedOnAdd();

        modelBuilder.Entity<User>().HasPartitionKey(record => record.userId);
        #endregion

        base.OnModelCreating(modelBuilder);
    }
}

UserRepository.cs

public async Task<User> GetExistingDocumentsAsync(string userIds, bool withNoTracking, CancellationToken cancellationToken)
{
    var query = withNoTracking ? _dbContext.User.AsNoTracking() : _dbContext.User;

    var userDocs = new List<User>();
    var ids = userIds.Split(",").Select(s => s.Trim()).ToArray();
    var feedIterator = query.Where(x => ids.Contains(x.userId)).ToFeedIterator(); // Throws error here
    while (feedIterator.HasMoreResults) 
    {
        foreach(var document in await feedIterator.ReadNextAsync(cancellationToken))
        {
            userDocs.Add(document);
            // Iterate through documents
        }
    }
    
    return userDocs;
}

Solution

  • ToFeedIterator() method can be used with Container class object to convert LINQ query to iterator, as it is implemented with abstract methods GetItemQueryIterator(), GetItemQueryStreamIterator() etc. But same is not implemented in DbContext class. This is the reason when we use toFeedIterator() with DbContext object it throws the following exception

    ToFeedIterator is only supported on Cosmos LINQ query operations (Parameter 'linqQuery')

    We can still use LINQ queries with DbContext object using ToListAsync or ToList methods with Cosmos Db and Entity Framework Core,

    Below is the sample code using PartitionKey, which is working as expected with the use of WithPartitionKey() and ToFeedIterator().

    As mentioned earlier in this case I have used ToFeedIterator() with GetItemLinqQueryable() using Container class object.

    CosmosClient _client = new CosmosClient({CosmosDBConnectionString});
    var _database = _client.GetDatabase({DatabaseName});
    var _container = _database.GetContainer({ContainerName});
    
    var userDocs = new List<User>();
    var ids = userIds.Split(",").Select(s => s.Trim()).ToArray();
    
    var feedIterator = _container.GetItemLinqQueryable<User>()
    .WithPartitionKey(partitionKey.ToString())
    .Where(x=>ids.Contains(x.userId))
    .ToFeedIterator();
    while (feedIterator.HasMoreResults)
        {
            foreach (var document in await feedIterator.ReadNextAsync())
            {
                userDocs.Add(document);
            }
        }
    

    Model (User.cs)

    public class User
        {
            public string? id { get; set; }
            public string? userId { get; set; } //PartitionKey
            public string? name { get; set; }
        }
    

    Using contains() should work fine for large datasets as well.