Bulk Upload/Import of JSON files to Azure Cosmos DB from .NET Core

Question

I'm generating some JSON files in .NET Core app. I want to import a json file to Azure Cosmos DB as soon as it is created.

Is there some way to achieve it from a .NET Core code?

Answer 1

According to my test, if you want to use Cosmos DB Bulk insert in .Net core application, you need to use the .Net CosmosDB SDK V3 and its version must be larger than 3.4.0. For more details, please refer to the document.

My.json file

[{
        "id": "1",
        "name": "test1",
        "age": "20"
    }, {
        "id": "2",
        "name": "test2",
        "age": "21"
    }, {
        "id": "3",
        "name": "test3",
        "age": "22"
    }, {
        "id": "4",
        "name": "test4",
        "age": "23"
    },
    {
        "id": "5",
        "name": "test5",
        "age": "24"
    }, {
        "id": "6",
        "name": "test6",
        "age": "25"
    }, {
        "id": "7",
        "name": "test7",
        "age": "26"
    }, {
        "id": "8",
        "name": "test8",
        "age": "27"
    }
]

My code

 private const string EndpointUrl = "";
        private const string AuthorizationKey = "";
        private const string DatabaseName = "testbulk";
        private const string ContainerName = "items";
        async static Task Main(string[] args)
        {
            string json = File.ReadAllText(@"E:\test.json");
            
            List<Item> lists = JsonConvert.DeserializeObject<List<Item>>(json);

            CosmosClientOptions options = new CosmosClientOptions() { AllowBulkExecution = true };
            CosmosClient cosmosClient = new CosmosClient(EndpointUrl, AuthorizationKey,options);
            
            Database database = await cosmosClient.CreateDatabaseIfNotExistsAsync(DatabaseName);
            Console.WriteLine(database.Id);
            Container container = await database.CreateContainerIfNotExistsAsync(ContainerName, "/id");
            Console.WriteLine(container.Id);
            
            List<Task> tasks = new List<Task>();
            foreach (Item item in lists)
            {
                tasks.Add(container.CreateItemAsync(item, new PartitionKey(item.Id))
                    .ContinueWith((Task<ItemResponse<Item>> task) =>
                    {


                        Console.WriteLine("Status: " + task.Result.StatusCode + "    Resource: "+ task.Result.Resource.Id );



                    }));


            }
            await Task.WhenAll(tasks);
}
class Item {

        [JsonProperty(PropertyName = "id")]
        public string Id { get; set; }

        [JsonProperty(PropertyName = "name")]
        public string Name { get; set; }

        [JsonProperty(PropertyName = "age")]
        public string Age { get; set; }
    }

For more details about how to develop your application, please refer to the blog

---

Update

I run my code in .Net 4.6.1 console application.

My code

 class Program
    {

        private const string EndpointUrl = "";
        private const string AuthorizationKey = "";
        private const string DatabaseName = "testbulk";
        private const string ContainerName = "items";
        async static Task Main(string[] args)
        {
            string json = File.ReadAllText(@"E:\test.json");

            List<Item> lists = JsonConvert.DeserializeObject<List<Item>>(json);

            CosmosClientOptions options = new CosmosClientOptions() { AllowBulkExecution = true };
            CosmosClient cosmosClient = new CosmosClient(EndpointUrl, AuthorizationKey, options);

            Database database = await cosmosClient.CreateDatabaseIfNotExistsAsync(DatabaseName);
            Console.WriteLine(database.Id);
            Container container = await database.CreateContainerIfNotExistsAsync(ContainerName, "/id");
            Console.WriteLine(container.Id);

            List<Task> tasks = new List<Task>();
            foreach (Item item in lists)
            {
                tasks.Add(container.CreateItemAsync(item, new PartitionKey(item.Id))
                    .ContinueWith((Task<ItemResponse<Item>> task) =>
                    {


                        Console.WriteLine("Status: " + task.Result.StatusCode + "    Resource: " + task.Result.Resource.Id);



                    }));


            }
            await Task.WhenAll(tasks);
            Console.ReadLine();
        }
        class Item
        {

            [JsonProperty(PropertyName = "id")]
            public string Id { get; set; }

            [JsonProperty(PropertyName = "name")]
            public string Name { get; set; }

            [JsonProperty(PropertyName = "age")]
            public string Age { get; set; }
        }
    }

My package.json

<?xml version="1.0" encoding="utf-8"?>
<packages>
  <package id="Microsoft.Azure.Cosmos" version="3.4.1" targetFramework="net461" />
  <package id="Newtonsoft.Json" version="10.0.2" targetFramework="net461" />
  <package id="System.Buffers" version="4.4.0" targetFramework="net461" />
  <package id="System.Configuration.ConfigurationManager" version="4.5.0" targetFramework="net461" />
  <package id="System.Memory" version="4.5.1" targetFramework="net461" />
  <package id="System.Numerics.Vectors" version="4.4.0" targetFramework="net461" />
  <package id="System.Runtime.CompilerServices.Unsafe" version="4.5.1" targetFramework="net461" />
  <package id="System.Security.AccessControl" version="4.5.0" targetFramework="net461" />
  <package id="System.Security.Permissions" version="4.5.0" targetFramework="net461" />
  <package id="System.Security.Principal.Windows" version="4.5.0" targetFramework="net461" />
  <package id="System.ServiceModel.Primitives" version="4.5.0" targetFramework="net461" />
  <package id="System.Threading.Tasks.Extensions" version="4.5.1" targetFramework="net461" />
  <package id="System.ValueTuple" version="4.5.0" targetFramework="net461" />
</packages>