Search code examples
c#performanceasynchronousparallel-processing

How to parallelize asynchronous integration work that includes JSON deserialization?


One question about parallel programming.

I need to POST 2 APIs (one after another) to get a Order list. First one is for getting token..

After I got the list, I need to POST 3 APIs (one after another) to integrate these Orders.

These 3 APIs don't accept arrays, so I need to send one by one. I cannot send batch data. With 1 thread it only integrates 10 orders in 1 minute. I need more performance. How can I run foreach part in parallel?

using System.Net.Http;
using System.Text;
using System.Text.Json;

namespace Order_Integrator
{
    public class Program
    {

        static readonly HttpClient client = new HttpClient();

        static async Task Main()
        {

            //Auth
            var connectResponse = await client.PostAsync(connectUrl, connectContent);
            var connectResponseString = await connectResponse.Content.ReadAsStringAsync();
            var connect = JsonSerializer.Deserialize<connectResponse>(connectResponseString);
            
            var token = connect.Token;
            //Get Order List
            var orderResponse = await client.PostAsync(orderUrl, orderContent);
            var orderResponseString = await orderResponse.Content.ReadAsStringAsync();
            var orders = JsonSerializer.Deserialize<orderResponse>(orderResponseString);

            foreach (var order in orders)
            {
            
                //Get Order Details
                //Generate getOrderDetailsContent with order 
                var getOrderDetailsResponse = await client.PostAsync(getOrderDetailsUrl, getOrderDetailsContent);
                var getOrderDetailsResponseString = await getOrderDetailsResponse.Content.ReadAsStringAsync();
                var getOrderDetails = JsonSerializer.Deserialize<getOrderDetailsResponse>(getOrderDetailsResponseString);

                //Create Order
                //Generate createOrderContent with GetOrderDetails
                var createOrderResponse = await client.PostAsync(createOrderUrl, createOrderContent);
                var createOrderResponseString = await createOrderResponse.Content.ReadAsStringAsync();
                var createOrder = JsonSerializer.Deserialize<createOrderResponse>(createOrderResponseString);

                //Create Log
                //Generate createLogContent with CreateOrderResponse
                var createLogResponse = await client.PostAsync(createLogUrl, createLogContent);
                var createLogResponseString = await createLogResponse.Content.ReadAsStringAsync();
                var createLog = JsonSerializer.Deserialize<createLogResponse>(createLogResponseString);
            }
        }
    }
}

Solution

  • You can use Parallel.ForEachAsync

     var options = new ParallelOptions()
    {
        MaxDegreeOfParallelism = 20
    };
    await Parallel.ForEachAsync(orders, options, async (OrderNumber, ct) => {
    
    var getOrderDetailsResponse = await client.PostAsync(getOrderDetailsUrl, getOrderDetailsContent);
        
    });