send async azure event hub data method

Question

In my api (c#), i'm currently answering lots of request (15-20 in a second). Now i want to send the data to event hub for some purposes. But i dont want to delay the user for sending data to azure event hub. So i need to make async request to event hub beacuse while my app sending data to azure i dont want to get user waited for my answer. I need to send response as quick as possible, azure may last 2-3 second.

How can i success that? I ve done smth but didnt get what i want. My code:

public static async Task<string> SendEvents(List<object> messages)
{
    string eventHubName = "rcmds";

    var connectionString = GetServiceBusConnectionString();

    CreateEventHub(eventHubName, connectionString);

    var eventHubClient = EventHubClient.CreateFromConnectionString(connectionString, eventHubName);

    try
    {
        List<Task> tasks = new List<Task>();

        for (int i = 0; i < messages.Count; i++)
        {
            var serializedMessage = JsonConvert.SerializeObject(messages[i]);

            EventData data = new EventData(Encoding.UTF8.GetBytes(serializedMessage));

            // Mesajları Event Hub a yolla
            tasks.Add(eventHubClient.SendAsync(data));
        }

        Task.WaitAll(tasks.ToArray());
        System.Threading.Thread.Sleep(7000);
    }
    catch (Exception ex)
    {
        new ExceptionHandler(ex, "Event Hub Library Sender - SendEvents");
    }
    finally
    {
        eventHubClient.CloseAsync().Wait();
    }
    return "";
}

and i call this method like:

 static async void method()


 {
            List<object> list = new List<object>();
            list.Add("dogrudur");
            await Utilities.EventHub.Sender.SendEvents(list);
        }

As you can see, there is "thread.sleed" code but i waited 7 seconds :/

Answer 1

I think you need to learn more about how to really use async/await in a good manner. You are mixing async code with blocking code.

The correct code should be something like this:

public static async Task<string> SendEvents(List<object> messages)
{
    string eventHubName = "rcmds";

    var connectionString = GetServiceBusConnectionString();

    CreateEventHub(eventHubName, connectionString);

    var eventHubClient = EventHubClient.CreateFromConnectionString(connectionString, eventHubName);

    try
    {
        List<Task> tasks = new List<Task>();

        for (int i = 0; i < messages.Count; i++)
        {
            var serializedMessage = JsonConvert.SerializeObject(messages[i]);

            EventData data = new EventData(Encoding.UTF8.GetBytes(serializedMessage));

            // Mesajları Event Hub a yolla
            tasks.Add(eventHubClient.SendAsync(data));
        }

        await Task.WhenAll(tasks.ToArray());
        System.Threading.Thread.Sleep(7000);
    }
    catch (Exception ex)
    {
        new ExceptionHandler(ex, "Event Hub Library Sender - SendEvents");
    }
    finally
    {
        await eventHubClient.CloseAsync();
    }

    return "";
}

The calling code should be:

static async Task method()
 {
        List<object> list = new List<object>();
        list.Add("dogrudur");
        await Utilities.EventHub.Sender.SendEvents(list);
    }

Now, back to the question. The current code waits untill all messages are send to the event hub and than sleeps for 7 seconds. In you calling method you await the SendEvents method so of course your application will take 7 seconds + the time required to send the data to the event hub.

What you could do is implement some kind of fire-and-forget mechanism. Remove the Thread.Sleep in the code and modify the calling method like this:

static void method()
{
    List<object> list = new List<object>();
    list.Add("dogrudur");
    Utilities.EventHub.Sender.SendEvents(list);
}

The method will now no longer wait before continuing, but in return you will never know when the event sending is completed. In general fire and forget methods should be avoided.

Another important step you can take to improve the performance is to send the events in batches. Currently each event is send using eventHubClient.SendAsync but there is also a eventHubClient.SendBatchAsync method (https://msdn.microsoft.com/en-us/library/microsoft.servicebus.messaging.eventhubclient.sendbatchasync.aspx) so instead of sending a lot of small messages you can send fewer bigger messages than contain more than one event. Be aware that there is a maximum message size though.

For an example implementation of sending event using a batch see the method private async Task SendAutoSizedBatchAsync(IEnumerable collection) in this file https://github.com/DeHeerSoftware/SemanticLogging.EventHub/blob/master/SemanticLogging.EventHub/EventHubAmqpSink.cs