QueueTrigger Azure function not processing message and exhibiting bizarre behavior

Question

I am implementing a queue message handler in a web job on an azure app service. But I am experiencing some very bizarre behavior in the handler app. The process is very simple - I have a trigger app that creates a message and sends that message to the queue. Then I have my listener/handler app in a separate app service that picks up the message from the queue and processes it. Below are two very small snippets of code from those two apps.

//TRIGGER APP
QueueClient queue = new QueueClient(storageConnectionString, queueName);
queue.CreateIfNotExists();
queue.SendMessage(JsonConvert.SerializeObject(myModel));


//HANDLER APP
public class Functions
{
    IConfiguration configuration;
    private static List<CosmosClient> cosmosClients = new List<CosmosClient>();
    public Functions(IConfiguration _configuration)
    {
        configuration = _configuration;
    }
    public void ProcessQueueMessageAsync([QueueTrigger("%QueueName%")] string message)
    {
        Console.WriteLine(message);
        // do more stuff here...
    }
}

The bizarre behavior is that my "handler app" immediately puts the message onto the poison queue. There are no logs of errors anywhere that I could find. I've searched the console log, app service log, even the log streamer in real time, and no errors are logged anywhere. But all messages are getting put onto the poison queue. And they are getting placed in the poison queue before the code even reaches that first line in the handler (Console.WriteLine(message);) Even more bizarre, is that I can go to the poison queue, find the message that was poisoned, and manually copy and paste that "poisoned" message back onto the regular queue (same queue as in the code above) within the Azure portal (not in code). And after manually adding it back to the queue using the portal, the Console.WriteLine(message); is hit, and it logs the message, and goes through the rest of the app just fine. I am baffled by this.

I have also watched this in real-time. I see the message get placed on to the regular queue. I inspect it, and it looks fine. After a few seconds, it disappears and shows up in the poison queue. I inspect it and it looks exactly the same. I copy that queue message from the poison queue, and "Add message" back on the regular queue and simply paste what was on the poison queue. Then it gets processed fine (doesn't go back on to the poison queue). I also have other logging mechanisms to verify that it processes successfully after "pasting" it back.

Am I adding the message to the queue in an incorrect way in code (trigger app above)? What am I missing here?

Answer 1

Although I found a resolution to this issue, I still don't know why this was happening. The solution was to use a different queue client in the trigger. There must be some sort of encoding difference between the two clients when a message is added to the queue (that's my best guess anyways). The solution was to change this:

//TRIGGER APP
QueueClient queue = new QueueClient(storageConnectionString, queueName);
queue.CreateIfNotExists();
queue.SendMessage(JsonConvert.SerializeObject(myModel));

to this:

//TRIGGER APP
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(storageConnectionString);
CloudQueueClient queueClient = storageAccount.CreateCloudQueueClient();
CloudQueue queue = queueClient.GetQueueReference(queueName);
queue.CreateIfNotExistsAsync().Wait();
queue.AddMessageAsync(new CloudQueueMessage(JsonConvert.SerializeObject(myModel))).Wait();