I would like to send a batch of events with different partition key in event hub using python client. I can send a single message with the partition key set but what if I need to send a batch of events containing different partition key to event hub
Here is the code for 1 event
message = [{'id': 1, 'msg': 'this is test msg 1'},
{'id': 2, 'msg': 'this is test msg 2'},
{'id': 3, 'msg': 'this is test msg 3'}]
for msg in message:
client.run()
event_data = EventData(msg['msg'])
event_data.partition_key = str(msg['id'])
sender.send(event_data)
I would like to do the same with batch sending all 3 messages at the same time
I think it's impossible, send batch method
would send all the events in batch to the same one partition.
Here is the test code, I can specify different partition keys to event data, then send with batch(you can modify it to meet your need):
from azure.eventhub import EventHubClient, Sender, EventData
# use this method to generate event data, and set partition key.
def data_generator():
for i in range(15):
data = EventData("hi, hello world!")
if i % 2 ==0:
data.partition_key=b'0'
else:
data.partition_key=b'1'
temp = data.body_as_str()
yield temp
ADDRESS = "xxxx"
# SAS policy and key are not required if they are encoded in the URL
USER = "RootManageSharedAccessKey"
KEY = "xxx"
client = EventHubClient(address=ADDRESS,username=USER,password=KEY)
sender = client.add_sender()
client.run()
data = EventData(batch=data_generator())
sender.send(data)
print("*completed*")
It sends successfully, but when I read the even data, I can see all the data are actually sent to one same partition.
And if the reason you're using batch send method
to improve performance, and must use different partition key, you can consider using send async method to achieve this.