In producer using librdkafka, is it possible to know the number of items produced but not sent yet. I have turned ack off, only interested in sending the message out from the producer. I wish to avoid producing more messages if there are certain number of messages on the send queue(ie the messages produced but not yet sent).
If you set statistics.interval.ms in the producer configuration file, it will start sending the internal metrics in the callback you register. The callback you can set using the function rd_kafka_conf_set_stats_cb. The statistics are received in the callback in the form of json data. One the fields from the json data, msg_cnt, tell you the queued messages that you are looking for. More details are available at https://github.com/edenhill/librdkafka/blob/master/STATISTICS.md