I'm using TPL DataFlow with BatchBlock which is triggered automatically when the number of currently queued or postponed items is less than the BatchSize after a timeout as given below :
Timer triggerBatchTimer = new(_ => _batchBlock.TriggerBatch());
TransformBlock<string, string> timeoutTransformBlock = new((value) =>
{
triggerBatchTimer.Change(_options.Value.TriggerBatchTime, Timeout.Infinite);
return value;
});
var actionBlock = new ActionBlock<IEnumerable<string>>(action =>
{
GenerateFile(action);
});
_buffer.LinkTo(timeoutTransformBlock);
timeoutTransformBlock.LinkTo(_batchBlock);
_batchBlock.LinkTo(actionBlock);
Ie with max batch size : 4 and timeout 10sec
Current behavior
BatchBlock (items): +---------1------------2------3---------------------------------|
Timeout (sec) : +-10--9--10--9--8--7--10--9--10--9--8--7--6--5--4--3--2--1--0---|
ActionBlock : +----------------------------------------------------------Call-|
Expected
BatchBlock (items): +--------1-----------2-----3---------|
Timeout (sec) : +-10--9--8--7--6--5--4--3--2--1--0---|
ActionBlock : +-------------------------------Call-|
But my problem is how to avoid timeout to be reset to 0 each time the block receive a new item ?
Thanks to @Theodor Zoulias for help, this is solution :
Just initialize the timer with the same value for both dueTime
and period
and remove triggerBatchTimer.Change
in TransformBlock
.
Thus batchblock fires every X seconds or when the batchSize has been reached and the timer is not reset for each new item in the batchblock.
Timer triggerBatchTimer = new(_ => _batchBlock.TriggerBatch(), null, options.Value.TriggerBatchTime, options.Value.TriggerBatchTime);
TransformBlock<string, string> timeoutTransformBlock = new((value) =>
{
return value;
});
var actionBlock = new ActionBlock<IEnumerable<string>>(action =>
{
GenerateFile(action);
});
_buffer.LinkTo(timeoutTransformBlock);
timeoutTransformBlock.LinkTo(_batchBlock);
_batchBlock.LinkTo(actionBlock);