Search code examples
c#.net-coretpl-dataflow

How to avoid reset timeout in batchblock


I'm using TPL DataFlow with BatchBlock which is triggered automatically when the number of currently queued or postponed items is less than the BatchSize after a timeout as given below :

Timer triggerBatchTimer = new(_ => _batchBlock.TriggerBatch());

TransformBlock<string, string> timeoutTransformBlock = new((value) =>
{
    triggerBatchTimer.Change(_options.Value.TriggerBatchTime, Timeout.Infinite);
    return value;
});

var actionBlock = new ActionBlock<IEnumerable<string>>(action =>
{
    GenerateFile(action);
});

_buffer.LinkTo(timeoutTransformBlock);
timeoutTransformBlock.LinkTo(_batchBlock);
_batchBlock.LinkTo(actionBlock);

Ie with max batch size : 4 and timeout 10sec

Current behavior
BatchBlock (items): +---------1------------2------3---------------------------------|
Timeout (sec)     : +-10--9--10--9--8--7--10--9--10--9--8--7--6--5--4--3--2--1--0---|
ActionBlock       : +----------------------------------------------------------Call-|

Expected
BatchBlock (items): +--------1-----------2-----3---------|
Timeout (sec)     : +-10--9--8--7--6--5--4--3--2--1--0---|
ActionBlock       : +-------------------------------Call-|

But my problem is how to avoid timeout to be reset to 0 each time the block receive a new item ?


Solution

  • Thanks to @Theodor Zoulias for help, this is solution :

    Just initialize the timer with the same value for both dueTime and period and remove triggerBatchTimer.Change in TransformBlock. Thus batchblock fires every X seconds or when the batchSize has been reached and the timer is not reset for each new item in the batchblock.

    Timer triggerBatchTimer = new(_ => _batchBlock.TriggerBatch(), null, options.Value.TriggerBatchTime, options.Value.TriggerBatchTime);
    
    TransformBlock<string, string> timeoutTransformBlock = new((value) =>
    {
        return value;
    });
    
    var actionBlock = new ActionBlock<IEnumerable<string>>(action =>
    {
        GenerateFile(action);
    });
    
    _buffer.LinkTo(timeoutTransformBlock);
    timeoutTransformBlock.LinkTo(_batchBlock);
    _batchBlock.LinkTo(actionBlock);