Suspend posting to full BufferAction

Question

I have a simple bounded BufferBlock which I post and receive data from. The elements I push into are rows from db. To save some memory and not load all rows into BufferBlock I use BoundedCapacity.

The problem is that it works not as I expect it to work, actually it doesn't post it to the BufferBlock if it's full as expected, but it keeps reading from db and saves it (probably) somewhere in memory so it can push it to the BufferBlock later in the moment it can.

while (reader.Read()) {
    // queue is the bounded BufferBlock
    // why does it keep reading even though queue is full?
    MyObj obj = GetMyObjByReader(reader);
    await queue.SendAsync(obj);
}

The queue is linked to a transformer which does his job, but it's not that relevant.

I need somehow to await until the queue is not full.

Answer 1

You say:

"The queue is linked to a transformer which does his job, but it's not that relevant."

If by "transformer" you mean TransformBlock (and I assume you do) what probably happens is that you link the TransformBlock to the BufferBlock but only limit the size of the BufferBlock with BoundedCapacity.

Since the TransformBlock is unbounded (which is the default) everything you post to the BufferBlock is immediately transferred to the TransformBlock's InputQueue and that's why your memory increases.

Bounding just a single block in the pipe doesn't bound the entire pipe. You need to bound each block individually.