Search code examples
apache-flinkflink-streaming

will the Broadcast state source block process?


use flink version 1.13.0

i use the Broadcast state in my application, and it will load large data per 2 minutes(about 500'000 data in Map type)

and i see the topological graph in web-ui find that every time the Broadcast source load, it's has 50%-100% backpressure, and the process which joined has in 50%-100% busy. i want to know in this time will the process has been blocked and deal data slowly or stop to deal data?


Solution

  • BroadcastProcessFunction is thread-safe. When processBroadcastElement is called to load state, the processElement function will not be executed.

    Therefore, when the state is relatively large, normal data processing will be blocked, and back pressure will also be generated.