Search code examples
apache-flinkflink-statefun

Flink Statefun concurrent state update

I'm trying to implement messaging scenario using apache flink stateful functions. One of my state is able to updated by two different functions which is provided to MatchBinder. These two functions basically checks the current state and updates the state accordingly.

  • What happens if these two functions are called concurrently for the same key?
  • Is there a queue mechanism for stateful functions called for the same key?
  • Can we lock the state access/update for sequential access ?
Solution

  • What happens if these two functions are called concurrently for the same key?

    The MatchBinder is basically a convenient way to write a single StateFun function, that starts its execution by first matching the type (or properties) of the incoming message. It is basically a way to avoid writing code like this:

    ...
if (message instanceof A) {
  handleA((A) message);
} else if (message instanceof B) {
  handleB((B) message);
}
...

    So in reality, although you are providing "different" Java functions to each bind case, this is the same StateFun function being invoked and the correct bind case would be selected.

    Is there a queue mechanism for stateful functions called for the same key?

    Yes, StateFun functions would be invoked sequentially per address. While a function is applied for a specific address, no other message for that address would be applied concurrently. This comes almost for free, thanks to having Apache Flink as the actual runtime.

    Can we lock the state access/update for sequential access ?

    State access and modifications are atomic and sequential per address.