Search code examples
neural-networkdeep-learningartificial-intelligencetorchreinforcement-learning

AlphaGo Zero board evaluation function uses multiple time steps as an input... Why?

According to AlphaGo Cheat Sheet, AlphaGo Zero uses a sequence of consecutive board configurations to encode its game state.

In theory, all the necessary information is contained in the latest state, and yet they include the previous 7 configurations.

Why did they choose to inject so much complexity ?

What are they listening for ??

AlphaGoZero

Game State

Solution

  • The sole reason is because in all games - Go, Chess, and Shogi - there is a repetition rule. What this means is that the game is not fully observable from the current board position. In other words, there may be two identical positions with two very different evaluations. For example in one Go position there may be a winning move, but in an identical Go position that move is either illegal or one of the next few moves in the would-be-winning continuation creates an illegal position.

    You could try feeding in only the current board position and handling repetitions in the tree only. But I think this would be weaker because the evaluation function would be wrong in some cases, leading to a horizon effect if that branch of the tree had not been explored deeply enough to correct the problem.