Zobrist hashing in chess: why treat castling and En passant separately?

Question

I'm trying to implement Zobrist Hashing for a board game, so I read about it and its particular use in Chess games. It seems you first need to initialize an array of random 64 bits integer of size 64 (8x8), multiplied by the number of pieces.

The thing I don't understand, is that you also need to add castling and "en passant", as well as other combinations:

• One number for each piece at each square (ok with that)
• One number to indicate the side to move is black (?)
• Four numbers to indicate the castling rights (?)
• Eight numbers to indicate the file of a valid En passant square, if any (?)

Bullet points marked with (?) are the ones I don't understand. Since every piece has already 64 possible possible moves (e.g. the whole chess board), why do you need to add castling and "en passant"? Why do you need to also add the side to move?

I'm trying to understand this, because in other board games with other rules, I wouldn't know how to select which positions/combinations should be stored in the Zobrist initialization array.

Thanks

Answer 1

That's because two positions can have absolutely identical piece placement on the board but still be different positions with vastly different evaluation.

Zobrist hashing is an attempt to uniquely (or at least as uniquely as possible) distinguish different (chess) positions with as little memory and computational footprint as possible.

This hash is then used to index a tranpositions table, which really isn't anything else than a hash map. In this hash map you store positions and their associated evaluation score (and possibly some other useful information) - this serves to speed up your search by remembering previously searched positions and being able to immediatelly retrieve their score, without having to evaluate them again.

Now consider a position where the white players pawn is attacking the black queen. If white player is currently on the move, the position most likely has a very different score than if the black player was currently moving and could potentially evade the queen being captured. If your zobrist hash couldn't distinguish between these two positions then you could retrieve a very wrong score from your transposition table and make your AI play the wrong move. That's why adding the color of the player that is currently moving to the hash is useful. Same applies to en-passant and castling. Eg for castling your rook might've left the starting square and then later returned back (and the other player made a move and then returned back as well), now the piece placement on the board is the same as before, the player to move is the same as well, but you lost your right to castle - so it is a different position and it will have different properties (evaluation).