Search code examples
algorithmmathmarkov

Markov expectation: How many stones will the hero cost?


In a game, a hero has 100% probability to go from level 0 to level 1.

When at level 1, he has 1/3 probability to go to level 2, 1/3 probability to level 0, 1/3 probability to stay at level 1.

When at level 2, he has 1/9 probability to win, 4/9 probability to go to level 1, 4/9 probability to stay at level 2.

If each step (upgrade/downgrade/stay at the same level) costs one stone, how many stones will the hero pay to win averagely?


Solution

  • You could introduce 3 random variables A, B, C as follows.

    A: expected number of stones from level 0 to success
    B: expected number of stones from level 1 to success
    C: expected number of stones from level 2 to success

    Then you can formulate the following equations:

    A = 1 + B
    B = 1 + 1/3 * A + 1/3 * B + 1/3 * C
    C = 1 + 1/9 * 0 + 4/9 * B + 4/9 * C

    Then, you can solve for A, B and C.