Skip to content

Gittens Index

[…] the discounted value (between 0 and 1) of the next pull of a bandit arm, taking into account the results of all pulls of that arm to date. The calculation of the index is a bit involved (it involves finding the present value of infinite uncertain sequences), but here are a few values taken from Gittens? 1979 paper introducing the index:





















Table of Gittens index values for given numbers of wins and losses

For example, given a discount rate of 75%, and given that you have pulled the arm 3 times and won twice, the Gittens index is 0.7465, meaning that continuing to pull for an indefinite period is the equivalent of a certain immediate payout of 0.7465.  Note that for all the 50/50 cells where wins = losses, despite the fact that the data appear to be suggesting parity between wins and losses, the Gittens index is > 0.5. This reflects the hidden value in the ability to continue to explore.