We consider a system with a finite number of states, 1, 2, ⋯, S. Periodically we observe the current state of the system and perform an action, a, from a finite set A of possible actions. As a joint ...