Finite-stage reward functions having the Markov adequacy property

Victor Pestien, Xiaobo Wang

Research output: Contribution to journalArticlepeer-review

2 Scopus citations


If X is a countable state space and g is a bounded reward function on Xn, then say g has the Markov-adequacy property if every strategy has a corresponding randomized Markov strategy which gives g the same integral as the original strategy. A complete characterization of functions having the Markov-adequacy property is given. In particular, if g is permutation-invariant and X has at least three elements, then g has the Markov-adequacy property if and only if g has the linear sections property, a condition which is easy to verify.

Original languageEnglish (US)
Pages (from-to)129-151
Number of pages23
JournalStochastic Processes and their Applications
Issue number1
StatePublished - May 1993


  • Markov strategy
  • randomized strategy
  • rewars function

ASJC Scopus subject areas

  • Statistics and Probability
  • Modeling and Simulation
  • Applied Mathematics


Dive into the research topics of 'Finite-stage reward functions having the Markov adequacy property'. Together they form a unique fingerprint.

Cite this