In this paper we describe recent progress in our work on Value Function Discovery (VFD), a novel method for discovery of value functions for Markov Decision Processes (MDPs). In a previous paper we described how VFD discovers algebraic descriptions of value functions (and the corresponding policies) using ideas from the Evolutionary Algorithm field. A special feature of VFD is that the descriptions include the model parameters of the MDP. We extend that work and show how additional information about the structure of the MDP can be included in VFD. This alternative use of VFD still yields near-optimal policies, and is much faster. Besides increased performance and improved run times, this approach illustrates that VFD is not restricted to learning value functions and can be applied more generally.
Additional Metadata
Keywords Markov processes, Dynamic programming
THEME Logistics (theme 3)
Publisher ACM
Persistent URL dx.doi.org/10.1145/2825236.2825239
Journal ACM SIGMETRICS Performance Evaluation Review
Project Realisation of Reliable and Secure Residential Sensor Platforms
Citation
Onderwater, M, Bhulai, S, & van der Mei, R.D. (2015). Learning Optimal Policies in Markov Decision Processes with Value Function Discovery. ACM SIGMETRICS Performance Evaluation Review, 43(2), 7–9. doi:10.1145/2825236.2825239