We investigate control of a queueing system in which a component of the state space is subject to aging. The controller can choose to forward incoming queries to the system (where it needs time for processing), or respond with a previously generated response (incurring a penalty for not providing a fresh value). Hence, the controller faces a tradeoff between data freshness and response times. We model the system as a complex Markov decision process, simplify it, and construct a control policy. This policy shows near-optimal performance and achieves lower costs than both a myopic policy and a threshold policy.
Markov Decision Processes, Controlled Queueing System, Aging State Information, One-step Policy Improvement, Ordered Performance Curve, Value Iteration, Difference Equations, Optimization
Logistics (theme 3)
Stochastic Models
Realisation of Reliable and Secure Residential Sensor Platforms

Onderwater, M, Bhulai, S, & van der Mei, R.D. (2015). On the Control of a Queueing System with Aging State Information. Stochastic Models, 31(4), 588–617. doi:10.1080/15326349.2015.1058716