Wang, S., Si, N., Blanchet, J., & Zhou, Z. (2024). Statistical Learning of Distributionally Robust Stochastic Control in Continuous State Spaces. ArXiv. /abs/2406.11281
Abstract
We explore the control of stochastic systems with potentially continuous state and action spaces, characterized by the state dynamics . Here, , , and represent the state, action, and exogenous random noise processes, respectively, with denoting a known function that describes state transitions. Traditionally, the noise process is assumed to be independent and identically distributed, with a distribution that is either fully known or can be consistently estimated. However, the occurrence of distributional shifts, typical in engineering settings, necessitates the consideration of the robustness of the policy. This paper introduces a distributionally robust stochastic control paradigm that accommodates possibly adaptive adversarial perturbation to the noise distribution within a prescribed ambiguity set. We examine two adversary models: current-action-aware and current-action-unaware, leading to different dynamic programming equations. Furthermore, we characterize the optimal finite sample minimax rates for achieving uniform learning of the robust value function across continuum states under both adversary types, considering ambiguity sets defined by -divergence and Wasserstein distance. Finally, we demonstrate the applicability of our framework across various real-world settings.
Authors
Shengbo Wang, Nian Si, Jose Blanchet, Zhengyuan Zhou
Publication date
2024/6/17
Journal
arXiv preprint arXiv:2406.11281
