Finite-time analysis of Multi-timescale Stochastic Optimization Algorithms

View PDF HTML (experimental)

Abstract:We present a finite-time analysis of two smoothed functional stochastic approximation algorithms for simulation-based optimization. The first is a two time-scale gradient-based method, while the second is a three time-scale Newton-based algorithm that estimates both the gradient and the Hessian of the objective function $J$. Both algorithms involve zeroth order estimates for the gradient/Hessian. Although the asymptotic convergence of these algorithms has been established in prior work, finite-time guarantees of two-timescale stochastic optimization algorithms in zeroth order settings have not been provided previously. For our Newton algorithm, we derive mean-squared error bounds for the Hessian estimator and establish a finite-time bound on $\min\limits_{0 \le m \le T} \mathbb{E}| \nabla J(\theta(m)) |^2$, showing convergence to first-order stationary points. The analysis explicitly characterizes the interaction between multiple time-scales and the propagation of estimation errors. We further identify step-size choices that balance dominant error terms and achieve near-optimal convergence rates. We also provide corresponding finite-time guarantees for the gradient algorithm under the same framework. The theoretical results are further validated through experiments on the Continuous Mountain Car environment.

Subjects:

Machine Learning (cs.LG)

Cite as: arXiv:2603.29380 [cs.LG]

(or arXiv:2603.29380v1 [cs.LG] for this version)

https://doi.org/10.48550/arXiv.2603.29380

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Kaustubh Kartikey [view email] [v1] Tue, 31 Mar 2026 07:50:11 UTC (67 KB)