Simultaneous perturbation stochastic approximation of nonsmooth functions

Abstract

A simultaneous perturbation stochastic approximation (SPSA) method has been developed in this paper, using the operators of perturbation with the Lipschitz density function. This model enables us to use the approximation of the objective function by twice differentiable functions and to present their gradients by volume integrals. The calculus of the stochastic gradient by means of this presentation and likelihood ratios method is proposed, that can be applied to create SPSA algorithms for a wide class of perturbation densities. The convergence of the SPSA algorithms is proved for Lipschitz objective functions under quite general conditions. The rate of convergence [...] of the developed algorithms has been established for functions with a sharp minimum, as well as the dependence of the rate of convergence is explored theoretically as well as by computer simulation. The applicability of the presented algorithm is demonstrated by applying it to minimization of the mean absolute pricing error for the calibration of the Heston stochastic volatility model.