When the nonconvex problem is complicated by stochasticity, the sample complexity of stochastic first-order methods may depend linearly on the problem dimension, which is undesirable for large-scale problems. To alleviate this linear dependence, we adopt non-Euclidean settings and propose the usage of non-smooth proximal terms when taking the stochastic gradient steps. This approach leads to stronger convergence metric, incremental computational overhead, and potentially dimension-insensitive sample complexity. We also consider further acceleration through variance reduction which achieves near optimal sample complexity and, to our best knowledge, is the first such result in the l_1/l_∞ setting.

I will also present an Armijo-enabled stochastic linesearch framework with standard zeroth- and first-order oracles. The resulting steplength sequence is non-monotone and requires neither knowledge of Lipschitz smoothness constant L nor any other problem parameters – hence (problem-)parameter-free. The proposed method allows for a simple nonsmooth convex component in the objective, addressed through proximal gradient updates. Analogous guarantees are provided in the Polyak-Łojasiewicz (PL) setting and convex regimes. Preliminary numerical experiments are seen to be promising.