Large-Scale Optimization

Optimization is a modeling tool that has been used in many engineering fields for a long time. It can be used, e.g., for optimal control, financial decision making, signal reconstruction, route planning, statistical estimation, and training of supervised learning machines. Different optimization problems have different properties and fall into different categories. They can be coarsely divided into convex or nonconvex problems, smooth or nonsmooth problems, and small-scale or large-scale problems. Contemporary optimization problems in, e.g., machine learning, signal reconstruction, control, and statistical estimation are often large-scale. The research in this group is focused on understanding and developing efficient algorithms for solving such problems. We focus on convex and nonsmooth problems with a primary focus is on so-called operator splitting methods and their stochastic variants. In particular, we develop frameworks for understanding a wide range of operator splitting methods that allow for a unified analysis and paves the way for design of new and improved algorithms. We also develop tools for automated algorithm analysis in which a so-called performance estimation optimization problem is formulated that exactly captures the worst possible performance of an optimization algorithm for some user-specified class of optimization problems. A solution to this, typically small-scale, performance estimation problem can give convergence guarantees for the analyzed algorithm.