My main research focus is within optimization, which is a modeling tool that has been extensively used as a core component for a wide range of problems, such as, optimal control, financial decision making, signal reconstruction, route planning, statistical estimation, and machine learning training. Different optimization problems have different properties and fall into different categories. They can be coarsely divided into convex or nonconvex problems, smooth or nonsmooth problems, and small-scale or large-scale problems. Contemporary optimization problems in, e.g., machine learning, signal reconstruction, control, and statistical estimation are often large-scale. The research in this group is focused on understanding and developing efficient algorithms for solving such problems. We focus on convex and nonsmooth problems with a primary focus is on so-called operator splitting methods and their stochastic variants. In particular, we develop frameworks for understanding a wide range of operator splitting methods that allow for a unified analysis and paves the way for design of new and improved algorithms. We also develop tools for automated algorithm analysis in which a so-called performance estimation optimization problem is formulated that exactly captures the worst possible performance of an optimization algorithm for some user-specified class of optimization problems. A solution to this, typically small-scale, performance estimation problem can give convergence guarantees for the analyzed algorithm.