Optimization is a core methodology in operations research and has been widely used in data science, machine learning, and management applications. Motivated by the growing size of data, there has been a tremendous amount of recent research on scaling up and speeding up the solving of optimization problems during the past two decades. The goal of the course is twofold: (i) introducing the recent developments of large-scale optimization algorithms with solid theoretical guarantees; (ii) introducing the new computational toolbox for large-scale optimization problems. We will particularly discuss how the recent development in large-scale optimization can help scale up two important classes of optimization problems, that is, linear programming and deep learning. We will use python for the computational tasks.