Continuous Optimization for Data Science

The text is divided into three main parts: unconstrained optimization, constrained optimization, and linear programming. The first part addresses unconstrained optimization in single-variable and multivariable functions, introducing key algorithms such as steepest descent, Newton, and quasi-Newton methods. The second part focuses on constrained optimization, starting with linear equality constraints and extending to more general cases, including inequality constraints. It details optimality conditions, sensitivity analysis, and relevant algorithms for solving these problems. The third part covers linear programming, presenting the formulation of LP problems, the simplex algorithm, and sensitivity analysis. Throughout, the text provides numerous applications to data science, such as linear regression, maximum likelihood estimation, expectation-maximization algorithms, support vector machines, and linear neural networks.