优化理论入门学习路径及面向数学学习者的参考资料咨询

Hey there! Let's break this down for you since you're aiming to dive into optimization theory with a math-focused approach, especially with an eye toward ML later on.

First, let's address the two books you mentioned:

• Convex Optimization by Boyd and Vandenberghe: While it’s often framed for practitioners, its theoretical foundations are actually solid—though it does prioritize intuitive, application-focused explanations over ultra-rigorous mathematical proofs. If your goal is to dig into the "why" behind algorithms at a deep mathematical level, it might leave you wanting more in terms of formal theory.
• Bierlaire's Optimization book: As you noticed, it’s light on convex optimization, which is indeed a foundational pillar for both ML and modern optimization theory. So it’s probably not the best core resource for your needs.

Now, here are some math-focused introductory references that align better with your goals:

• 《Convex Analysis》by R.T. Rockafellar: This is the gold standard for convex optimization theory, written entirely from a rigorous mathematical perspective. It starts with fundamental concepts like convex sets and functions, then dives deep into duality theory, subgradients, and convex conjugacy—all the theoretical building blocks you’ll need to understand why convex optimization algorithms work. It’s a staple in graduate-level math optimization courses.
• 《Nonlinear Programming》by Dimitri Bertsekas: Split into theoretical and algorithmic sections, the theory portion is extremely rigorous, covering unconstrained and constrained optimization, KKT conditions, duality, and convergence analysis of algorithms. It balances deep mathematical theory with connections to practical algorithms, making it great for linking theory to the ML algorithms you’ll encounter later.
• 《Optimization by Vector Space Methods》by David G. Luenberger: This book approaches optimization from a functional analysis/vector space perspective, which is highly mathematical and abstract. It’s perfect if you want to build a more general, theoretical framework for optimization—useful if you plan to explore advanced ML topics that involve infinite-dimensional spaces down the line.
• 《Introduction to Optimization》by Edwin Chong and Stanislaw Zak: A more accessible yet math-focused entry point. It covers linear programming, convex optimization, and nonlinear optimization basics with clear, rigorous derivations, making it ideal for transitioning from your math background to core optimization concepts without being overly intimidating.

For a learning path that builds gradually:

1. Start with Introduction to Optimization (Chong & Zak) to lay a broad, solid foundation of basic optimization concepts and proofs.
2. Move on to Convex Analysis (Rockafellar) to master the theoretical core of convex optimization—this is critical for understanding the underpinnings of most ML optimization algorithms.
3. Then tackle Nonlinear Programming (Bertsekas) to expand into non-convex optimization theory and deepen your understanding of algorithm convergence, which will help you grasp exactly why ML algorithms like gradient descent or Newton’s method behave the way they do.
4. If you want to go further, Optimization by Vector Space Methods (Luenberger) will give you a more abstract, generalized view of optimization theory.

As a side note: The two books you initially found can still serve as supplementary resources. After you’ve built a strong theoretical base, Boyd’s book can help you see how that theory translates to practical algorithm design, and Bierlaire’s book might fill in some gaps in non-convex optimization topics if you’re curious.

备注：内容来源于stack exchange，提问作者DarkCube