Distributed optimization and statistical learning via the alternating direction method of multipliers [electronic resource] / Stephen Boyd ... [et al.].

Contributor(s):

Boyd, Stephen

Material type: Text

TextSeries: Foundations and trends in machine learning (Online) ; v. 3, issue 1, p. 1-122.Publication details: Hanover, Mass. : Now Publishers, c2011.Description: 1 electronic text (122 p.) : ill., digital fileISBN:

9781601984616 (electronic)

Subject(s):

DDC classification:

004.01/5196 22

LOC classification:

QA76.9.C62 B693 2011eb

Online resources:

Abstract with links to full text:

Available additional physical forms:

Also available in print.

Contents:

1. Introduction -- 2. Precursors -- 3. Alternating direction method of multipliers -- 4. General patterns -- 5. Constrained convex optimization -- 6. l-norm problems -- 7. Consensus and sharing -- 8. Distributed model fitting -- 9. Nonconvex problems -- 10. Implementation -- 11. Numerical examples -- 12. Conclusions -- Acknowledgments -- A Convergence proof -- References.

Abstract: Many problems of recent interest in statistics and machine learning can be posed in the framework of convex optimization. Due to the explosion in size and complexity of modern datasets, it is increasingly important to be able to solve problems with a very large number of features or training examples. As a result, both the decentralized collection or storage of these datasets as well as accompanying distributed solution methods are either necessary or at least highly desirable. In this review, we argue that the alternating direction method of multipliers is well suited to distributed convex optimization, and in particular to large-scale problems arising in statistics, machine learning, and related areas. The method was developed in the 1970s, with roots in the 1950s, and is equivalent or closely related to many other algorithms, such as dual decomposition, the method of multipliers, Douglas-Rachford splitting, Spingarn's method of partial inverses, Dykstra's alternating projections, Bregman iterative algorithms for /1 problems, proximal methods, and others. After briefly surveying the theory and history of the algorithm, we discuss applications to a wide variety of statistical and machine learning problems of recent interest, including the lasso, sparse logistic regression, basis pursuit, covariance selection, support vector machines, and many others. We also discuss general distributed optimization, extensions to the nonconvex setting, and efficient implementation, including some details on distributed MPI and Hadoop MapReduce implementations.

Holdings ( 1 )
Title notes ( 18 )

Holdings
Item type	Current library	Collection	Call number	Status	Date due	Barcode	Item holds
eBook	e-Library	EBook		Available

Total holds: 0

Includes bibliographical references (p. 111-122).

Restricted to subscribers or individual document purchasers.

Google Scholar

Google Book Search

INSPEC

Scopus

ACM Computing Guide

DBPLP Computer Science Bibliography

Zentralblatt MATH Database

AMS MathSciNet

ACM Computing Reviews

Many problems of recent interest in statistics and machine learning can be posed in the framework of convex optimization. Due to the explosion in size and complexity of modern datasets, it is increasingly important to be able to solve problems with a very large number of features or training examples. As a result, both the decentralized collection or storage of these datasets as well as accompanying distributed solution methods are either necessary or at least highly desirable. In this review, we argue that the alternating direction method of multipliers is well suited to distributed convex optimization, and in particular to large-scale problems arising in statistics, machine learning, and related areas. The method was developed in the 1970s, with roots in the 1950s, and is equivalent or closely related to many other algorithms, such as dual decomposition, the method of multipliers, Douglas-Rachford splitting, Spingarn's method of partial inverses, Dykstra's alternating projections, Bregman iterative algorithms for /1 problems, proximal methods, and others. After briefly surveying the theory and history of the algorithm, we discuss applications to a wide variety of statistical and machine learning problems of recent interest, including the lasso, sparse logistic regression, basis pursuit, covariance selection, support vector machines, and many others. We also discuss general distributed optimization, extensions to the nonconvex setting, and efficient implementation, including some details on distributed MPI and Hadoop MapReduce implementations.

Stephen Boyd, Neal Parikh, Eric Chu, Borja Peleato and Jonathan Eckstein (2011) "Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers", Foundations and Trends in Machine Learning: Vol. 3: No 1, pp 1-122.

Also available in print.

Mode of access: World Wide Web.

System requirements: Adobe Acrobat Reader.

Title from PDF (viewed on September 12, 2011).

Back to results