MIT News - Laplacians
http://news.mit.edu/topic/mitlaplacians-rss.xml
MIT News is dedicated to communicating to the media and the public the news and achievements of the students, faculty, staff and the greater MIT community.enFri, 01 Mar 2013 15:00:03 -0500Short algorithm, long-range consequences
http://news.mit.edu/2013/short-algorithm-long-range-consequences-0301
A new technique for solving ‘graph Laplacians’ is drastically simpler than its predecessors, with implications for a huge range of practical problems.Fri, 01 Mar 2013 15:00:03 -0500Larry Hardesty, MIT News Officehttp://news.mit.edu/2013/short-algorithm-long-range-consequences-0301<p>In the last decade, theoretical computer science has seen remarkable progress on the problem of solving graph Laplacians — the esoteric name for a calculation with hordes of familiar applications in scheduling, image processing, online product recommendation, network analysis, and scientific computing, to name just a few. Only in 2004 did researchers first propose an algorithm that solved graph Laplacians in “nearly linear time,” meaning that the algorithm’s running time didn’t increase exponentially with the size of the problem.<br />
</p>
<div class="video_captions" style="width: 368px; float: right; margin: 0 0 10px 10px;"><img src="/sites/mit.edu.newsoffice/files/images/2013/laplacians.gif" style="border-width: 0px; border-style: solid; width: 368px;" /> <span class="image_caption">This animation shows two different "spanning trees" for a simple graph, a grid like those used in much scientific computing. The speedups promised by a new MIT algorithm require "low-stretch" spanning trees (green), in which the paths between neighboring nodes don't become excessively long (red).</span> <span class="image_credit">Images courtesy of the researchers</span></div>
<p>At this year’s ACM Symposium on the Theory of Computing, MIT researchers will present <a href="http://arxiv.org/pdf/1301.6628" target="_blank">a new algorithm</a> for solving graph Laplacians that is not only faster than its predecessors, but also drastically simpler. “The 2004 paper required fundamental innovations in multiple branches of mathematics and computer science, but it ended up being split into three papers that I think were 130 pages in aggregate,” says Jonathan Kelner, an associate professor of applied mathematics at MIT who led the new research. “We were able to replace it with something that would fit on a blackboard.”<br />
<br />
The MIT researchers — Kelner; Lorenzo Orecchia, an instructor in applied mathematics; and Kelner’s students Aaron Sidford and Zeyuan Zhu — believe that the simplicity of their algorithm should make it both faster and easier to implement in software than its predecessors. But just as important is the simplicity of their conceptual analysis, which, they argue, should make their result much easier to generalize to other contexts.<br />
<br />
<strong>Overcoming resistance</strong><br />
<br />
A graph Laplacian is a matrix — a big grid of numbers — that describes a <a href="/newsoffice/2012/explained-graphs-computer-science-1217.html" target="_blank">graph</a>, a mathematical abstraction common in computer science. A graph is any collection of nodes, usually depicted as circles, and edges, depicted as lines that connect the nodes. In a logistics problem, the nodes might represent tasks to be performed, while in an online recommendation engine, they might represent titles of movies.<br />
<br />
In many graphs, the edges are “weighted,” meaning that they have different numbers associated with them. Those numbers could represent the cost — in time, money or energy — of moving from one step to another in a complex logistical operation, or they could represent the strength of the correlations between the movie preferences of customers of an online video service.<br />
<br />
The Laplacian of a graph describes the weights between all the edges, but it can also be interpreted as a series of linear equations. Solving those equations is crucial to many techniques for analyzing graphs.<br />
<br />
One intuitive way to think about graph Laplacians is to imagine the graph as a big electrical circuit and the edges as resistors. The weights of the edges describe the resistance of the resistors; solving the Laplacian tells you how much current would flow between any two points in the graph.<br />
<br />
Earlier approaches to solving graph Laplacians considered a series of ever-simpler approximations of the graph of interest. Solving the simplest provided a good approximation of the next simplest, which provided a good approximation of the next simplest, and so on. But the rules for constructing the sequence of graphs could get very complex, and proving that the solution of the simplest was a good approximation of the most complex required considerable mathematical ingenuity.<br />
<br />
<strong>Looping back</strong><br />
<br />
The MIT researchers’ approach is much more straightforward. The first thing they do is find a “spanning tree” for the graph. A tree is a particular kind of graph that has no closed loops. A family tree is a familiar example; there, a loop might mean that someone was both parent and sibling to the same person. A spanning tree of a graph is a tree that touches all of the graph’s nodes but dispenses with the edges that create loops. Efficient algorithms for constructing spanning trees are well established.<br />
<br />
The spanning tree in hand, the MIT algorithm then adds back just one of the missing edges, creating a loop. A loop means that two nodes are connected by two different paths; on the circuit analogy, the voltage would have to be the same across both paths. So the algorithm sticks in values for current flow that balance the loop. Then it adds back another missing edge and rebalances.<br />
<br />
In even a simple graph, values that balance one loop could imbalance another one. But the MIT researchers showed that, remarkably, this simple, repetitive process of adding edges and rebalancing will converge on the solution of the graph Laplacian. Nor did the demonstration of that convergence require sophisticated mathematics: “Once you find the right way of thinking about the problem, everything just falls into place,” Kelner explains.<br />
<br />
<strong>Paradigm shift</strong><br />
<br />
Daniel Spielman, a professor of applied mathematics and computer science at Yale University, was Kelner’s thesis advisor and one of two co-authors of the 2004 paper. According to Spielman, his algorithm solved Laplacians in nearly linear time “on problems of astronomical size that you will never ever encounter unless it’s a much bigger universe than we know. Jon and colleagues’ algorithm is actually a practical one.”<br />
<br />
Spielman points out that in 2010, researchers at Carnegie Mellon University also presented a practical algorithm for solving Laplacians. Theoretical analysis shows that the MIT algorithm should be somewhat faster, but “the strange reality of all these things is, you do a lot of analysis to make sure that everything works, but you sometimes get unusually lucky, or unusually unlucky, when you implement them. So we’ll have to wait to see which really is the case.”<br />
<br />
The real value of the MIT paper, Spielman says, is in its innovative theoretical approach. “My work and the work of the folks at Carnegie Mellon, we’re solving a problem in numeric linear algebra using techniques from the field of numerical linear algebra,” he says. “Jon’s paper is completely ignoring all of those techniques and really solving this problem using ideas from data structures and algorithm design. It’s substituting one whole set of ideas for another set of ideas, and I think that’s going to be a bit of a game-changer for the field. Because people will see there’s this set of ideas out there that might have application no one had ever imagined.”</p>
Image courtesy of the researchersAlgorithms, Graph theory, Laplacians, Linear algebra, Graph Laplacian, Mathematics, Theoretical computer scienceFirst improvement of fundamental algorithm in 10 years
http://news.mit.edu/2010/max-flow-speedup-0927
The max-flow problem, which is ubiquitous in network analysis, scheduling, and logistics, can now be solved more efficiently than ever.Mon, 27 Sep 2010 04:00:01 -0400Larry Hardesty, MIT News Officehttp://news.mit.edu/2010/max-flow-speedup-0927The maximum-flow problem, or max flow, is one of the most basic problems in computer science: First solved during preparations for the Berlin airlift, it’s a component of many logistical problems and a staple of introductory courses on algorithms. For decades it was a prominent research subject, with new algorithms that solved it more and more efficiently coming out once or twice a year. But as the problem became better understood, the pace of innovation slowed. Now, however, MIT researchers, together with colleagues at Yale and the University of Southern California, have <a href="http://math.mit.edu/~kelner/Publications/Docs/maxFlow.pdf" target="_blank">demonstrated</a> the first improvement of the max-flow algorithm in 10 years.<br /><br />The max-flow problem is, roughly speaking, to calculate the maximum amount of “stuff” that can move from one end of a network to another, given the capacity limitations of the network’s links. The stuff could be data packets traveling over the Internet or boxes of goods traveling over the highways; the links’ limitations could be the bandwidth of Internet connections or the average traffic speeds on congested roads.<br /><br />More technically, the problem has to do with what mathematicians call graphs. A graph is a collection of vertices and edges, which are generally depicted as circles and the lines connecting them. The standard diagram of a communications network is a graph, as is, say, a family tree. In the max-flow problem, one of the vertices in the graph — one of the circles — is designated the source, where all the stuff comes from; another is designated the drain, where all the stuff is headed. Each of the edges — the lines connecting the circles — has an associated capacity, or how much stuff can pass over it. <br /><br /><strong>Hidden flows</strong><br /><br />Such graphs model real-world transportation and communication networks in a fairly straightforward way. But their applications are actually much broader, explains Jonathan Kelner, an assistant professor of applied mathematics at MIT, who helped lead the new work. “A very, very large number of optimization problems, if you were to look at the fastest algorithm right now for solving them, they use max flow,” Kelner says. Outside of network analysis, a short list of applications that use max flow might include airline scheduling, circuit analysis, task distribution in supercomputers, digital image processing, and DNA sequence alignment.<br /><br />Traditionally, Kelner explains, algorithms for calculating max flow would consider one path through the graph at a time. If it had unused capacity, the algorithm would simply send more stuff over it and see what happened. Improvements in the algorithms’ efficiency came from cleverer and cleverer ways of selecting the order in which the paths were explored.<br /><br /><strong>Graphs to grids</strong><br /><br />But Kelner, CSAIL grad student Aleksander Madry, math undergrad Paul Christiano, and Professors Daniel Spielman and Shanghua Teng of, respectively, Yale and USC, have taken a fundamentally new approach to the problem. They represent the graph as a matrix, which is math-speak for a big grid of numbers. Each node in the graph is assigned one row and one column of the matrix; the number where a row and a column intersect represents the amount of stuff that may be transferred between two nodes. <br /><br />In the branch of mathematics known as linear algebra, a row of a matrix can also be interpreted as a mathematical equation, and the tools of linear algebra enable the simultaneous solution of all the equations embodied by all of a matrix’s rows. By repeatedly modifying the numbers in the matrix and re-solving the equations, the researchers effectively evaluate the whole graph at once. This approach, which Kelner will describe at <a href="http://www.csail.mit.edu/events/eventcalendar/calendar.php?show=event&id=2711" target="_blank">a talk at MIT's Stata Center on Sept. 28</a>, turns out to be more efficient than trying out paths one by one.<br /><br />If N is the number of nodes in a graph, and L is the number of links between them, then the execution of the fastest previous max-flow algorithm was proportional to (N + L)<sup>(3/2)</sup>. The execution of the new algorithm is proportional to (N + L)<sup>(4/3)</sup>. For a network like the Internet, which has hundreds of billions of nodes, the new algorithm could solve the max-flow problem hundreds of times faster than its predecessor.<br /><br />The immediate practicality of the algorithm, however, is not what impresses John Hopcroft, the IBM Professor of Engineering and Applied Mathematics at Cornell and a recipient of the Turing Prize, the highest award in computer science. “My guess is that this particular framework is going to be applicable to a wide range of other problems,” Hopcroft says. “It’s a fundamentally new technique. When there’s a breakthrough of that nature, usually, then, a subdiscipline forms, and in four or five years, a number of results come out.”<br /><br />Graphic: Christine DaniloffAlgorithms, Computer Science and Artificial Intelligence Laboratory (CSAIL), Computer science and technology, Laplacians, Max-flow