SAN JOSE, Calif. — Six companies are contributing a total $6 million to kick off a three-year project at Stanford University to explore fresh models for parallel programming. The effort is one of three recently funded by a computer industry increasingly concerned software cannot keep pace with the evolution of multicore processors.
Advanced Micro Devices, Hewlett-Packard, IBM, Intel, NVidia and Sun Microsystems are funding Stanford's new Pervasive Parallelism Lab which will consist of about nine faculty and as many as 30 graduate students. Kunle Olukotun, a Stanford computer science professor seen as the father of Sun's multicore Niagara processor, will head the new lab.
"People are starting to build multicore hardware without really knowing how they can productively program it and that's becoming a huge problem, said Bill Dally, chairman of Stanford's computer science department. "The current use of threads and locks is error prone and hard to maintain, so if we don't find a new approach soon we will be saddled with some very bad legacy software," he added.
The problem requires genuine breakthroughs in computer science. Top researchers worked unsuccessfully for more than a decade to develop parallel programming models for high-end supercomputers, often funded by the Defense Advanced Research Projects Agency (DARPA).
"The last seven or eight years has been something of a Dark Ages in research on parallelism after DARPA pulled its funding for the area," said Dally. "Now people are seeing a real problem and the industry is feeling the pain and so dropping a lot of money into it," he said.
In March, Intel and Microsoft officially announced their plan to spend a total of $20 million over five years to fund work at new parallel computing labs at the University of California at Berkeley and University of Illinois at Urbana-Champaign. To date, Berkeley researchers have been the most specific about how they will address the technical issues.
At a high level, the Stanford approach sounds strikingly similar to the one the Berkeley lab adopted. That doesn't prevent the separate groups from coming up with very different solutions to the many thorny problems ahead.
Indeed, some see a healthy competition on the technology brewing between the Bay Area academic giants that could benefit the computer industry as a whole. "There's a need for many people to be working on these problems," said Dally.
Both Berkeley and Stanford will task some researchers to develop next-generation applications using domain-specific languages. Both will develop new runtime environment to help automate the job of scheduling and synchronizing many processes running in parallel.
In addition, both are researching new hardware structures that could ease the parallel programming job. And both aim to use FPGA-based simulators to test out their work.
Stanford has asked researchers who won a recent DARPA Grand Challenge in robotic vehicle design to write a parallel app in their field. Another group will write a parallel app for running a virtual world.
The runtime effort will attempt to merge the best of two different approaches developed at Stanford. Dally has pioneered work in stream computing that uses a very structured way of scheduling computations on large data sets. It will be combined with Olukotun's work on transactional memory, an alternative to data locking, which uses dynamic scheduling techniques.
On the hardware front, Stanford is exploring synchronization techniques at three levels. It will look into way to share registers within a single clock cycle between neighboring cores as well as ways to share related threads across a die within ten clock cycles using message passing techniques. Finally, it will explore new systems-level synchronization techniques leveraging new data structures and language semantics.
"We could quickly get to the stage where CPUs have more cores than threads, so we have to look at ways to spawn little threads to do more work in parallel," Dally explained.
The new lab hopes to show prototypes of its separate projects working in tandem on an FPGA-based system simulator by the time its initial three-year term ends. "Ideally this effort will continue on beyond the current three-year term," Dally said.
Stanford has its own simulator, called the Stanford Farm. However, it has also worked with Berkeley and other universities on the so-called Research Accelerator for Multiple Processors (RAMP). Microsoft research has helped develop a third-generation RAMP system that Berkeley's lab will use.
The Stanford lab plans to make all the results of its work publically available. Similarly, Berkeley researchers said they would publish their results in academic journals and make it available probably through the BSD open source licensing.
