Today, Microprocessor performance far exceeds that of traditional supercomputers costing many times more. This cost advantage allows a system of scalable computing resources to be built in many different ways: clusters, MPPs, and SMPs to name a few. Because high-speed, low-latency standard communications technologies now rival those available from proprietary sources, the open systems cluster paradigm can flourish. Software for several layers of standard tools for distributed computing now exists. It ranges from communications protocols like UDP/IP and TCP/IP to high-level programming environments such as OSF's DCE and ONC+.
In my experience over the past 10 years, he best reasons to favor clusters over vendor proprietary MPP solutions have little to do with technology but rather with complexity and market share. System administration costs for clusters are vastly lower than for dispersed workstation resources. The third party software available for the workstation components makes clusters far more attractive than their MPP brethren. Simply put, clusters leverage the benefits of the component platforms.
A typical supercomputers day-to-day workload resembles a cascade of boulders and sand. The boulders represent applications with very large memory, CPU, and disk requirements; the sand is at the other end of the spectrum. Typically, many of these applications are very complex legacy codes for whom it is not cost effective to attempt to parallelize. Accommodating this workload requires a cluster of elements with very large memories. Because memory quickly dominates the cost of the cluster, it is worthwhile to add more processors to each memory element. The boulders and sand workload and the SMP's cost effectiveness in matching this workload is the fundamental motivation for SMP cluster computing.
While deciding whether SMP is right for your organization, it is important to understand its place in the quest for computing power. Individual machines (nodes) in a cluster, which themselves can be SMP (symmetric multiprocessing) machines, communicate via high-speed connections--such as Infiniband, Gigabit switched or Myrinet and proprietary interconnects as numaflex, to give a single application access to the CPUs, disks, and memory of all connected nodes. On the downside, the overhead required to coordinate disk and CPU access from one machine to another prevents clusters from scaling as efficiently as SMP implementations, which have closer ties between processors, memory, and disks.
Given these options, how does one determine which architectures to deploy ? My view is that this will mainly depended on the applications required to run on the systems. The user with a broad application mix, which is the case most of the time, may choose to deploy both clusters and large shared-memory systems, each running the applications best suited for them in different compute workflows. As computing problems grow increasingly complex and data intensive, the real challenge is not how to buy the most processors for the budget, but how to best map the numerous architectural options to the users’ specific problems.
As leading processor vendors as Intel and AMD turn to muti-core architectures, the compute horsepower available to clusters is on the rise, along with more and more scientific and technical applications are being written in MPI, this provides clusters with good application availability.
I see the market share of cluster systems increase in the TOP 500 supercomputers in the years to come, with large SMP systems being confined to specific application usage.
In my experience over the past 10 years, he best reasons to favor clusters over vendor proprietary MPP solutions have little to do with technology but rather with complexity and market share. System administration costs for clusters are vastly lower than for dispersed workstation resources. The third party software available for the workstation components makes clusters far more attractive than their MPP brethren. Simply put, clusters leverage the benefits of the component platforms.
A typical supercomputers day-to-day workload resembles a cascade of boulders and sand. The boulders represent applications with very large memory, CPU, and disk requirements; the sand is at the other end of the spectrum. Typically, many of these applications are very complex legacy codes for whom it is not cost effective to attempt to parallelize. Accommodating this workload requires a cluster of elements with very large memories. Because memory quickly dominates the cost of the cluster, it is worthwhile to add more processors to each memory element. The boulders and sand workload and the SMP's cost effectiveness in matching this workload is the fundamental motivation for SMP cluster computing.
While deciding whether SMP is right for your organization, it is important to understand its place in the quest for computing power. Individual machines (nodes) in a cluster, which themselves can be SMP (symmetric multiprocessing) machines, communicate via high-speed connections--such as Infiniband, Gigabit switched or Myrinet and proprietary interconnects as numaflex, to give a single application access to the CPUs, disks, and memory of all connected nodes. On the downside, the overhead required to coordinate disk and CPU access from one machine to another prevents clusters from scaling as efficiently as SMP implementations, which have closer ties between processors, memory, and disks.
Given these options, how does one determine which architectures to deploy ? My view is that this will mainly depended on the applications required to run on the systems. The user with a broad application mix, which is the case most of the time, may choose to deploy both clusters and large shared-memory systems, each running the applications best suited for them in different compute workflows. As computing problems grow increasingly complex and data intensive, the real challenge is not how to buy the most processors for the budget, but how to best map the numerous architectural options to the users’ specific problems.
As leading processor vendors as Intel and AMD turn to muti-core architectures, the compute horsepower available to clusters is on the rise, along with more and more scientific and technical applications are being written in MPI, this provides clusters with good application availability.
I see the market share of cluster systems increase in the TOP 500 supercomputers in the years to come, with large SMP systems being confined to specific application usage.
No comments:
Post a Comment