The application we were testing the MPI optimization for is the “SMHI RCO application” which can use the ScaliMPI or the PMPI (Platform MPI).

At first we have tested the application performance on Scali MPI and Platform MPI and achieved the following results (using 144 ranks, 18 hosts 8 ranks each on the “Helios” cluster). The original Scali MPI based run was 474 seconds and the original Platform-MPI was 550 seconds.

We have modified the Platform MPI and the optimized results with Platform MPI demonstrated 450 seconds for the application run.

We want to thank the HPC Advisory Council for providing the resources for us to evaluate our optimizations and provide a better solution for the customer.

Regards,
Perry Schmidt
Platform

Share and Bookmark

First up: The HPC Advisory Council and the Swiss Supercomputing Centre will host the HPC Advisory Council Switzerland Workshop 2010  in the Lugano Convention Centre, Lugano, Switzerland, from March 15-17, 2010. This is an excellent training and educational opportunity for European-based, HPC and data center IT professionals (See AGENDA). Attendance is FREE and registration is now OPEN.

Second: The HPC Advisory Council will hold the 2010 European Workshop on May 30th, 2010, in conjunction with the ISC’10 conference in Hamburg Germany. The workshop will be a full day workshop, and will include coffee breaks, lunch and dinner, courtesy of the HPC Advisory Council. Attendance to the workshop is a nominal fee of  30€ . Register today for the conference by clicking on the Council Workshop box when registering for ISC.

More workshop are being planned for the second half of 2010. Stay tuned for more info.

Hope to see you all there,

Brian Sparks

Share and Bookmark

With that, I wanted to draw your attention to a partnership between Platform and Instrumental to advance High Performance Cloud Computing for the Government Sector.  The partnership enhances Platform’s global service capabilities and gives users an end-to-end, full service solution that maximizes the value of Platform’s private cloud management and HPC cloud-enabling software solutions, Platform ISF and Platform ISF Adaptive Cluster.  The full press release can be seen here. 

Dave Ellis
Principal Technologist
Instrumental, Inc.

Share and Bookmark

Video is posted here: http://insidehpc.com/hpc-2010-technology-issues-opportunities/

Share and Bookmark

One of our council members, Scalable Graphics, helped to create a highly visual demo of a Peugeot car rendering in 3D (see picture below). The demo included an interactive component, where viewers wear glasses that tracked their position and moved the image accordantly.

Also during the conference, we held our semi-annual meeting, where we reviewed our plans for 2010. To recap, we will be building off our successful HPC Advisory Council Workshop in China and we will be hosting 4 workshops next year; the first one in March will be hosted in Switzerland together with the Swiss Supercomputing Center, and the second in May as part of the International Supercomputing Conference. More info can be found on the council main web pages.

Best regards,

Gilad, HPC Advisory Council Chairman.

Share and Bookmark

In summary, there are a large number of HPC applications that need the lowest possible latency for best performance or the highest bandwidth (for example Oil&Gas applications as well as weather related applications). There are some HPC applications that are not latency sensitive. For example, gene sequencing and some bioinformatics applications are not sensitive to latency and scale well with TCP-based networks including GigE and 10GigE. For HPC converged networks, putting HPC message passing traffic and storage traffic on a single TCP network may not provide enough data throughput for either. Finally, there is a number of examples that show 10GigE has limited scalability for HPC applications and InfiniBand proves to be a better performance, price/performance, and power solution than 10GigE.

The complete report can be found under the HPC Advisory Council case studies or by clicking here.

Share and Bookmark

The workshop was well organized by Ada Gavrilovska (Georgia Tech) and Pavan Balaji (Argonne National Lab) with the help from Steve Poole (Oak Ridge National Lab). Other interesting sessions were given by Nagi Rao (ORNL) on wide area InfiniBand, James Hofmann (Naval Research Lab) on Large Data project, Hari Subramoni (The Ohio State University) on InfiniBand RDMA over Ethernet (LLE) and others.

The next council event is the HPC China workshop. More data on the workshop is posted at – http://www.hpcadvisorycouncil.com/events/china_workshop/

Gilad Shainer HPC Advisory Council Chairman
gilad@hpcadvisorycouncil.com

Share and Bookmark

A deeper examination of the world’s Top500 systems based on commodity clusters shows two main interconnect solutions that are being used to connect the servers for creating those compute powerful systems – InfiniBand and Ethernet. If we divide the systems according to the interconnect family, we will see that the same CPUs, memory speed and other settings are common between the two groups. The only difference between the two groups, besides the interconnect, is the system efficiency, or how many of CPU cycles can be dedicated to the application work, and how many of them will be wasted. The below graph list the systems according to their interconnect setting, and their measured efficiency.

As seen, systems connected with Ethernet achieves an average 50% efficiency, which means that 50% of the CPU cycles are wasted on non-application work or are idle, waiting for data to arrive.  Systems connected with InfiniBand achieve an above 80% efficiency average, which means that less than 20% of the CPU cycles are wasted. Moreover, the latest InfiniBand based systems have demonstrated up to 94% efficiency (the best Ethernet connected systems demonstrated 63% efficiency).

People might argue that the Linpack benchmark is not the best benchmark for measuring parallel application efficiency, and does not fully utilize the network. The graph results are a clear indication that even for the Linpack application, the network does make a difference, and for better parallel application, the gap will be much higher.

When choosing the system setting, with the notion of maximizing return on investment, one needs to make sure no artificial bottlenecks will be created. Multi-core platforms, parallel applications, large databases etc require fast data exchange and lots of it. Ethernet can become the system bottleneck due to latency/bandwidth and CPU overhead due to the TCP/UDP processing (TOE solutions introduce other issues, sometime more complicated, but this is a topic for another blog) and reduce the system efficiency to 50%. This means that half of the compute system is wasted, and just consumes power and cooling. Same performance capability could have been achieved with half of the servers if they were connected with InfiniBand. More data on different application performance, productivity and ROI, can be found at the HPC Advisory Council web site, under content/best practices.

While InfiniBand will demonstrate higher efficiency and productivity, there are several ways to increase Ethernet efficiency. One of them is optimizing the transport layer to provide zero copy and lower CPU overhead (not by using TOE solutions, as those introduce single points of failure in the system). This capability is known as LLE (low latency Ethernet). More on LLE will be discussed in future blogs.

Gilad Shainer HPC Advisory Council Chairman
gilad@hpcadvisorycouncil.com

Share and Bookmark