Author Archives: brian

Inauguration of 1st European Petaflop Computer in Jülich, Germany

On Tuesday, May 26, the Research Center Jülich reached a significant milestone of German and European supercomputing with the inauguration of two new supercomputers: the supercomputer JUROPA and the fusion machine HPC FF. The symbolic start of the systems were triggered by the German Federal Minister for Education and Research, Prof. Dr. Annette Schavan, the Prime Minister of North Rhine-Westphalia, Dr. Jürgen Rüttgers, and Prof. Dr. Achim Bachem, Chairman of the Board of Directors at Research Center Jülich as well as high-ranking international guests from academia, industry and politics.

JUROPA (which stands for Juelich Research on Petaflop Architectures) will be used Pan-European-wide by more than 200 research groups to run their data-intensive applications. JUROPA is based on a cluster configuration of Sun Blade servers, Intel Nehalem processors, Mellanox 40Gb/s InfiniBand and Cluster Operation Software ParaStation from ParTec Cluster Competence Center GmbH. The system was jointly developed by experts of the Jülich Supercomputing Center and implemented with partner companies Bull, Sun, Intel, Mellanox and ParTec. It consists of 2,208 compute nodes with a total computing power of 207 Teraflops and was sponsored by the Helmholtz Community. Prof. Dr. Dr. Thomas Lippert, Head of Jülich Supercomputing Center, explains the HPC Installation in Jülich in the video below.

HPC-FF (High Performance Computing – for Fusion), drawn up by the team headed by Dr. Thomas Lippert, director of the Jülich Supercomputing Centre, was optimized and implemented together with the partner companies Bull, SUN, Intel, Mellanox and ParTec. This new best-of-breed system, one of Europe’s most powerful, will support advanced research in many areas such as health, information, environment, and energy. It consists of 1,080 computing nodes each equipped with two Nehalem EP Quad Core processors from Intel. Their total computing power of 101 teraflop/s corresponds, at the present moment, to 30th place in the list of the world’s fastest supercomputers. The combined cluster will achieve 300 teraflops/s computing power and will be included in the rating of the Top500 list, published this month at ISC’09 in Hamburg, Germany.

40Gb/s InfiniBand from Mellanox is used as the system interconnect. The administrative infrastructure is based on NovaScale R422-E2 servers from French supercomputer manufacturer Bull, who supplied the compute hardware and the SUN ZFS/Lustre Filesystem. The cluster operating system “ParaStation V5″ is supplied by Munich software company ParTec. HPC-FF is being funded by the European Commission (EURATOM), the member institutes of EFDA, and Forschungszentrum Jülich.

Complete System facts: 3288 compute nodes ; 79 TB main memory; 26304 cores; 308 Teraflops peak performance.

Gilad Shainer,
HPC Advisory Council Chairman

ScaleMP SPEC CPU Benchmark

ScaleMP just announced record-breaking results for x86 systems.  vSMP Foundation based platform is the world’s fastest x86 system based on the SPEC CPU 2006 benchmark.  SPECfp_rate_base2006 achieved is 666, which is the best ever x86-based result and 2x faster than the previous best x86 published result. This performance is achieved on 32 Intel Xeon (2.93GHz) cores with HT enabled connected with Mellanox QDR HCAs and switch.  It is among the top 30 results ever published. The official results can be viewed at web site


SPEC CPU Benchmark is the industry-standard, CPU-intensive benchmark suite, stressing a system’s processor, memory subsystem and compiler. It is designed to provide a comparative measure of compute-intensive performance across the widest practical range of hardware using workloads developed from real user applications.

ScaleMP continues to deliver on the unique and innovative value proposition vSMP Foundation delivers; delivering unmatched scalability and performance with the simplified operating model of large SMP systems at the cost of managed clusters, to bring tremendous value for High Performance Computing customers.  To put this in perspective, x86 virtual SMP systems based on Mellanox QDR and ScaleMP’s vSMP Foundation are performing equal to or better than traditional large systems that cost 2x to 3x the price.  It is also noteworthy that ScaleMP’s software was available to customers supporting the new Intel Nehalem processors the day they were launched, and fulfilling on the promise of delivering High Performance and Technical Computing to the masses.

You can find out more about ScaleMP and its products by going to

Shai Fultheim

Notes from the LS-DYNA Users Conference

I recently had the pleasure to go to Salzburg, Austria, and present in the 7th European LS-DYNA conference. LS-DYNA is a software solution from Livermore Software Technology Corporation (LSTC) that is a general purpose structural and fluid analysis simulation software package capable of simulating complex real world problems. It is widely used in the automotive industry for crashworthiness analysis, occupant safety analysis, metal forming and much more, and in most cases, LS-DYNA is being used in cluster environments as they provide the needed flexibility, scalability and efficiency for such simulations.

I have presented a paper on “LS-DYNA Productivity and Power-aware Simulations in Cluster Environments”. The paper was written by Mellanox, Dell and AMD with the help of Sharan Kalwani from GM and LSTC. The paper covers clustering interconnect analysis, CPUs performance, application and networking profiling and providing recommendations for increasing productivity (or jobs per day) while reducing power and cooling expenses. The paper can be downloaded from the “content/conferences” section on the HPC Advisory Council web site.

There were some very interesting sessions at the conference (beside mine …J). The automotive makers have expressed their need to be more economical and ecological (without compromising the brand names), the challenges of light weight design, increase demands from regulations, new materials, alternative drive engines, cost efficiency, increase safety, design of energy management equipment and much more. All of those items continue to increase the need for more simulation and higher complexity in order to create a design that fulfills those requirements and enables faster solution time to market. The paper that I presented provides information and guidelines on how to build next generation systems from one side, and how to optimize current systems for higher productivity on the other.

I also did manage to find some time in the late evening, and walk throughout the old city of Salzburg and see the house that Mozart was born in. It is a lovely city, with many sight seeing and nice places to sit down and drink beer (or coke, if you know me….).


Gilad Shainer
HPC Advisory Council Chairman

HPC Advisory Council at the 32nd HPC User Forum

This week the 32nd HPC User Forum was held in Roanoke, Virginia. This was a great opportunity to meet, talk and hear from industry experts and end-users. There were very interesting sessions on the state of high-performance computing, the current problems, and what work is necessary to move to exascale computing. It was also a great opportunity to meet many of the HPC Advisory Council members.

The HPC Advisory Council had a session during the HPC User Forum, and I would like to thank the members that participated in the panel, and in particular to Jennifer Koerv (AMD), Donnie Bell (Dell), Sharan Kalwani (GM), Lynn Lewis (Microsoft), Stan Posey (Panasas), Lee Porter (ParTec) and Arend Dittmer (Penguin Computing).

Some of the talks at the User Forum were on HPC futures, not only on building the next PetaScale/ExaScale supercomputers, but how to make HPC easier and more productive. Platform Computing talked on HPC in a cloud and services, and NVIDIA on using GPUs. This is one of the main research activities right now in the HPC Advisory Council – enabling efficient HPC as a Service (HPCaaS) and smart scheduling strategies. Initial results are available on the HPC Advisory Council web, and you are encouraged to take a look (the focus at the event was on bioscience applications). We will extend the research to add QCD codes (quantum chemistry), with the help and support from Fermi National Lab.

We are having our first member’s conference call on May 4th, so don’t forget to accept the invite that you got, and if you did not get it, please let me know at

Best Regards,

Gilad Shainer, HPC Advisory Council Chairman

A Q&A Roundtable with the HPC Advisory Council

We recently performed an interview with Addison Snell, General Manager at Tabor Research, where we highlighted the council’s activities for the past year, and provided some insight into our future direction.

Participating were:
Gilad Shainer – Chair, HPC Advisory Council
Brian Sparks – Media Relations Director, HPC Advisory Council
Gautam Shah – CEO, Colfax International
Scot Schultz – Senior Strategic Alliance Manager, AMD
Peter Lillian – Senior Product Marketing Manager, Dell

It’s amazing to me what the Council has been able to accomplish in under a year. Sometimes it all flies by so fast that you don’t have time to sit back and try to take it all in. Am I being a little grandiose here? Ya, sure, but a lot of folks from various companies have put in a huge amount of work…and it’s nice to see it all come into fruition where it benefits all members. Thank you everyone for helping the Council become what it is today.

You can find the whole interview here.

Talk with you soon,

Brian Sparks

ISC Demonstration and Member Meeting

Replicating our success from SC’08, the HPC Advisory Council has joined the ISCnet team at the International Supercomputing Conference (ISC’09) to showcase and demonstrate a large 40Gb/s InfiniBand eco-system, connecting multiple booths on the showfloor. Several members are providing the needed resources to support this activity (providing hardware, software and a 3D visualization demo for this network).

The ISCnet team will be dropping 40Gb/s InfiniBand fiber links to participating booths. Pease note: There is a conference fee (500.00 Euros) for each fiber drop. For more info and for a short presentation that provides additional details and system requirements for the demo please contact You can also read the case study of our activities during SC’08 here.

Please check it out and I hope that you will be able to take part in this demonstration. Please let me know if you have additional questions.Semi-annual meeting: We are also looking into possibilities of holding a HPC Advisory Council Member meeting during ISC’09. During the meeting we will meet and greet the council members, review the council activities (application expertise and HPC outreach) and review our plan for this year’s activities and goals. Please let us know if you or your group will be attending the show, and if you would be interested in attending by sending an email to

Gilad Shainer
HPC Advisory Council Chair

The HPC Advisory Council Cluster Center – update

Recently we have completed a small refresh in the cluster center. The Cluster Center offers an environment for developing, testing, benchmarking and optimizing products free of charge. The center, located in Sunnyvale, California, provides on-site technical support and enables secure sessions onsite or remotely. The Cluster Center provides a unique ability to access the latest clustering technology, sometimes even before it reaches public availability.

In the last few weeks, we have completed the installation of a Windows HPC Server 2008 cluster, and now it is available for testing (via the Vulcan cluster). We have also received the Scyld ClusterWare™ HPC cluster management solution from Penguin Computing (a member company) and installed it on the Osiris cluster.

Scyld was designed to make the deployment and management of Linux clusters as easy as the deployment and management of a single system. A Scyld ClusterWare cluster consists of a master node and compute nodes. The master node is the central point of control for the entire cluster. Compute nodes appear as attached processor and memory resources. More information on Scyld can be found here.

Adding Scyld to Osiris helps the Council with the best practices research activities that provide guidelines to end-users on how to maximize productivity for various applications using 20 and 40Gb/s InfiniBand 20 or 10 Gigabit Ethernet. I would like to thank Matt Jacobs and Joshua Bernstein from Penguin Computing for their donation and support during the Scyld installation.

Best regards,
Gilad Shainer
Chairman of the HPC Advisory Council

Notes from the Oil and Gas High-Performance Computing Workshop

The 2009 Oil and Gas High-Performance Computing Workshop was held on March 5th, 2009, and was hosted by the Ken Kennedy Institute for Information Technology and the Energy and Environmental Systems Institute at Rice University. The goal of the workshop was to discuss industry specific needs and challenges and to engage in a dialog with HPC hardware and software vendors, as well as academic research communities. The focus of this particular workshop was on accelerators and hybrid computing, the future of parallel programming and tools, and storage and I/O solutions and the needs associated with systems deployed in oil and gas HPC environments.


The workshop was organized very well, and it was a great opportunity to meet and talk with many oil and gas users, as well as software and hardware vendors. In spite of the current recession, users still need to stay competitive and to increase their productivity. So new investments are definitely being made, but those investments are being carefully designed to ensure maximum ROI, and must certainly be future proofed.


The HPC Advisory Council presented recent work performed by its members – Gilad Shainer & Tong Liu (Mellanox Technologies), Joshua Mora (AMD), Jacob Liberman (Dell) and Owen Brazell (Schlumberger) on “ECLIPSE: Performance Benchmarking and Profiling”. Schlumberger’s ECLIPSE Reservoir Engineering software is a widely used oil and gas reservoir numerical simulation suite. Maximizing end-user productivity with ECLIPSE requires a deep understanding of how each component impacts the overall solution. Moreover, as new hardware and software comes to market, design decisions are often based on assumptions or projections rather than empirical testing. The presentation was targeted to removes the guesswork from cluster design for ECLIPSE by providing best practices for increased performance and productivity. The presentation included scalability testing, interconnect performance comparisons, job placement strategies, and power efficiency considerations.


The presentation can be downloaded from the HPC Advisory Council web (technical content/conferences section).


Commodity clusters, CPUs and interconnects can together provide the most efficient and productive systems for high-performance applications. The secret is in gathering the right components to create a balanced system, and with the right components and integration, one can maximize the system’s capability and to ensure system and application scaling. For more information in particular to ECLIPSE, please review the presentation.


For more questions and comments, we can be reached (as always) at

Best regards,

Gilad, Chairman of the HPC Advisory Council