UCS, Clouds and HPC

How many buzz terms can I squeeze into a blog title? I wanted to add a few more, like ARRA but enough is enough :)

Cisco's positioning of UCS is as a complete data center infrastructure. They say that clouds are driving data centers towards UCS. However, it's not really a complete data center infrastructure because it leaves out most of the facility components. That means it is designed to be dropped in place into existing data centers. I don't think that's where clouds are going, in fact, I think they are going to the companies that do the most vertical integration all the way to the power generation facility.

So, Cisco is trying to sell a radically new IT architecture into established IT shops (those that have data centers fully or partially populated now and are looking for incremental expansion or replacement).

Given the pre-recession resource realities (oil at $150/barrel and heading higher) the incremental improvements in operating expense might have been enough to sway these IT shops. But that world is now gone, at least for awhile, it will come back.

So the cloud vendors might use UCS but probably few will because if you want to survive in the cloud market you need to innovate from top to bottom (think power generation, think building construction) and drive costs as low as they can go.

Clouds and HPC have been the subject of a certain amount of academic debate. Most of the naysayers are those that want to have the last ounce of performance, whatever the cost. As you can tell from my previous paragraph that's not in the clouds...(pun intended). So what is in the clouds is that researchers need to learn how to effectively use cloud resources. In one sense, they are already doing that with grids, like the Teragrid. But in the large scale roll out world, I continually run into researchers and small research teams who follow the 'build it yourself' model of HPC and it's close relative - higher an integrator to build it for you. This is the 100 to 1000 'core' market and while some of it needs the last ounce of performance, the fact is that many of these users can't extract the maximum performance from what they have in the first place. Parallel programing for HPC is just too hard or too obscure. And here is where both the solution and the problem comes for cloud vendors. Showing these 1000's of small research teams how to effectively use cloud resources. How to permanently move large data sets into the cloud infrastructure and thus avoid the nasty performance issues (not to mention billing issues) of multi-Terabyte data set access. How to use functional programming to solve their algorithmic problems, such as MapReduce.

If this sounds more like consulting, then you are right, it is more like consulting and less like buying generic off-shelf X86 servers. And that is also a sign of the issues here, because the paradigm of buying generic, off the shelf, PC inspired X86 servers is well cemented into the bulk of the research community that has not yet moved to HPC.

So far, my experience in trying to set up an HPC support team for Bio-Informatics that is software focused has been met with apathy. Nearly everyone want's to talk hardware and which processor are you using? If I say it really doesn't matter as long as you can get the job done in an acceptable time frame at acceptable costs, well they turn away and go back to browsing the hardware vendor web sites. You just gotta have the latest processor, not using Nehalem yet? well get with it, it's so much better.

I do think these things will change and here is why. Scientific research is a competitve arena as much as building and selling products or services is. If cloud computing can deliver the productivity gains that I believe it can for research computing, then those who adopt cloud computing will soon be out-competing those that don't (of course this assumes that at least some of the world renowned researchers who get big grants sign on to this). It also means, that just like in IT, a small startup (think smaller school here) that is funded well enough to land a top scientist, will probably come to the clouds first. That is because they are not hobbled with existing infrastructure, either in data centers or older HPC systems.

The big guys (think large academic research univerisites) will also 'get it' but claim to not see the demand for cloud computing. I suppose that in the more fully de-centralized universities, less funded researchers might see the opportunity and use it to do research that at one time could have only been done at a few high end places......

Hope springs eternal.