First, on the "my horse for a drink of water" front, our installation of close to a half million in hardware has been held up for lack of a 1 meter long fiber cable. Now have plenty of fiber cables, LC to LC and SC to SC, but what we don't have is SC to LC and that is what we need. The vendor was supposed to ship us one, but they don't have them either!
If you have not guessed, LC and SC are connectors! SC are bigger than LC and seem to be prevalent in switching gear, at least what we get from Cisco and Procurve. LC are prevalent at the server side of things, such as 10GbE cards or Fiber Channel cards. Yes, you guessed it, the 'enterprise' switch vendors are in an enterprise that consists of switching gear and no servers. How long have servers been needing switches to connect them to the network? As another example of this, just recently ProCurve announced a new switch line designed for the machine room, which mainly means it has front to back cooling, with the front defined as where all the connectors are. Up until now, both CISCO and Procurve switches had either side to back or back to front cooling while all servers have front to back cooling. Try mixing that up in your racks for some turbulent air flow. Of course, this is all a matter of perspective since while the 'front' of switches is the front of the rack for most installations, the 'front' of the server is bereft of any connectors aside from the occasional USB port. Servers put their connectors in the 'back'
So, as a guy who installs servers, I really don't care which perspective is right or 'better', just that everyone agree's!
As a footnote, not all server vendors put their connectors on the back, a rather small vendor (Capricorn) which supplies the Internet Archive with it's servers, put's all the connectors on the front. Seems like they must have visited a machine room......
Now, for something completely different......
The second thing is, you may remember our NFS problems? Look at past postings for more information. You may also remember our 10GbE issues (hardware, drivers) as well. We have now demonstrated on the 1/2 of 10 identical servers which can't transfer much NFS data, that they can't transfer much of any kind of data, FTP, SFTP, etc.....They can, however transfer data just fine over Infiniband and link aggregated 1 GbE links!
So we are certain that this problem is in the 10GbE path. Whether it's software, firmware, or chipset issues, we don't know yet.
Hey, didn't you say these servers were identical? Yes, they all came in on the same shipment, their BIOS and Firmware all seems to be the same and it doesn't matter which OS we load, official Solaris, OpenSolaris, NexentaStor the one's that have a problem have it with all three OS versions.
But, there must be some difference somewhere, more sleuthing is required. By the way, when these kinds of things happen, don't believe the vendors that they are ready to support you, you are on your own.
I know the enterprise switch vendors are trying to tell us that 10GbE is the server room fabric of the future, but personally, Infiniband is far more robust and even has better performance to boot! It's also cheaper! But eventually we need to get to Ethernet to get on the Internet, thanks CISCO and Procurve for ignoring Infiniband and leaving us small fry with endless grief.
No comments:
Post a Comment