This past week was exciting for me as I finally got to unbox our first Cisco UCS gear! We are doing a fairly major revamp of datacenters, which includes:
- Replacing our server infrastructure with Cisco UCS Blades
- Replacing our existing EqualLogic storage with NetApp (and trading iSCSI as our primary storage protocol with FCoE)
- Replacing our Catalyst cores with Nexus switches
I’m going to try and cover all the configuration steps in upcoming posts. I’ve already discovered a few things that are needed that aren’t in the manuals yet! In this post I’m just going to cover initial setup and first impressions.
The Parts
The setup discussed below is for a secondary location that happened to be our “test” of UCS as it’s currently not a critical site. Our setup here consists of:
Network:
- 2 x Cisco Nexus 5548UP 10GbE Switches (32 ports each, Layer-3 Daughtercards added)
- 1 x Cisco Nexus 2224TP Fabric Extender (24 1GbE Ports)
Servers:
- 2 x Cisco 6120XP Fabric Interconnects (20 ports, plus an addon module that provides 4 more 10GbE ports and 4 4Gb Fibre Channel ports)
- 1 x Cisco UCS 5108 Blade Chassis (8 slots, with 2 Cisco 2104 Fabric Extenders)
- 6 x Cisco B200 M2 Blade servers (64GB RAM each, no local storage, Cisco Palo VIC mezzanine cards)
Storage:
- 1 x NetApp FAS3210 Filer (dual controllers, with 2-port 10GbE Unified Target Adapters)
- 1 x NetApp DS4243 Disk Shelf (24 x 1TB SATA Disks)
All cabling will be 1meter and 3meter TwinAx except for 4 “true” fiber links between the Fabric Interconnects and the Nexus switches (as we cannot yet do multi-hop FCoE).
Out of Box Experience
The hardware installation of all components was fairly straightforward with no real surprises. Racked everything up one day and started connecting things on day 2. First major roadblock was in setting up the 24-port Fabric Interconnect. It wouldn’t work, never getting past a blinking amber status light no matter what we did. My gut told me it was defective, so we contacted Cisco TAC and spent a few hours on the phone while they verified. Looking back, it shouldn’t have taken as long as it did. The Fabric Extenders are basically remote line cards and aren’t very complicated to set up; it shouldn’t have taken TAC so long to RMA the unit (we had to convince them!). We got a replacement the next day and it worked right away.
The NetApp setup was a bit more confusing. As someone new to NetApp systems in general, I found the initial cabling really confusing. I’m also not sure why all the supplied cables were so long…with a simple 1 head / 1 disk shelf setup racked right on top of each other, the area behind them looks terrible. NetApp, you might want to consider offering 1-foot cables as an option. All in, there are 19 cables involved for this little two-box setup. With most of the supplied cables being 6 and 10 feet, you can imagine how it looks.
The actual UCS hardware was a breeze. We did have 1 dead blade out of 6 however, but I’ll wait until the UCS software side is up and running before trying to RMA it (as I suspect they will want to do some diags…the issue is the power for that blade just doesn’t come on).
Because of the slowdown with the FEX, I didn’t get to do much in the way of configuration while onsite. But my primary goal was to get things remotely manageable to continue setup from a comfy chair and not on the datacenter floor.
What’s Next
Next steps will be configuring the UCS system. Stay tuned!