So I was doing some BOM’s for a customer project today and thought I’d share more information on what UCS Mini bundles are out there right now. As well as what your options are to add to those bundles.
Starter Bundles
Right now Cisco has three “Smart Play” bundles to get started with the new UCS Mini, they include a “Value Plus”, “Entry Plus”, and Entry” bundle. Basically if you want the cheapest option you would want to go with the Entry bundle, and if you wanted to go all out go with the Value Plus.
Chassis/Fabric Interconnect Components
All bundles come with the same chassis, the same 6324 FI’s, and the same SFP modules for those FI’s. The chassis sub-bundle includes the following components:
- 4 – Power Cords (for US NEMA they send L6-20 twist lock cords)
- 2 – 6324 Fabric Interconnects
- 4 – GLC-T 1Gbps UTP SFP modules (2 per FI)
- 4- 10Gbps SR SFP modules (2 per FI)
- 1 – UCS Central Domain License (a base license of UCS Central is still required if you want to use it)
- 4 – 208-240v Power Supplies
- 8 – UCS 5108 Fan Modules
The biggest thing to note here is that there are NO Fiber Channel SFP modules that come with the bundle by default. For Fiber Channel support you will need to add DS-SFP-FC8G-SW=, these are the Cisco Fiber Channel SFP modules. You will want 4 per chassis (2 per FI) for a proper setup. If you are going to use Ethernet based storage (iSCSI, NFS, FCoE) you may want to ask your partner to provide 4 additional 10GB SR modules so that you can either link directly to your storage (using the FI appliance port mode) or simply to run 4-10Gbps links to your upstream switch.
The Blades
The difference between the bundles is in the blades they contain. Each bundle has the same base B200 M3 blades, however they have different CPU and Memory combinations. I put together a quick chart to show the differences.
The configs are pretty straight forward, basically you decide with CPU is going to provide the proper number of cores and from there you can upgrade the blades to meet your exact needs.
Personally I think that Cisco has made one mistake with their Entry Plus bundle; and that is the bundle of blades it has in it. A lot of customers that I can see being interested in the UCS Mini may also be looking at just going with VMware Essentials Plus, which has a 3 physical server limit. so customers may wonder what they are going to do with that extra blade. So if it were me… I would revise the Entry Plus bundle to include 3 blades, and let customers decide if they want to add more or not.
With that being said, lets take a look at what you need to know to order additional blades for your new UCS Mini.
Additional Blades
Adding additional blades is pretty easy… you just need one part number.
Entry Expansion Blade: UCS-EZ7-B200-E (list price $8,231)
Entry Plus Expansion Blade: UCS-EZ7-B200-EP (list price $8,400)
Value Plus Expansion Blade: UCS-EZ7-B200-VP (list price $15,600)
Other Expansion Options
Here are some other things you might want to add to your blades.
Memory Upgrades:
4 x 8GB Dimm Bundle (32GB total): UCS-EZ7-M8GB-4
4 x 16GB Dimm Bundle (64GB total): UCS-EZ7-M16GB-4
Hard Drives:
2 x 300GB 10k Bundle: UCS-EZ7-300G10K-2
2 x 600GB 10k Bundle: UCS-EZ7-600GB-2
One last note
As Tony mentioned int he comments below my thinking was incorrect because of how the internal traces are routed inside of the chassis. For a great explanation of why you WOULD want the VIC1240 expansion card please check out this blog post http://ucsguru.com/2013/03/13/20gb-20gb-10gb-m3-blade-io-explained/
Here is the summary of that post in relation to the UCS Mini though.
Because the 6324’s act much like 2204 IO Modules (based on how traces are connected from the Mez slot and VIC1240 LOM slot), you will only have 10Gbps of converged bandwidth PER FABRIC to each of the blades in a UCS Mini. So that means if you are using 8Gbps Fiber Channel and are assigning 2x10Gbps virtual nics to VMware (1 vHBA and 1x10Gbps Ethernet per fabric) you are effectively over committing by almost 2:1. So if you intend to have a lot of storage and or ethernet traffic you will certainly want to add the VIC1240 expansion card to your blades.
I’m not sure why I was looking for it, but I was trying to see if the 6324’s support the VIC1240 Expansion module. Basically this module goes into a blade that has a VIC1240 (which can provide 2 x 10Gbps lanes to each FI) and it upgrades it to provide an additional 2 x 10Gbps lanes to each FI…. at least in a normal UCS FI environment.
What I noticed is that the 6324 only has 16 x 10Gbps lanes to the blades (listed as “Server ports” in the below table), this means that no matter if you have 1 blade or 8 blades… they have to share 16 lanes. So if you plan to have more than 4 blades you will not be able to use the VIC1240 expansion module because the FI’s will run out of Server ports.
To help explain this here is what the difference looks like when you are using normal UCS with FI’s and fabric extenders.
And here is what it typically looks like with the VIC1240 expansion module.
So as you can see if you were to populate all 8 slots and have the expansion modules on each blade you would need a total of 64 server ports on your FI’s (32 per side)…. yet the 6324 only supports 16.
NOW before you rule out the UCS Mini…. let me add that I personally have not ran into an issue with not having the expansion module yet…. you will still have 40Gbps into each blade with is enough for 10Gbps of storage traffic per fabric and 10Gbps of Ethernet traffic per side, but still something to keep in mind if you are running something crazy (which would normally mean I would just use 6248 FI’s and a traditional UCS config).
If you get stuck or just want some design help drop me an email to [email protected] or send me a message on twitter.
Justin
you can use the VIC 1240 Expansion mezzanine and 8 blades with the 6324 Fabric Interconnect. The 6324 is connected in the same way as the 2204XP so rather than getting 4 10Gbit lanes to each blade from the 6324 you 2x 10Gbit lanes. One goes to the MLOM and one goes to the mezz slot so max bandwidth from each blade is 20Gbit to each Fabric Interconnect or 40GBit total.
With 16x 10Gbit ports on the 6324 for servers this allows 2x 10Gbit per blade with 8 blades using the VIC 1240 MLOM and expander.
Colin Lynch @ucsguru has a good explanation of the way the 2204XP/2208XP work here http://ucsguru.com/2013/03/13/20gb-20gb-10gb-m3-blade-io-explained/
Hey Tony, Thanks for the comment.
One question though, according to the documentation you can have 8x 10Gbps links to the 2208XP’s if you use the Expansion card….. that is not the case with the 6324 though from what i can see. like you are saying, you only get two links to each 6324 (reguardless of if they are coming from VIC1240 card or mez card… its still only 4 total) where as the traditional IOModules and FI’s give you 8 per blade…. am i still thinking incorrectly ?Tony, I checked out the post and see what you mean. I will revise my post! Thank you very much for the heads up 🙂
Justin,
When using SKU “UCS-SP8-M-B200-E” for the UCS Mini Entry Bundle, I’m only getting qty 1 of the the FI for my BOM. Does the bundle indeed come with 2 FI’s? That’s what I’ve been told.
UCS-SP-FI-M-6324 1 UCS SP 6324 In-Chassis FI
Thanks,
Matt
Hey matt, I just checked on Commerce workspace and it has the two FI’s as different line items. Here is an image of what the left side bar looks like when you select options for the 5108 chassis…. bottom line CCW wont let you validate a config without two 6324’s and all the other minimum components to make things work.
I am new to UCS but have 2 UCS 5108 and I see that they have the 6324 Fabric Inter-connectors on the chassis. I tried getting some SFP modules so that I connect the UCS servers directly to my C6809-XL switch via fibre but the links show that they are down. I would like to ask how I can get the UCS connected to my ethernet network, do I need to purchase nexus devices?
Paddington,
You should be able to use an SFP module as long as it is supported by Cisco. Check the logs to see if there are errors that the system has disabled the port because the SFP is unknown. If you are using a Cisco SFP, and the port is showing as up, then I would refer you to the UCS setup manual, as there are a lot of things that need to be configured before a blade will pass traffic on to your LAN fabric.
You will need to configure a service profile, a VLAN, Uplinks to LAN switches, then map things together in the service profile. So read through the setup guide and you should be all set.