Power… it is the only thing that you will find more prevalent in a datacenter than racks, yet many times when discussing upgrades and new installations it’s the part that no one ever mentions. I assume this is for one of a few reasons:
- the IT team isn’t in charge of the power design (leased building, union, or separate electrical department)
- have always just used 120v “normal” stuff under 1800 watts
- aren’t an electric engineer/don’t understand what Amps, Volts, Watts are
- don’t understand all of the options for connectors/cords
So there isn’t much I can do about the first problem, however, if that is the situation you are in I would encourage you to still figure out what you need so that when you go to the people in charge of things you will be able to speak their language. This will most certainly speed up installation time and the need for a possible rip and replace if it’s not correct the first time. So what do you need to know?
The rest of the bullet points we can, however, educate you on.
Volts, Amps, Watts?
A 120v (15 amp) outlet looks like this (duh right?):
This is what you probably have through out your home if you are in the United States, it is by far the most common and what we see almost every day. A standard 120v 15Amp outlet can provide up to 1800 watts of power, after that you will almost certainly trip a breaker. (although Underwriters Lab (UL) says you can only pull 80% of the maximum which is why you won’t find anything over 1500watts on a 15 Amp plug)
Why? Well, there is a simple equation that relates volts, amps, and watts to each other.Watts = Volts x Amps 1800Watts = 120Volts x 15Amps
If you are looking at UPS systems you may notice that they are typically rated in “VA” or Volt Amps, and if they have a watt rating on them its typically a little lower than the VA rating.VA = Volts x Amps 1800VA = 120Volts x 15Amps
So why do you care? Well, we have to care otherwise we will end up buying the wrong UPS’s. The biggest UPS that you could ever run on this outlet is an 1800VA…. although I’ve only ever heard of 1500VA (or smaller) because again UL says we can only pull 80% of the maximum.
However before we go any farther, we need to talk about something called “Power Factor”, ALL UPS systems will certainly tell you what their power factor is (you will probably need to look in the manual). Basically, this is the efficiency of the UPS.
An update from my UPS guys…
“The difference between the kVA and the kW output of the UPS is determined by the Power Factor (pf) as you point out in the article. The pf is actually the differentiator between real power (kW ) and Apparent Power (kVA ). ( this is where it can get really deep, really fast 🙂 ). It has nothing to do with the UPS efficiency. The efficiency is strictly the determined by how much power you have to put into a UPS to get the rated kW out.
Servers and other equipment have a power factor rating as well. Most systems today are close to unity pf for all practical purposes so the kVA draw is almost the same as the kW draw.
Therefore, the kW is the most important factor when talking about a UPS. Because, in all cases, since you cannot exceed either number in a UPS ( kVA or kW ), you will always run out of kW first if the loads you are powering are close to unity power factor.”
My go to brand is the Liebert GXT3 series UPS’s, they have a power factor of .9 at the time of this writing. (Although I think my GXT2-2000 is only .7 power factor) So to calculate the maximum wattage that a system will support we need to do some more math.Consumable Watts = Volt-Amps * Power Factor
So for a 1500VA GXT3-1500RT120 we can expect the following:1350watts = 1500VA * .9PF
So if your server load is going to be more than 1350watts, it will be time to step into a bigger UPS… which will require more Amps…. which will require a different receptacle. This is where the fun starts.
WE NEED MORE POWER!!!
If you need more than 15 Amps we typically have two options, we can either bump up the amperage or the voltage.
The best way that I can explain the difference is to compare the situation to roads. If we need to move more cars than a road can handle we have two options: increase vehicle speed or add more lanes.
Bumping up the Amperage:
If we bump up the amps on a given circuit it is like bumping up the speed limit on a road, there are however some things that we MUST account for if we do this. For example, if we take a 45 Mph road and make it 200 mph we will certainly need to make sure the road is in very good condition… maybe a little ticker pavement, no pot holes, etc. Otherwise, we will certainly have some “accidents”, and electricity is no different. If we bump the amps up we are essentially creating an accident waiting to happen if we don’t properly prepare. This is why you have to run thicker wires for higher amperage circuits… basically, that thicker wire is the nicer road. This is also why thinner wires will get hot and cause fires if you are drawing too much through them.
A great example of this is a hair dryer or bathroom heater, they use cheap cords but pull lots of power, which makes their cords get hot. I guess they assume since you won’t have them on too long its ok, but it’s also why these things cause lots of fires. But 1500watts /120v = 12.5 amps on that single 120v lane, which is tied back to a circuit breaker rated at probably 15 amps. So if you put two hair dryers on the same circuit (rated at 15 amps) you would need 25 amps to run them…. which would in turn trip the breaker.
Bumping up the Voltage:
If we bump the voltage we are essentially bumping our lane count, so if 120v is like one lane, 208/220/240v would be like two lanes. (Note, most datacenter power is 208v but I will be using 240v in my examples just because the math is easier.) So by doing this we can keep the speed at which the vehicles are traveling the same, but still deliver more usable power. BTW Amperage (or Amps), by definition, is the amount of electrical current passing by a point at any given time.
So back to the hair dryer, if you had a 240v hair dryer that still consumed 1500watts, it would still be drawing 12.5 amps total, but only 6.25 Amps per lane (“pole” is the official term). So if we were to put two hair dryers on the same 240v circuit we would still be pulling 25 amps total, but only 12.5 amps on each pole. because a 240v breaker is essentially two breakers of that amperage that means that each pole can support the designated amperage. In this case 15 amps per pole, and because we are only pulling 12.5 amps per pole we would NOT trip the breaker running the same two hair dryers.
So what does this mean in terms of servers?
First let’s look at why the United States uses 110/120volts in the first place, and why higher voltage systems are more efficient.“When the U.S. rolled out the first electrical grid, light bulb filaments were very fragile and quickly burned out
on 208-volt lines. Dropping the voltage to 110/120 volts increased filament life—thus, 120 volts became the
standard in the U.S. By the time Europe and the rest of the world built out their power grids, advances in
filament design had largely eliminated the high-voltage problem, which is why the 230/250 volt power system
is more prevalent across most of the rest of the world.
It’s important to note that each time voltage is stepped down, a transformer is used, and power is lost. The
loss may be as little as 1 or 2 percent per transformer, but over time, the penalty for transformer use adds
up. By switching to a 208-volt system, one less transformer is needed in the chain, thereby reducing wasted
Moreover, 208-volt systems are safer and more efficient as less current is required to push the same
wattage through 208 volts than 120, lowering the risk of injury and minimizing power losses in transit.”
Source: Eaton’s “Increase Server Efficiencies by using High-Voltage Power Supplies and 208V UPS’s” PDF
And here is my attempt at explaining how 240 delivers more volt-amp while being safer. The basic take away is that a 240v 30 Amp connector is able to deliver 7200VA (theoretical) while it would take a 120v 60 Amp connection to deliver the same 7200VA. Again… two lane road with traffic going 100MPH vs one lane road with traffic doing 200MPH.
So the bottom line is less equipment is needed when you use 240v. If a rack of equipment is drawing 5000watts total you would need at least three or four 120v 15amp UPS’s to handle the load. But with 240v UPS’s you may only need one. That means fewer breakers, less electrical runs to the rack, and fewer installation hours by an electrician.
Bottom line is that if you are upgrading or building a new datacenter… put in higher voltage.
So how do I know if this pile of parts will work together?
This next section is about making sure that you have the right connectors on all of your gear. Let’s face it we are spoiled… have you ever worried about getting anything plugged in in your home? No, everything uses the same plug. Definitely not the case when using higher voltages and different amperages. Trust me there is nothing more frustrating than having skids of new equipment to open and install… and then realizing that your PDU is an L6-20P but your UPS is only L5-30P or L6-30P.
I’ve broken the connection types down into 120v and 208/240v sections.
Higher Amperage (lower voltage) Options
In most datacenters we will see a few different receptacle types of 120volt power. Typically these options are 15, 20, and 30 amps. Normally the plugs will have their rating stamped into them, and if you are unsure, or if you need a different receptacle then you will want to call an electrician as they can also test the wire quality to see what it can safely handle.
We also have “Locking” connectors, the only advantage of these connectors is that they cannot be accidentally unplugged, as they must be twisted in order to be unlocked. I think on 120volt connectors the only one to get a locking plug is the 30 amp. I personally haven’t seen lower amp 120v connections with locking plugs.
120Volt Locking (for 30Amp connection):
As we said before, there is only so much you can do with 120volts before you have to crank up the amps to crazy levels. So at some point 120v is just going to run out of steam. In the datacenter most UPS’s wont go above 30 Amps on 120V connections. So our theoretical maximum is 3600VA… but the largest UPS you will see on 120Volt is 3000VA or about 2700 Watts. If your load is higher than that you can either purchase a bunch of 120V UPS’s or switch to the 200+ volt range.
Higher Voltage Option
Almost all datacenter 208-240v connectors (although the connectors are actually rated at 250v) will be locking style connectors. My guess is that they figure if you were serious enough to go 200+ volts, you probably don’t want that shit coming unplugged 🙂 LOLJust a quick note: Some of these pictures have a “P” or an “R” after the name. this is to indicate whether they are the male or female portion of the connector. So for each connection, you will need an “R” (or female) receptacle and a “P” (or male) plug. The only exception to this rule is the C13/14 and C19/C20 type connections listed below. For those, the C13 and C19 are the female sides and the C14/C20 is the male side.
By far the most common is probably the L6-30 for UPS’s. They use this connection type because you can provide up to 6000VA or 5400 Watts of power over it, so it has pretty much become the standard for UPS’s that are 208+volt and rated between 3000 – 6000 watts.
C-Style or “Hooded” Connectors
Up till now, we have only talked about connectors that will get the power out of the wall and into your UPS. There are other connectors that are commonly seen in a datacenter, but these connectors are typically seen on PDU strips or on the equipment itself.
C19 / C20
The C19/C20 style connectors are typically found on blade chassis or other stuff that can pull serious watts, I know that HP and Cisco both use this type of connection on their chassis. So if you are implementing a Cisco UCS 5108 you will need to have a PDU with a C20 receptacle or an open L6-20R:
The last connector I wanted to talk about is the C13/C14 style. These cables can be used to deliver both 125 or all the way up to 250volts and are commonly used on servers to connect into a PDU.
C13 / C14
So if you only picked up one thing from this article it should be to call an electrician. But if you are planning a datacenter remember that there are lots of options, and one of the biggest options is 208-240volt. It is nothing to be afraid of or avoided, in fact, because almost ALL datacenter equipment is dual voltage these days so I would highly recommend moving to 208-240 volts.
Have a question or need more info? Let me know.