Power… it is the only thing that you will find more prevalent in a datacenter than racks, yet many times when discussing upgrades and new installations it’s the part that no one ever mentions. I assume this is for one of a few reasons:
- the IT team isn’t in charge of the power design (leased building, union, or separate electrical department)
- have always just used 120v “normal” stuff under 1800 watts
- aren’t an electric engineer/don’t understand what Amps, Volts, Watts are
- don’t understand all of the options for connectors/cords
So there isn’t much I can do about the first problem, however, if that is the situation you are in I would encourage you to still figure out what you need so that when you go to the people in charge of things you will be able to speak their language. This will most certainly speed up installation time and the need for a possible rip and replace if it’s not correct the first time. So what do you need to know?
The rest of the bullet points we can, however, educate you on.
Volts, Amps, Watts?
A 120v (15 amp) outlet looks like this (duh right?):
This is what you probably have through out your home if you are in the United States, it is by far the most common and what we see almost every day. A standard 120v 15Amp outlet can provide up to 1800 watts of power, after that you will almost certainly trip a breaker. (although Underwriters Lab (UL) says you can only pull 80% of the maximum which is why you won’t find anything over 1500watts on a 15 Amp plug)
Why? Well, there is a simple equation that relates volts, amps, and watts to each other.
Watts = Volts x Amps 1800Watts = 120Volts x 15AmpsIf you are looking at UPS systems you may notice that they are typically rated in “VA” or Volt Amps, and if they have a watt rating on them its typically a little lower than the VA rating.
VA = Volts x Amps 1800VA = 120Volts x 15AmpsSo why do you care? Well, we have to care otherwise we will end up buying the wrong UPS’s. The biggest UPS that you could ever run on this outlet is an 1800VA…. although I’ve only ever heard of 1500VA (or smaller) because again UL says we can only pull 80% of the maximum.
However before we go any farther, we need to talk about something called “Power Factor”, ALL UPS systems will certainly tell you what their power factor is (you will probably need to look in the manual). Basically, this is the efficiency of the UPS.
An update from my UPS guys…
“The difference between the kVA and the kW output of the UPS is determined by the Power Factor (pf) as you point out in the article. The pf is actually the differentiator between real power (kW ) and Apparent Power (kVA ). ( this is where it can get really deep, really fast 🙂 ). It has nothing to do with the UPS efficiency. The efficiency is strictly the determined by how much power you have to put into a UPS to get the rated kW out.
Servers and other equipment have a power factor rating as well. Most systems today are close to unity pf for all practical purposes so the kVA draw is almost the same as the kW draw.
Therefore, the kW is the most important factor when talking about a UPS. Because, in all cases, since you cannot exceed either number in a UPS ( kVA or kW ), you will always run out of kW first if the loads you are powering are close to unity power factor.”
My go to brand is the Liebert GXT3 series UPS’s, they have a power factor of .9 at the time of this writing. (Although I think my GXT2-2000 is only .7 power factor) So to calculate the maximum wattage that a system will support we need to do some more math.
Consumable Watts = Volt-Amps * Power FactorSo for a 1500VA GXT3-1500RT120 we can expect the following:
1350watts = 1500VA * .9PFSo if your server load is going to be more than 1350watts, it will be time to step into a bigger UPS… which will require more Amps…. which will require a different receptacle. This is where the fun starts.
WE NEED MORE POWER!!!
If you need more than 15 Amps we typically have two options, we can either bump up the amperage or the voltage.
The best way that I can explain the difference is to compare the situation to roads. If we need to move more cars than a road can handle we have two options: increase vehicle speed or add more lanes.
Bumping up the Amperage:
If we bump up the amps on a given circuit it is like bumping up the speed limit on a road, there are however some things that we MUST account for if we do this. For example, if we take a 45 Mph road and make it 200 mph we will certainly need to make sure the road is in very good condition… maybe a little ticker pavement, no pot holes, etc. Otherwise, we will certainly have some “accidents”, and electricity is no different. If we bump the amps up we are essentially creating an accident waiting to happen if we don’t properly prepare. This is why you have to run thicker wires for higher amperage circuits… basically, that thicker wire is the nicer road. This is also why thinner wires will get hot and cause fires if you are drawing too much through them.
A great example of this is a hair dryer or bathroom heater, they use cheap cords but pull lots of power, which makes their cords get hot. I guess they assume since you won’t have them on too long its ok, but it’s also why these things cause lots of fires. But 1500watts /120v = 12.5 amps on that single 120v lane, which is tied back to a circuit breaker rated at probably 15 amps. So if you put two hair dryers on the same circuit (rated at 15 amps) you would need 25 amps to run them…. which would in turn trip the breaker.
Bumping up the Voltage:
If we bump the voltage we are essentially bumping our lane count, so if 120v is like one lane, 208/220/240v would be like two lanes. (Note, most datacenter power is 208v but I will be using 240v in my examples just because the math is easier.) So by doing this we can keep the speed at which the vehicles are traveling the same, but still deliver more usable power. BTW Amperage (or Amps), by definition, is the amount of electrical current passing by a point at any given time.
So back to the hair dryer, if you had a 240v hair dryer that still consumed 1500watts, it would still be drawing 12.5 amps total, but only 6.25 Amps per lane (“pole” is the official term). So if we were to put two hair dryers on the same 240v circuit we would still be pulling 25 amps total, but only 12.5 amps on each pole. because a 240v breaker is essentially two breakers of that amperage that means that each pole can support the designated amperage. In this case 15 amps per pole, and because we are only pulling 12.5 amps per pole we would NOT trip the breaker running the same two hair dryers.
So what does this mean in terms of servers?
First let’s look at why the United States uses 110/120volts in the first place, and why higher voltage systems are more efficient.
“When the U.S. rolled out the first electrical grid, light bulb filaments were very fragile and quickly burned outon 208-volt lines. Dropping the voltage to 110/120 volts increased filament life—thus, 120 volts became the
standard in the U.S. By the time Europe and the rest of the world built out their power grids, advances in
filament design had largely eliminated the high-voltage problem, which is why the 230/250 volt power system
is more prevalent across most of the rest of the world.
It’s important to note that each time voltage is stepped down, a transformer is used, and power is lost. The
loss may be as little as 1 or 2 percent per transformer, but over time, the penalty for transformer use adds
up. By switching to a 208-volt system, one less transformer is needed in the chain, thereby reducing wasted
energy.
Moreover, 208-volt systems are safer and more efficient as less current is required to push the same
wattage through 208 volts than 120, lowering the risk of injury and minimizing power losses in transit.”
Source: Eaton’s “Increase Server Efficiencies by using High-Voltage Power Supplies and 208V UPS’s” PDF
And here is my attempt at explaining how 240 delivers more volt-amp while being safer. The basic take away is that a 240v 30 Amp connector is able to deliver 7200VA (theoretical) while it would take a 120v 60 Amp connection to deliver the same 7200VA. Again… two lane road with traffic going 100MPH vs one lane road with traffic doing 200MPH.
So the bottom line is less equipment is needed when you use 240v. If a rack of equipment is drawing 5000watts total you would need at least three or four 120v 15amp UPS’s to handle the load. But with 240v UPS’s you may only need one. That means fewer breakers, less electrical runs to the rack, and fewer installation hours by an electrician.
Bottom line is that if you are upgrading or building a new datacenter… put in higher voltage.
So how do I know if this pile of parts will work together?
This next section is about making sure that you have the right connectors on all of your gear. Let’s face it we are spoiled… have you ever worried about getting anything plugged in in your home? No, everything uses the same plug. Definitely not the case when using higher voltages and different amperages. Trust me there is nothing more frustrating than having skids of new equipment to open and install… and then realizing that your PDU is an L6-20P but your UPS is only L5-30P or L6-30P.
I’ve broken the connection types down into 120v and 208/240v sections.
Higher Amperage (lower voltage) Options
In most datacenters we will see a few different receptacle types of 120volt power. Typically these options are 15, 20, and 30 amps. Normally the plugs will have their rating stamped into them, and if you are unsure, or if you need a different receptacle then you will want to call an electrician as they can also test the wire quality to see what it can safely handle.
120Volt Non-Locking:
We also have “Locking” connectors, the only advantage of these connectors is that they cannot be accidentally unplugged, as they must be twisted in order to be unlocked. I think on 120volt connectors the only one to get a locking plug is the 30 amp. I personally haven’t seen lower amp 120v connections with locking plugs.
120Volt Locking (for 30Amp connection):
120 Limitations:
As we said before, there is only so much you can do with 120volts before you have to crank up the amps to crazy levels. So at some point 120v is just going to run out of steam. In the datacenter most UPS’s wont go above 30 Amps on 120V connections. So our theoretical maximum is 3600VA… but the largest UPS you will see on 120Volt is 3000VA or about 2700 Watts. If your load is higher than that you can either purchase a bunch of 120V UPS’s or switch to the 200+ volt range.
Higher Voltage Option
Almost all datacenter 208-240v connectors (although the connectors are actually rated at 250v) will be locking style connectors. My guess is that they figure if you were serious enough to go 200+ volts, you probably don’t want that shit coming unplugged 🙂 LOL
Just a quick note: Some of these pictures have a “P” or an “R” after the name. this is to indicate whether they are the male or female portion of the connector. So for each connection, you will need an “R” (or female) receptacle and a “P” (or male) plug. The only exception to this rule is the C13/14 and C19/C20 type connections listed below. For those, the C13 and C19 are the female sides and the C14/C20 is the male side.250Volt Locking:
250Volt Non-Locking:
By far the most common is probably the L6-30 for UPS’s. They use this connection type because you can provide up to 6000VA or 5400 Watts of power over it, so it has pretty much become the standard for UPS’s that are 208+volt and rated between 3000 – 6000 watts.
C-Style or “Hooded” Connectors
Up till now, we have only talked about connectors that will get the power out of the wall and into your UPS. There are other connectors that are commonly seen in a datacenter, but these connectors are typically seen on PDU strips or on the equipment itself.
C19 / C20
The C19/C20 style connectors are typically found on blade chassis or other stuff that can pull serious watts, I know that HP and Cisco both use this type of connection on their chassis. So if you are implementing a Cisco UCS 5108 you will need to have a PDU with a C20 receptacle or an open L6-20R:
or
The last connector I wanted to talk about is the C13/C14 style. These cables can be used to deliver both 125 or all the way up to 250volts and are commonly used on servers to connect into a PDU.
C13 / C14
Takeaway:
So if you only picked up one thing from this article it should be to call an electrician. But if you are planning a datacenter remember that there are lots of options, and one of the biggest options is 208-240volt. It is nothing to be afraid of or avoided, in fact, because almost ALL datacenter equipment is dual voltage these days so I would highly recommend moving to 208-240 volts.
Have a question or need more info? Let me know.
[mauticform id=”3″]
“But 1500watts /120v = 12.5 watts on that single 120v lane, which is tied back to a circuit breaker rated at probably 15 amps. So if you put two hair dryers on the same circuit (rated at 15 amps) you would need 25amps to run them…. which would in turn trip the breaker.”
I think you mean 12.5 amps instead of 12.5 watts.
Thanks Gene!
“It’s important to note that each time voltage is stepped down, a transformer is used, and power is lost. The
loss may be as little as 1 or 2 percent per transformer, but over time, the penalty for transformer use adds
up. By switching to a 208-volt system, one less transformer is needed in the chain, thereby reducing wasted
energy.”
Between 240V and 120V there is no “extra transformer”. The difference in transforming is more of how far it needs to be stepped down, i.e.:
13,200V -> 240V
13,200V -> 120V
(That being said, 120V is still a little less efficient because transmission losses, even inside a home electrical system, are higher but they are still slightly safer)
Home electrical installations in the US are actually supplied with 240V. Split-phase power happens to be common in the US because both 120V and 240V are used:
https://en.wikipedia.org/wiki/Split-phase_electric_power#North_America
208V is actually just 3x120V (120 * sqrt 3 = 207.85). There is of course 416V which is simply 3x240V. I’ve never heard of a 3-phase power supply for servers; most likely what’s being used is one leg of the 3-phase circuit, meaning each server/device will receive 120V in the end anyway. 3-phase power is often supplied to industrial areas and buildings, most likely because it’s the best for big motors, but for datacenters I’m not really sure why they need it. Take a look at power lines in industrial areas as well as power lines on power pylons and you’ll notice lots of “threes”.
Thanks for the great information MD! I find myself always learning a little bit the more I did into things, and I certainly was wondering how the heck they got 208v when most things were in 120/240/480v type stuff.
If you dont mind me asking…. why do some people say 110v instead of 120v ? is there really a difference or is it just a misconception? the Kill-A-Watt I put on my home outlets reads 120v (Actually 122v most of the time), yet we plug all sorts of “110v” things in.
Again, thanks for reading and all the great information!
Those situations are simply related to the power being consumed in your house and the grid. Ohm’s law, V = IR, states that, as current (I) increases, the the voltage drop increases. Wires have small amounts of resistances (R) which means that if you increase the current running through the wire, your very small voltage drop can become big. If your house is eating a lot of power the voltage in your house will be slightly lower. You can observe this by turning on a high power device like a microwave, electric dryer, or fridge (motors eat a lot of current to start up) and seeing the lights dim.
All devices have certain voltage tolerance ranges so none of this is a big deal. A typical PSU will tolerate 85-130V.
“Why? Well their is a simple equation”
You need a new editor. That’s embarrassing.
@MD: Two circumstances where you’ll see three-phase in the data center:
1. Blade Chassis – the HP c7000, for exampe, can take 3-phase, or it can take single-phase, depending on which power supply you use.
2. UPS — There are plenty of UPSes out there that will take 3-phase input and emit single-phase output.
Hey Mike! Thanks for the input. I agree, while I have not seen 3 phase into a chassis (mainly because most customers dont have it) I have seen lots of 3 phase “datacenter” UPS’s…. infact the UPS that protects my pile of colo gear is a 3phase liebert.
I agree…. I dont think I have ever pretended to know how to spell or write. I did however catch myself using their instead of there like 10 times yesterday…
Anyhow thanks for the catch.
This article could not have come at a better time. I’m having an eletrician run 2 new 30 amp circuits in my server room next week.
My question – I have a Power Edge SC440 that has the switch in the back (120/240), and all my new UPSs will be 208. Can I run the Power Edge SC440 off 208 even though the switch indicates 240. I’m thinking I can’t.
I agree Josh, 208 is 3 phase while 220/240 is single phase so i think you would certainly cause some damage. Most UPS’s will accept 208 though and can output 240… thats is probably your safest bet.
Great blog. I know this is an old thread but,, We have recently updated power in our datacenter and are still in the process of migrating things. We are a 24x7x365 organization and downtime isn’t really an option. Our new power distribution is 208V. I know the severs will run off of it no problem. I know our EMC VNX E3100 will also. My question is this: can I safely move each power connection for the EMC VNX E3100 from the 120V circuit to the 208V circuit without shutting the entire system down? In other words can one PSU be operating from a 120V circuit and one from a 208V circuit? It would only be this way for a short period of time. Just long enough to move on connection, the PSU come up and stabilize and then move to the next one. It is a small config with the DPE shelf and a single DAE shelf.
Thanks
Jamey
Yup Sure can. They are auto sensing and will recognize that the new power is 208 just fine. My VNX5300 ran for weeks with half on 240v and have on 120v
Awesome!!! Thanks a ton for the reply and info. I have been searching for this answer for a while and have put off moving the VNX because of the uncertainty. Definitely bookmarking the site. Lots of great info.
Thanks again.
Jamey
Glad I could help!
Thanks for the great information… Now, I have to ask… I have recently received some old capital equipment that included an IBM Power5 machine that requires 220v (it is not switchable). Would a residential electrician know how to wire me a single outlet that can power this machine? I have a great opportunity to learn this system if I can get it powered properly in my home. One big question I have, is what type of plug, and does it really matter so long as it’s capable of the voltage/amperage?
Thanks,
R
Hey Rich,
That should be a pretty easy day for an electrician.
So essentially I would tell you to look at the IBM machine and see what it takes. Most datacenter stuff I have bumped into is fairly standard (so chances are you can find the plug at a hardware store or home improvement store).
You will probably also need a new circuit breaker as well. Depending on your panel type you might be talking 10-50$ for the breaker and another 10-40 dollars for the recepticle.
But back to your original question… Any electrician should be able to look at the machine, determine what it needs and then wire it accordingly. Should not be much different than wiring up an air conditioner or a 220v well pump.
Is there any disadvantage going from an APC 3000VA 3Oamp to a 3000VA 20 amp? I only ask because we messed up and didn’t order the high voltage option for the new one.
It will have probably have a different input plug on the cord if it has one… if its hardwired you wont really see a difference.
Pingback: Why I recommend CoSchedule for your blog! | Justin's IT Blog
This may be an older article, but it is still helpful and there doesn’t seem to be many more others like it. Thank you.