Few things are as important to the running of data centers as power. Without power, no data center could support its customers’ systems, and businesses would quickly grind to a halt. So power should of course be a key consideration when planning and organizing your systems. Of course, getting to grips with what you need to know about power design and infrastructure is no mean feat, particularly if you’re looking at more complex setups. But we’re here to help you through the process, and our experts are always on hand to answer any questions you might have.
Whether you’re thinking about moving data centers, or you’d simply like to learn more about how data center power design and infrastructure works, take a look at our brief guide to the subject. Here’s everything you need to know, in a nutshell!
Data center power: What is the typical infrastructure set-up?
In the vast majority of cases, data centers will use the municipal electric grid as the main source of their power. The electric grid will be responsible for supplying energy, which can then be transformed on-site to ensure that it is of the optimal voltage and current for the data center’s requirements.
Some data centers look to other energy sources in addition to, or instead of, the electric grid. Data centers typically have their own generators, which can be used in case of an emergency. Sometimes these generators will also supplement the power supply in the data center.
Power is not delivered ready to use. It must then be taken to Main Distribution Boards, before being transported to a series of different endpoints, including Uninterrupted Power Supply (UPS) systems and load banks. This practice helps to avoid damaging complex systems through unexpected power surges and ensures that servers get the right amount of power for their requirements.
There’s an element of security involved in this process, too. UPS systems can ensure that power continues to flow even if an outage or another problem is causing an interruption in the power supply. This means that short interruptions in power won’t affect the systems reliant on the data center’s power supply, and businesses won’t feel an impact from these. UPS systems tend to provide a temporary solution, which gives the data center enough time to switch to its backup generator power and keep dependent systems running.
Data center power and energy requirements
In order to keep a continuous power supply with minimal interruptions, data centers require a significant amount of energy. It’s not just the IT equipment that needs power in data centers, either. Facilities must also be kept at the optimal temperature for the systems housed within them, and other equipment such as humidifiers and monitors are all essential to the daily running of data centers.
The exact energy consumption of a data center depends on variables – the square footage of the facility, the power supply for each server, the facility voltage (usually 120 V or 240 V), the number of server racks and the number of servers per rack. Once this is determined, it’s easier to figure out if a facility is running as efficiently as it could be.
Power Usage Effectiveness
Data centers are coming under increasing pressure to ensure that they are as energy efficient as possible, particularly as their energy requirements are expected to increase considerably in the near future. Currently, this is being measured through a metric known as Power Usage Efficiency (PUE).
PUE is calculated to explain how effective a data center is, based on the energy it consumes and how much of that energy is dedicated to its servers. A disproportionate amount of energy being used by equipment other than servers brings a data center’s score down, while a dedication to servers only ensures a higher score. A recent survey from the Uptime Institute found that on average, data centers have a score of 1.58. However, this score has been declining over time.
Consolidation efforts
From 2010, data center energy usage grew by far less than it had in the previous decade. The reasons? Increased hardware efficiency, better management and consolidation effort. Even though data center workloads have increased, energy consumption has increased relatively slowly in comparison due to advances in efficient technology and the adoption of more effective data center management strategies. Businesses are looking for ways to consolidate their infrastructure and reduce consumption by abandoning private data centers and moving away from the power-hungry legacy hardware within them in favor of colocation services.
Best Practice For Data Center Power Design
Isolating Redundant And Non-Redundant Loads
Oftentimes these two are mixed within a single cabinet. However, if you have equipment with a single power cord, steps should be taken to keep them separated and instead provide a static transfer switch at the point of use.
Backup Generators
A key point to remember about your backup generation system is that the standby rating is not necessarily what you should be designing towards. You should aim to have something closer to the prime rating instead.
Each generator provider will have its own set of standards, the MTU calls it the DCC (Data Center Continuous) standard, denoting if a facility can run for an unlimited number of hours for the duration of any outage at the rated amount of power. Diesel generators are the state of the art generators that are still widely used today (including by TRG) due to the fast response and the ability to control the fuel source.
Fuel sources should be isolated from each other. You should also be polishing your fuel, ensuring the diesel being used is healthy. This is to make sure your generators will actually run. When it comes to generators, you should make sure you’re testing regularly and changing the batteries approximately every two years. This will all help to contribute towards a high DCC score, ensuring generators will be able to ‘go the distance’.
Breaker Coordination and Ground Fault Protection
Breakers and panels now have evolved technology that enables enhanced modeling capabilities on flawed breakers. You can now also get ground fault protection all the way down to the branch T breaker level.
Lightning Protection
Utility protection shouldn’t be overlooked – particularly things like lightning protection. Any utilities going in or coming out of the building should have a surge protection device on the panels. It’s also best practice to ensure lightning protection around the entire building.
Uninterruptible Power Supplies
What needs UPS and what doesn’t? This will oftentimes depend on the density being run. TRG uses UPS for the control systems on our crack units which eradicates the reboot time after a transfer to a generator. We find this is an often overlooked factor but one which can result in up to a minute added to reboot times. At a certain density, however, it will pay off in the long run if the entire system is on UPS
Level of Redundancy
A big design consideration is determining what level of redundancy and reliability you want for your facility. We have more information in our guide here, but this will play a significant role in determining the power and infrastructure layouts.
Batteries
Battery selection is something that gets misconstrued within the industry. The IEEE has standards which lay out that a battery at the end of its life should be able to support a load for a stated period of time (generally 5-8 minutes). However, you should also consider the performance of the battery and its ability to deliver that power instantaneously. This is why TRG uses Pure Lead products which utilize refined lead as opposed to recycled lead, allowing for a higher battery performance.
Invest Money in the Right Areas
One of the biggest takeaways should be this; be willing to spend more in the right areas. Cutting corners on core components can, and often will, lead to more headaches and higher price tags later down the road. In our experience, some of the areas where it’s worth spending more are: buying high quality batteries, getting generators with a good cylinder count, investing in protection and coordination for breakers and choosing (and implementing) the right level of redundancy.
Consider Using Experts
The design and infrastructure of data center power is a complex field – but one that is continuing to be optimized by data center providers. As mentioned before when discussing the build vs buy debate, it is often more economical to utilize a colocation data center (and the expert support that comes with one) especially when running below a 2-3 megawatt capacity.
Using a colocation service like TRG will remove the obstacles and stresses involved with the above, allowing you to reap the benefits of a purpose-built facility that has been developed with the most up-to-date understandings of data center power design and infrastructure.
—
If you’re interested in hearing more about how our power supply and infrastructure works, don’t hesitate to contact our team.
