Modelling and Analysis of a Datacenter

Objective: The objective of this challenge is to model a replica of the datacentre where high power distribution units are working. The main parts including are computer room air conditioning units (CRACs), server cabinets, power distribution units (PDUs), and perforated floor tiles in the datacenter. Analyze the flow distribution and behavior of the temperature of server stacks inside the datacentre.

Problem Description:

This tutorial considers a 1200 sq. ft. datacenter with a slab to slab height of 12 ft. The datacenter consists of a 1.5 ft underfloor plenum and a 2 ft ceiling plenum. The CRACs discharge cold air into the underfloor plenum. The cold air enters the main datacenter space mainly through the perforated floor tiles and returns back to the air conditioning units. The cooling load, as summarized below. Size and Capacity of Heat Sources in Datacenter corresponds to the heat output from the server cabinets and the PDUs. A PCB board, library files, and traces are imported to create the model. The model is first solved for conduction only, without the components, and then solved using the actual components with forced convection.

Heat Source	Size	Power
Server Cabinet	2 ft x 3 ft x 7 ft	3000 W
High Density	2 ft x 3 ft x 7 ft	7000 W
PDU	4 ft x 2 ft x 5 ft	3600 W

Introduction:

Data centers are simply centralized locations where computing and networking equipment is concentrated for the purpose of collecting, storing, processing, distributing or allowing access to large amounts of data. They have existed in one form or another since the advent of computers.

Because of their high concentrations of servers, often stacked in racks that are placed in rows, data centers are sometimes referred to as a server farm. They provide important services such as data storage, backup and recovery, data management, and networking. These centers can store and serve up Web sites, run e-mail and instant messaging (IM) services, provide cloud storage and applications, enable e-commerce transactions, power online gaming communities and do a host of other things that require the wholesale crunching of zeroes and ones.

The sole purpose of data center cooling technology is to maintain environmental conditions suitable for information technology equipment (ITE) operation. Achieving this goal requires removing the heat produced by the ITE and transferring that heat to some heat sink.

Geometry:

• Used the inbuilt option called macros to model the components in the datacenter.
• Ansys Icepak includes many macros designed to automate common model-building tasks. Macros are available to assist with creating different types of packages, PCBs, heat sinks, heat pipes, thermoelectric coolers, JEDEC test configurations, and DELPHI package characterization.
• Hereby using the macros, we have modeled the computer room air conditioning units (CRACs), server cabinets, power distribution units (PDUs), and perforated floor tiles.

• The above figure shows what are the parts we have modeled and where we positioned them. The picture right below it says the direction of the flow of air (hot & cold).

Mesh:

• The maximum element size taken is 1 * 0.5 * 1 inch. The minimum element size taken is 1 * 0.15 * 1 inch.
• A Mesher-HD mesh type is used to generate the mesh.
• The number of elements generated is 135918 and the number of nodes generated is 163728.
• Normal mesh parameters are used to generate the mesh.
• Object parameters are applied to the geometry to generate the mesh.

• Here the objects colored with red color are given with per-object parameters. This per-object parameter concept is used to generate a fine mesh without any overlaps of the elements.
• The object parameters will help us to generate more amount of elements and nodes so that we will capture physics accurately at some particular places like PDU's, CRAC's, tiles, etc.

Numerical setting:

Basic parameters:

• In the basic parameters where we have given whether the solution runs for temperature, flow, and setting up for the initial temperature as well as radiation temperature, etc.

• Here we are solving both the flow and temperature variables for the modeled geometry.
• We are neglecting the radiation effect because we are interested in solving major heat transfer spots.
• The turbulent model we are calculating is Zero-equation.
• A turbulent flow regime has to be calculated with the gravity vector of -9.81 m/s in Y-direction in the natural convection space.
• Internal convection usually cannot be treated using simple boundary layer theory because the entire fluid in the enclosure engages to the convection.
• When heat is added to a fluid and the fluid density varies with temperature, a flow can be induced due to the force of gravity acting on the density variations. Such buoyancy-driven flows are termed natural-convection (or mixed-convection) flows and can be modeled by ANSYS FLUENT.

• The temperature conditions maintained inside are ambient in nature, which is 20 degrees centigrade.
• The solid materials used in the model are Mica-Typical.In the electrical industry the same as thermal insulation, and electrical insulators in electronic equipment.
• The surface material used is Paint-non metallic.

• Here we have enabled the species section and provided the H2O gas as a species inside the model of the datacenter.

Basic Settings:

• Here in the basic settings panel, we need to give the number of iterations to 1000.
• The convergence criteria of Flow is set to 0.0001 and energy is set to 1e-6, turbulent kinetic energy to 0.001, and joule heating is set to 1e-7.

Parallel settings:

• Here I have toggled the parallel option and given the four processors because my PC supports till this point. We can increase or decrease the number of processors as per they are situated in the PC.

Advanced settings:

• Here in this panel, we give the inputs as the discretization schemes for the pressure, momentum, and temperature are of the first order.
• The double precision is given. Single precision means that the floating-point numbers will be represented in a 32-bit system whereas double-precision means that they will be represented in a 64-bit system. So Calculation in double precision will be more accurate.

• We have adjusted the under relaxation factors of momentum and body force for the convergence criteria.

Residual plot:

• From the above residual plot, we have seen that the solution gets converged, and we didn't find any kind of instability of the iterations of pressure, momentum, and energy in the residual plot.
• At the same time the velocity, continuity, and energy equation are well below the ranges we applied in the basic setting of iterations. Which means the solution gets very well converged.
• Debug divergence is also checked whether the solution has any divergence in iterations. The debug divergence is used for each and every iterations so that we can check for divergence in every iteration.

• The above temperature plot shows how the temperature has behaved at Computer Room Air Conditioner (CRAC) intake and exhaust positions.
• The incoming temperature is set to 12.78 degrees centigrade at both CRAC inlets with a mass flow rate of 7.21 kg/sec.

Results:

Temperature:

• Filled colors, give a good visualization for the temperature, pressure, and mass flow rate, etc.
• The color legends are used to show the temperature contour.

• The above contour shows the temperature behaviour inside the modeled datacentre.
• The inlet temperature from the CRAC units is 12.78 degrees centigrade. But due to the inside temperature or the server racks temperature, the temperature hike is observed and raised up to 29 degrees centigrade.
• In the total of 4 server racks, we have given different heat flux. So that the temperature difference at the server racks has been observed.

• The above picture is the Object face of some of the components we have modeled. The behaviour of the temperature of the server racks has been demonstrated with the contour colors.
• Because of the server rack, 4 is far from the tiles, the highest temperature is recorded at the one of the sides of the servers.
• Due to the difference in the heat flux of server racks, we have given 3000 Watt and 5000 watt for a couple of servers.

Temperature Animation: https://youtu.be/y_KhgLUhByk

Velocity:

• Through CRAC units the air intake has happened and through the exhausts of CRAC warmed air is circulated and went outside.
• The intake velocity of 0.5 m/s is provided in the Y-direction for the circulation of air inside the modeled data center.

• The inlet velocity is given in the Y-direction with 0.5 m/s.
• The maximum velocity circulated and calculated as 21.3 m/s.
• The max. velocity attained at the location of CRAC unit intake, and from there the air is circulated into the entire data center with the dissipated velocity.

Velocity Animation: https://youtu.be/MuFjW_Hyq_8

Vector Animation: https://youtu.be/HIt8CE_7ttM

Conclusion:

• The Datacentre is modeled with the macros as mentioned in the challenge. Using macros, the components like PDUs, CRACs, server racks, etc are provided.
• Using the grouping concept, all the components have been merged into particular groups.
• Using object-specific meshing parameters, the meshing of components like computer room air conditioning units (CRACs), server cabinets, power distribution units (PDUs), and perforated floor tiles have meshed to maintain the accuracy in the captured physics.
• Finally, to understand the flow of air and temperature stratification, the post-processing tools like contours, object faces, vectors, and particle traces have been used.