For many years the ESRF has been reliant on two data centres, one in central building and the other in the control room building. The basic principle is to ensure that the fast and expensive disk storage systems are kept separate from the tape-based backup systems. Similarly, several critical IT services have been installed using both data centres for additional safety in case of a failure of one of them. For years it has been apparent that the central building data centre would need to be upgraded or complemented by a new room; the increasing influx of experimental data requires substantial additional space, power, and cooling. This is why a new data centre was defined within the scope of the ESRF Upgrade Programme, and initially planned in the new premises of the experimental hall extensions. However, as the experimental hall extensions will not be available before 2013, it was decided instead to opt for a complete overhaul of the central building data centre scheduled to be operational at the beginning of 2011.

The new data centre is designed to improve three key parameters by increasing:

  • electrical power from 75 kW to 370 kW,
  • cooling power proportioned appropriately for the electrical power with a target temperature of 23°C ± 1°C,
  • floor space from 120 m2 to 300 m2 with a load resistance of 1000 kg/m2 instead of the existing 500 kg/m2.

Detailed specifications were drafted to incorporate state-of-the-art techniques, with the aim of improving both efficiency and reliability.

Bearing in mind that the IT infrastructure must run non-stop, the electrical network specifications include two separate circuits for each device in the computer room. The overall electrical supply will be composed of a 500 kVA uninterruptable power supply (UPS) with a 20 minute autonomous run time and two separate input sources from the mains network. All electrical supplies will be fed in through the ceiling, thus avoiding installing any equipment in the raised floor. The cooling systems will be based on two different supplies; direct chilled water for water-cooled racks and chilled water for air cooling. The chilled water production will rely on two independent production circuits (Figure 151). The new data centre will be equipped with fire detection and a powerful smoke extraction system; it also features fire-resistant walls. A state-of-the-art building management system will enable remote monitoring and surveillance.

Fig. 151: Installation of the heat exchangers on the roof of the ESRF central building.

The future rack distribution will orient the racks to create hot and cold aisles; computers take cold air from the front side of the racks and reject hot air through the rear side to the hot aisle where the extraction of the hot air is optimised. High power computing equipment will be concentrated in water-cooled racks located in a confined area of 22 m2. Last, but not least, the network cabling will be organised for optimal cable length and flexibility with the network electronics located in the middle of each row of racks.

It should be noted that keeping the data centre within the central building reduces the total construction cost and consequently the associated carbon foot print.

Works started in 2009 with a preparatory phase consisting of the modification of many adjacent rooms on the ground floor of the central building. Several storage rooms and laboratories have been moved, a few offices have been freed up to accommodate the future technical rooms, and two new corridors have been created to comply with safety regulations. Major works started in summer 2010 and the project is progressing on schedule.

The entire project will cost 2 M€ with a delivery foreseen in March 2011.

 

Authors

B. Lebayle, T. Marchial and R. Dimper.
ESRF