Why NICE was born

The ESRF beamlines increasingly produce large amounts of data.

A typical experiment can easily generate several thousand high-resolution images - our record in spring 2016 was 109 TB. With 40 operational beamlines the total amount of data can indeed be impressive.

  • How can we store such amounts of data?
  • How can we ensure that storage space is constantly available for beamline data?
  • What will be  the means to analyze these large data volumes at ESRF?
  • Finally, how will the scientist be able to transfer the data to her/his home institute?

The answers to these questions are not always trivial and imply some of the most advanced computer technologies available on the market. The framework within which TID gradually implements solutions, is called NICE and stands for Networked Inte ractive Computing Environment.

A NICE history

Looking back at the history of a project is philosophically interesting.  The advances of technology, in particular in computing, are breath taking. In 1984 the project ESRF was defined in the so-called green book. At that time a central computing facility was envisaged to take care of all major aspects of computing at ESRF. The facility would consist of two main-frames with shared disks and tape drives. The total envisaged disk capacity was 2.6 GBytes! 

3 years later, in 1987, a similar configuration was described in the red book. The red book was at the basis of the ESRF project. At this time a price tag of 25 MFF (400 kEuro) was estimated for an initial installation comprising already three main-frame like systems and a total of 3 GBytes of disk space. 

In the following years ESRF experienced (like many other places in the world) the trend for distributed computing. Instead of alphanumerical terminals, everybody wanted to have his personal computer on his desk. As a consequence the data communication network became fundamental for the well being of our institute, and a significant amount of money had to be spend to install a modern and flexible network infrastructure. Despite the numerous advantages of a distributed computing environment (there are now more than 11000 hosts on the ESRF network), it was always without doubt that some functionalities are best managed in a centralized installation. The coming into operation of the first set of beamlines triggered the ESRF Computing Services to propose a centralized facility.

 

In 1993 the initial specifications for NICE were defined in a large consultation with the ESRF staff. The implementation started in 1994, and the continuous investment in networking, data storage, compute servers, and software, reflects the central role NICE continues to play in the data acquisition process of the ESRF beamlines.