E N A B L I N G T E C H N O L O G I E S

1 5 8 H I G H L I G H T S 2 0 2 2 I

VISA A new solution for remote access to IT infrastructure

A new IT service deployed at the ESRF allows users to treat large volumes of complex experimental data onsite or remotely using a simple web browser. The service also allows collaborative data processing. The tool will be useful in the context of increasing dataset volumes as a result of the EBS.

Over the past few decades, the volume of datasets produced by synchrotron experiments has grown and nowadays commonly reaches 5 to 10 TB for a single session, with some exceptions rising to more than 50 TB. This trend will continue with the new flagship and renovated beamlines planned to fully benefit from the capacity of the ESRF s new Extremely Brilliant Source (EBS). The complexity of the data processing is also increasing and quite often requires a succession of steps managed by different software.

These volumes of data create challenges for our user community, who do not necessarily have the IT resources at hand to perform the treatment of such complex data. The ESRF has developed and continues to develop workflows to automate the data treatment when possible. Yet, despite these efforts, there are necessary and significant needs for scientist-driven data processing.

In order to help research infrastructure users face these difficulties, in the scope of the EU-funded Photon and Neutron Open Science Cloud (PaNOSC) project, a new service called VISA has been developed, in concertation with PaNOSC partners. VISA is an IT infrastructure that provides access to the raw data and already partially processed data, hardware resources (CPU, GPU, RAM) and preinstalled software, personalised to perform the specific treatment of data resulting from the experiment. In the case of the ESRF setup, a SLURM cluster has also been added to allow MPI and GPU intensive tasks to be performed. This infrastructure is accessible to our users from the ESRF premises or remotely using a simple web browser.

From a user s point of view, simply open a web browser and access https://visa.esrf.fr, select the data you want to analyse, wait 20 seconds for a machine tailored to your needs to be created, and you are ready to start processing. Two interfaces are available; either a graphical desktop (as if you were physically sitting in front of the screen of the machine, as illustrated in Figure 142) or a Jupyter notebook. These graphical interfaces to the machine can be shared between users to allow collaborative processing, but also with ESRF technical or scientific experts to ease ESRF support in case of need.

Fig 142: VISA graphical desktop interface.