Computing

From Electron Ion Collider
Jump to: navigation, search

The BNL EIC group has its own RACF (RHIC and ATLAS Computing Facility) computer cluster dedicated to simulation and analysis. Users with an RACF EIC account are free to use these nodes for EIC-related work. Any questions about EIC computing and software should be directed to the software mailing list.

Obtaining access

You should request an EIC computing account at RACF by following the instructions on this page. Note you must do this even if you already have an RACF computing account (e.g. as a member of the STAR of PHENIX experiments); you will be assigned a separate, EIC-specific account. Please also email Elke Aschenauer when submitting the request form so that she can approve the account creation.

General information about accounts and logging on to RACF can be found on the RACF webpage. Also, and especially if you are logging in from offsite, using the NX system can make working on RACF significantly easier.

Computing resources

There are the following dedicated nodes for EIC use:

  • eic0001
  • eic0002
  • eic0003
  • eic0004 [interactive use only]
  • eic0009
  • eic0010

These nodes should be used solely for EIC work. eic0005-eic0008 are reserved for the polarimetry group and should not be used for EIC work. To access a node, first log into the RACF gateway machine

ssh <your username>@rssh.rhic.bnl.gov

then select a node from the list above

ssh eic000#

eic0004 is reserved for interactive use and brief test jobs; long jobs (such as significant Monte Carlo generation and detector simulations) should be run on one of the other nodes. Only specific EIC nodes are currently part of a queue system, such as condor (eic0004, eic0007, and eic0008). Each node has 16 (eic0001-4) or 24 (eic0009+) cores, so you may run multiple jobs in parallel on a given node. However please be considerate of the needs of others before submitting very large numbers of computationally intensive jobs.

EIC disk space is divided into:

  • user area: /eic/u
  • data areas: /eic/data and /eicdata
  • AFS volume: /afs/rhic.bnl.gov/eic

When your EIC account is created, you will be given both an EIC user area and an EIC AFS user area:

/eic/u/<your username>
/afs/rhic.bnl.gov/eic/users/<your username>

Neither of these user areas should be used to store large quantities of data. There are no enforced quotas, but please be considerate of other users and do not take up more than your fair share of space. And be warned: if you fill a user area your files may be deleted by an administrator without warning! Large files should instead be placed under one of the EIC data areas, either

/eic/data

or

/eicdata/eic000#

where eic000# is the name of one of the nodes listed above. Your account gives you permission to create directories and files on any data area. Again, there are no quotas on these disks, but please avoid completely filling a disk. Note that while user areas are routinely backed-up, data areas are not, so any scripts, code etc. needed to reconstruct your analysis in case of data loss should be kept on a user area.

AFS

The EIC AFS region

/afs/rhic.bnl.gov/eic

collects all shared analysis tools, such as Monte Carlo generators, detector simulations and common analysis scripts. For most users, the most useful directory under the EIC AFS region will be PACKAGES:

/afs/rhic.bnl.gov/eic/PACKAGES

which collects links to installations of the various EIC simulation and analysis programmes. Please see the simulations page for details of the Monte Carlo generators present there. Space under AFS is limited, so please do not output data from any of these programmes to your AFS area - use one of the data disks.

Additionally there are useful code and analysis tools, such as installations of Boost and gnuplot, at

/afs/rhic.bnl.gov/eic/TOOLS

Most of these tools are installed under the bin/, include/ and lib/ directories in the main AFS area, and so can be used by appropriately setting your PATH and LD_LIBRARY_PATH environment variables. This is done automatically if you source the EIC logon script (below).

Configuring the environment

To simplify running EIC packages, it is recommended that you configure your computing environment appropriately.

EIC logon

Most of the configuration needed to run EIC packages is provided in the (t)csh script

/afs/rhic.bnl.gov/eic/etc/eic_cshrc

For many users it is simplest to add the following in a startup file (e.g. .cshrc):

source /afs/rhic.bnl.gov/eic/etc/eic_cshrc

Running this script will ensure use of the correct EIC installations of, amongst others: ROOT, Python, Boost and LHAPDF.

Multiple accounts

Many users may use RACF for computing tasks other than EIC (e.g. PHENIX, STAR), and may want to perform different logon tasks for each. One possible solution is to only set the EIC environment when logging onto an EIC node. For example under (t)csh:

if("eic" == `uname -n | sed 's/\([a-z]*\).*/\1/'`) then
   source /afs/rhic.bnl.gov/eic/etc/eic_cshrc
else
   <do your non-EIC logon>
endif

The EIC logon will then only performed when logging into a node named "eic…".

ROOT

The EIC group performs most of its analysis using ROOT, and maintains its own installation, documented here. All EIC ROOT code is compiled and tested against this version of ROOT, so it is recommended that this is the version used whenever using EIC ROOT libraries or scripts. This version is automatically selected (via the environment variable ROOTSYS) if the environment is configured by eic_cshrc.

Users may find it useful to run the script

/afs/rhic.bnl.gov/eic/CODE/etc/rootlogon_eic.C

when starting the EIC installation of ROOT. This script loads some commonly used libraries, and adds some EIC directories to the ROOT include path, simplifying compiling of analysis scripts. To load it automatically, create a file named .rootrc in your home directory (if it does not already exist) and add the line

Rint.Logon /afs/rhic.bnl.gov/eic/CODE/etc/rootlogon_eic.C

If you already run a ROOT logon script and wish to incorporate the EIC logon script you can add the following to your existing script:

gROOT->Macro("/afs/rhic.bnl.gov/eic/CODE/etc/rootlogon_eic.C");

or if you only wish to load it when on an EIC node:

if(TString(gSystem->Getenv("HOST")).BeginsWith("eic")) {
   gROOT->Macro("/afs/rhic.bnl.gov/eic/CODE/etc/rootlogon_eic.C");
}

Python

The AFS PACKAGES directory contains its own installation of Python. It is this version against which other EIC programmes are installed, so it is important to use this version whenever using other EIC programmes. eic_cshrc ensures this version is used and sets the environment variable PYTHONPATH appropriately. If the user sets either or both PYTHONPATH themselves, they should do so after sourcing eic_cshrc to avoid their settings being changed.

Troubleshooting

If you are using or writing some code, and its crashes or experiences problems, here are some useful resources:

  • GDB, the Gnu debugger, can be used to locate where in your code a crash is occurring.
  • Valgrind is available on the RACF nodes and can be used for a wide variety of debugging and profiling tasks.
  • Stack Overflow has lots of threads answering coding questions and problems - if you encounter a problem, you may well find an answer there.
  • Ask others on the software mailing list.