Computing

The BNL EIC group relies on the Scientific Data and Computing Center (SDCC) at BNL for its resources. The facility is sometimes referred to as "RACF" for historical reasons - the RHIC and ATLAS Computing Facility was the predecessor of the SDCC. The group has access to a computing cluster hosted by SDCC and dedicated to simulation and analysis. Users with a SDCC account accredited with EIC are free to use these nodes for EIC-related work. Any questions about EIC computing and software should be directed to the software mailing list.

Obtaining access to the computing facility

Useful information can be found on the "Getting Started" page on the SDCC website.

You should request a SDCC EIC computing account by following the instructions on this page and in this quickstart document. Note you must do this even if you already have a SDCC account (e.g. as a member of the STAR of PHENIX experiments). You will be assigned a separate, EIC-specific account. Please also email Elke Aschenauer when submitting the request form so that she can approve the account creation.

Pretty much any EIC-related activity will require access to one of these dedicated EIC nodes:

• eic0101, eic0102, eic0103, eic0104, eic0105, eic0106, eic0107, eic0108, eic0109, eic0110, eic0111, eic0112

Information about how to log on i.e. the SSH gateways and other modes of access can be found on the "Getting Started" page.

These nodes should be used solely for EIC work. All nodes can be used interactively, but beyond small tasks and tests, long jobs (such as significant Monte Carlo generation and detector simulations) should be submitted to the general condor queue, where significant amount of worker nodes have been added for EIC-related work.

The eic01## nodes are virtual machines currently configured with 8 cores and 16 GB of RAM each. You may run multiple jobs in parallel on a given node. However please be considerate of the needs of others before running large numbers of computationally intensive tasks.

Note: Older physical nodes in the range eic0001-eic0010 are currently still available but their use is strongly discouraged. eic0005-eic0008 are reserved for the polarimetry group and should not be used for EIC work at all. The remaining nodes are advancing in age and subject to retirement without notice.

Additional connection options

• To avoid having to retype your password as often, you can create a longer lived kerberos token before "rterm" with

kinit -l 7d # good for 7 days

• You can select a specific node via, e.g.,

rterm -i eic0###

• To speed up remote X11 performance you can try the -C flag for ssh. If that is insufficient, it is recommended to use NoMachine/NX.

EIC disk space

• User area: /eic/u. Your home directory resides here. This space is backed up with daily snap shots for seven days. It should be used for your code, documents, and other important items. It is not suitable for large amounts of data and has a quota of 10 GB.
• data areas:
  • /gpfs02/eic is the default large storage area with a quota of 375 TB, and you can create your own directory in it.
  • Large MC productions useful for collaborators and the community should be placed in /gpfs02/eic/DATA. The contents of this directory can be shared (read-only) via BNLbox.
  • /eic/data is a symbolic link to /gpfs01/eic/data. The 6TB quota on this disk is rather full, so it should not be used very much. It mostly serves as a legacy collection of symbolic links to /gpfs02/eic/DATA.
• AFS volume: /afs/rhic.bnl.gov/eic

Your account gives you permission to create directories and files on any data area.

When your EIC account is created, you will be given both an EIC user area and an EIC AFS user area:

/eic/u/<your username>
/afs/rhic.bnl.gov/eic/users/<your username>

Neither of these user areas should be used to store large quantities of data. There are no enforced quotas, but please be considerate of other users and do not take up more than your fair share of space. And be warned: if you fill a user area your files may be deleted by an administrator without warning! Large files should instead be placed under one of the EIC data areas mentioned above. Note that while user areas are routinely backed-up, data areas are not, so any scripts, code etc. needed to reconstruct your analysis in case of data loss should be kept on a user area.

EIC software

The EIC software collection of shared analysis tools, such as Monte Carlo generators, detector simulations and common analysis scripts is currently held in the EIC AFS region at /afs/rhic.bnl.gov/eic/restructured, but may soon be migrated to /cvmfs. You should avoid absolute paths and use the environment variable ${EICDIRECTORY}. It is set automatically together with PATH and LD_LIBRARY_PATH if you source the EIC logon script (below).

For most users, the most useful directory will be PACKAGES:

${EICDIRECTORY}/PACKAGES

which collects links to installations of the various EIC simulation and analysis programs and tools such as eic-smear. Please see the simulations page for details of the Monte Carlo generators present there. Space under AFS is limited, so please do not output data from any of these programs to your AFS area - use one of the data disks.

Most libraries, headers, and executables will also be installed in ${EICDIRECTORY}/lib, ${EICDIRECTORY}/include, and ${EICDIRECTORY}/bin for ease of use.

Configuring the environment

To simplify running EIC packages, it is recommended that you configure your computing environment appropriately.

EIC logon

Most of the configuration needed to run EIC packages is controlled by the environment variable EIC_LEVEL and provided in the (t)csh script currently at

/afs/rhic.bnl.gov/eic/restructured/etc/eic_cshrc.csh

For many users it is simplest to add the following in a startup file (e.g. ~/.cshrc):

setenv EIC_LEVEL pro
source /afs/rhic.bnl.gov/eic/restructured/etc/eic_cshrc.csh

EIC_LEVEL selects the release version in /afs/rhic.bnl.gov/eic/restructured/etc. pro and dev are symbolic links to the current production and development versions, respectively.

Running this script will ensure use of the correct EIC installations of, amongst others, ROOT, eic-smear, and LHAPDF. As noted above, the path to the eic_cshrc.csh script in your .cshrc should in almost all cases be the only instance of an absolute path, and you should use $EICDIRECTORY subsequently.

Multiple accounts

Many users may use RACF for computing tasks other than EIC (e.g. PHENIX, STAR), and may want to perform different logon tasks for each. One possible solution is to only set the EIC environment when logging onto an EIC node. For example under (t)csh:

if("eic" == `uname -n | sed 's/\([a-z]*\).*/\1/'`) then
   setenv EIC_LEVEL pro
   source /afs/rhic.bnl.gov/eic/restructured/etc/eic_cshrc.csh
else
   <do your non-EIC logon>
endif

The EIC logon will then only performed when logging into a node named "eic…".

Troubleshooting

If you are using or writing some code, and its crashes or experiences problems, here are some useful resources:

• GDB, the Gnu debugger, can be used to locate where in your code a crash is occurring.
• Valgrind is available on the RACF nodes and can be used for a wide variety of debugging and profiling tasks.
• Stack Overflow has lots of threads answering coding questions and problems - if you encounter a problem, you may well find an answer there.
• Ask others on the software mailing list.