Environment Management with nuwaenv

From Daya Bay
Jump to navigation Jump to search
Offline Documentation: [Offline Category] [FAQ] [Howto] [Reference] [Manual]


Offline Documentation
This article is part
of the Offline Documentation
Other pages...

Offline Category
FAQ
How Tos
Reference
Manual
Getting Started
Software Installation

See also...

General Help on this Wiki



A generic, user friendly setup manager is provided by nuwaenv. Before using it and as new releases are made available at your site some amount of configuration is needed.

Using nuwaenv

When properly configured users manage their environment with the command

nuwaenv [options] [file.cfg ...]

A brief synopsis of the options can be seen with

nuwaenv -h

This command is a shell function or alias. To define it your must do:

source nuwaenv.sh  # if you use bash
source nuwaenv.csh  # if you use tcsh

Your site manager (or you) need to assure that your shell sources this (see below for installation). It is okay to source these files for every shell as they only define the nuwaenv command and do not otherwise change your environment.

Setting up a base release

To set up using a base release do, for example:

nuwaenv --release-name trunk

or more using the shorter version of the command line option:

nuwaenv -r 1.5.0

If no release is specified it will default to whatever release has been set as default in the configuration.

Using your own project

See below for necessary set up to use this feature. You can set up your environment to use your own CMT projects with

nuwaenv [other options] -l project --project=myproject

Installing nuwaenv

Nuwaenv consists of:

  • Main program "nuwaenv.py"
  • Python module "nuwaenv"
  • Generated sh/csh scripts to be sourced by users.

To install, pick a location to be shared by all users, check out the code from SVN and generate the scripts:

cd ~dayabay/
svn co  http://dayabay.ihep.ac.cn/svn/dybsvn/installation/trunk/dybinst/python 
./python/nuwaenv.py -S `pwd`/nuwaenv

Users should then source the script appropriate for their shell. In this example that would be one of:

source ~dayabay/nuwaenv.sh  #for bash
source ~dayabay/nuwaenv.csh #for tcsh

Configuring nuwaenv

Nuwaenv must be configured with details about the NuWa base releases that exist on a system. A user can also configure it to know about personal projects.

Testing a Configuration

Before learning how to configure nuwaenv, know you can test a configuration with:

nuwaenv -t [...other options...]

Configuration Files

Implicitly, nuwaenv will look for any and all configuration files in the following order:

$NUWAENVRC
~dayabay/.nuwaenv.cfg
~dayabay/.nuwaenv.d/*.cfg
~/.nuwaenv.cfg
~/.nuwaenv.d/*.cfg
./.nuwaenv.cfg 
./.nuwaenv.d/*.cfg
any listed on the command line

If $NUWAENVRC or any listed on the command line are directories then all *.cfg files in the given directory will be loaded.

The configuration built up through one or more configuration files applied in series. Latter files are merged with or override configuration from prior ones.

Syntax

A configuration file has one or more named sections, each of which defines one or more variables. A section starts with its name in "[]". For example,

[mysection]

There is a special section called "[defaults]" which can define default variables for all other sections.

Variables can be defined with values that can use Python style interpolation using other variables defined in the section or in the special "[defaults]" section. Interpolations start with a "%" followed by a variable name between parenthesis and appended with a type code, usually "s" for string. For example: "%(VARIABLE)s". For example:

[defaults]
var1 = foo
var2 = bar
var3 = %(var1)s is %(var2)s

Here, var3 would be "foo is bar". Adding another section:

[mysection]
var2 = baz

makes var3 be "foo is baz".

This interpolation allows for flexible configurations. What follows is some guidance on how to exploit this.

Base Release Configuration

Base release configuration must provide just one thing:

base_release

This should be an absolute path to the release directory (the one holding dybgaudi/). This can either be defined in the "[defaults]" section or in a named section. Named sections can be selected using the -r|--release-name command line option. Note, these release names can be freely chosen and need not match the canonical release names used by the NuWa release themselves.

Some variables are provided to further help define base_release:

opt_or_dbg
  • Indicates to use optimized or debug builds of a release
  • set by --opt-or-dbg=[opt|dbg]|--optimize|--debug
  • takes value either "opt" or "dbg"
  • default is "dbg"
release_name
  • Indicate the name of a configuration section that provides base_release and can be used in interpolation
  • set by -r|--release-name=RELEASE
  • default is unset
  • can be set in [defaults] section.

Generally, the base releases should be configured by the same person that installs them and users should automatically pick up this configuration. Users then chose to append or modify that configuration to describe their own projects that they may have installed.

User Project Configuration

Users can tell nuwaenv about their own CMT projects such that they can setup for a project like:

nuwaenv -l base [-r RELEASE]
nuwaenv -p myprojectname

The setup level of "project" (explicitly set via -l project) is implicitly defined when -p|--project-name flag is seen.

For this to work, the configuration needs to supply the following variables:

project_dir

The absolute path to the project directory (analogous to "base_release" for base releases).

package

A path, relative to project_dir, to a package that provides the desired environment.

A simple example that you might put in ~/.nuwaenv.cfg:

[defaults]
project_dir = ~/work/dayabay/myproject
package = subdir/MyReleasePackage

Through the command line's -p|--package-name=PROJECT on can set the project_name. This names a configuration section that defines the required variables and can be used in interpolation. This is analogous to the release_name of the base release.

A larger example:

[defaults]
project_dir = ~/work/dayabay/%(project_name)s

[myproject]
package = subdir/MyReleasePackage

[myotherproject]
package = MyAnaPackage

Optional Configuration Parameters

Nuwaenv will honor some additional configuration parameters which are optional.

Hard coded environment setup

You can put environment setup directly in a configuration file stanza using these two parameters:

pre_env
environment setup enacted before that generated by nuwaenv
post_env
environment setup enacted after that generated by nuwaenv

In both cases this environment setup is incorporated into the environment settings that are generated by nuwaenv. For example, if the variable CMTCONFIG is set in pre_env it may affect many other variable settings that nuwaenv emits.

The environment setup is of the form of a list of command strings. They will be interpreted and the results will be incorporated by nuwaenv so as to emit shell-specific setup. As an example:

pre_env = ['<command> <cmd args>','<command cmd args>']

The supported commands and there arguments are:

source <file>
source the given <file>. The interpolation "%(shell_ext)s can be used to expand to an extension corresponding to the shell being used (ie, .sh or .csh). After interpolation the command is passed to the output untouched. As an example:
source ~/myfile.%(shell_ext)s
set <variable>=<value>
set an environment variable to the given value. As an example:
set MYSPECIALPLACE=$HOME/special
set VAR_WITH_SPACES="a trip down memory lane"

Todo

  • Add ability to specify external environment generation commands (like PDSF's "module") (work around still needed to call things like "module"
  • Add ability to specify environment directly into the config files
  • Add feature and cmd line flag to list all base release and project stanza names in the config files so users can see what is available.


Offline Software Documentation: [Offline Categories] [FAQ] [Offline Manual Category]