SVN Repository

From Daya Bay
Jump to navigation Jump to search

Daya Bay software development based on Gaudi is stored in a central Subversion (SVN) repository. For information on installing the offline software see the Offline Software Installation topic.

SVN Quickstart

Basic commands for the impatient user of CVS. 

svn help

Besides listing the svn commands, this will create the ~/.subversion directory with some files and a subdirectory. You will need to edit ~/.subversion/servers if you are behind a firewall.

See also Alternative Subversion Clients.

Generic SVN info

Daya Bay specific info

The root of the SVN repository is at 

http://dayabay.ihep.ac.cn/svn/dybsvn/

Checking out code

User name

You can check out code in one of two ways:

read-only (no commits)
username "dayabay"
commit privileges
personal username

Password for "dayabay" is the usual one. If you already have a CVS account you should have an SVN account, otherwise contact Qiumei Ma (maqm@ihep.ac.cn).

SVN account

To obtain an SVN account, contact Qiumei Ma (maqm@ihep.ac.cn).

Describing what to access with an SVN URL

The SVN URL looks like: 

http://dayabay.ihep.ac.cn/svn/REPOSITORY[/PATH]

Where REPOSITORY is one of:

toysvn
a "playground" to test out SVN
dybsvn
main development area

The optional PATH specifies what part of the repository to download. You almost always want to specify this, otherwise you will get a copy of every commit ever made. The PATH is typically made up of the branch or tag information as well as the directory for the code in which you are interested.

Example SVN URLs

Our offline software: 

http://dayabay.ihep.ac.cn/svn/dybsvn/dybgaudi/trunk

Development branch of our copy of Gaudi: 

http://dayabay.ihep.ac.cn/svn/dybsvn/gaudi/trunk

Just one subdirectory (LCG_Interfaces) 

http://dayabay.ihep.ac.cn/svn/dybsvn/lcgcmt/branches/dyb_52a/LCG_Interfaces/

Checkout command

Finally, you checkout like: 

svn co --username USERNAME URL [TARGET]

The optional TARGET path will cause the checkout to go to that location instead of the current working directory.

Checkout of sparse directories

Details here. Example (only checks out directory people/ ) 

svn co http://dayabay.ihep.ac.cn/svn/dybsvn/people --depth empty

Committing code

Guidelines

Some guidelines for committing:

  • Do not commit generated or derived files, such as "cmt/setup.sh", log files, PDFs, .pyc files, etc.
  • Before committing, Always check what svn status tells you.
  • Procedure:

   svn status          ## look for things like .pyc .pdf .so .a .o that you should not be committing
   svn revert *.{pyc,pdf,so,a,o}  ## revert any unintended additions
   svn status          ## check again that only sources are in your commit
   svn commit -m "carefully composed brief but informative commit message"

  • Use "svn rm" to clean up any unintentional derived file commits that got past your checks.
  • Avoid committing data files.
  • Your people area is yours. Commit any source files you like, as often as you want.
  • Before starting any major work on code written by others discuss what you will do with its original author(s).
  • When committing changes to dybgaudi, run all nosetests to make sure your new code does not cause problems. This can be done via dybinst trunk tests
  • New code should have unit tests written for it. (How to create and install unit test.)

How to add a new folder

   svn add mynewfolder
   cd mynewfolder

Add and Commit commands

SVN basics: 

svn add NAME

svn commit -m "Comments describing new files & directories"

where NAME can be a single file or a directory. If NAME is a directory, then entire contents of the directory are added. For a gentle introduction to doing your first commits see SVN_Commit.

Special Daya Bay Commit Messages

Special commands in the commit message can control how the slave code build and test system responds. More info at Commit Message Keywords.

Moving files in SVN

Documentation of svn move = svn mv = svn rename = svn ren

Example: 

svn  mv -m"move and rename module to do readout mixing for FMCP11a" http://dayabay.ihep.ac.cn/svn/dybsvn/people/djaffe/FMCP11a/racfmix_inputs.py http://dayabay.ihep.ac.cn/svn/dybsvn/dybgaudi/trunk/Production/FMCP11a/python/FMCP11a/runMix/__init__.py

Another example: 

svn mv -m"something meaningful" http://dayabay.ihep.ac.cn/svn/dybsvn/people/djaffe/python/hdrValid.py http://dayabay.ihep.ac.cn/svn/dybsvn/dybgaudi/trunk/RootIO/RootIOTest/python/RootIOTest/.

Backing out a commit

Here's an example as described by Simon Patton:

You wanted to 'back out' r11929. If your commit was the last one to the file(s) in question you should have been able to execute, in the 'jobs_HitMix' directory: 

svn merge -c -11929 http://dayabay.ihep.ac.cn/svn/dybsvn/dybgaudi/trunk/Production/FMCP11a/python/FMCP11a/jobs_HitMix/__init__.py __init__.py

Discussion, motivation in this thread

Maintaining your own code in SVN

Everyone is encouraged to maintain all their code in SVN. This includes:

  • Personal analysis code
  • Experimental forks of or additions to the main code

Personal code

You can (and should!) keep your personal code in the repository. The default place to store such code is in a subdirectory of: 

http://dayabay.ihep.ac.cn/svn/dybsvn/people/

Choose any name you like. To create the initial directory do: 

svn mkdir http://dayabay.ihep.ac.cn/svn/dybsvn/people/myname

you can then checkout and add your files or do a full import.

Setting your global ignores

You may find that 

 svn status -u

gives too much output with many question marks indicating unmanaged files. Get rid of this noise by setting the global-ignores in the miscellany section of 

 $HOME/.subversion/config

setting it to something like 

 global-ignores = setup.sh setup.csh cleanup.sh cleanup.csh Linux-i686* Darwin* InstallArea Makefile load.C *.o *.so *.a

Note that if by mistake you have added such derived files to the repository, then setting them to be ignored will not help. You will need to schedule them for deletion and commit the change to get rid of them. 

 svn delete setup.sh
 svn delete setup.csh   ## etc...

The repository is designed for storing source files and can also store some datafiles if they are useful to several people.

Experimental Forks

You may want to modify the main code but don't want other people to use these changes until they are ready (or maybe never at all). But, you should still keep these changes in SVN!

To do this, you start by copying the code you want to modify to some other location in the repository. Make as many copies as you want; don't worry about using up disk space on the server as SVN has tricks to minimize the space actually used.

A generic example: 

svn copy ORIGINAL_URL DESTINATION_URL
svn checkout --username=NAME DESTINATION_URL my_work_area
cd my_work_area
(start hacking)
svn commit -m "Add geometry for flux capacitors"

The ORIGINAL_URL might be one of:

Some suggestions for DESTINATION_URL are:

Migrating Old CVS Controlled Code Into SVN

If you have modified old CVS checkouts that never got committed back to CVS you can (and should!) put them in SVN. To do this follow one or more of the following steps.

Clean Out Unneeded Files

By default SVN will try to add all files including ones that don't need to be in the repository (.root, *~, Makefile, .so's .o's, etc). Make sure you have cleaned out all these files.

Add Modified Code to SVN

Next your files. The generic command is: 

svn import [PATH] URL

For example, say you want to add your modified version of G4dyb, you can do that like: 

cd /path/to/my/work/area
svn add G4Dyb http://dayabay.ihep.ac.cn/svn/dybsvn/people/NAME/DIR/

Where you can choose:

NAME
some username to identify this as your area
DIR
some directory name that identifies what your modifications are for

This URL is an example, you are free to use anything that makes sense to you.

Merge Recent Development Into Your Modified Code

Described at SVN_Merge

Merge Your Code Into Recent Development

Also described at SVN_Merge

SVN "Gotchyas" or "Gotchas"

There are a few things that may cause trouble with SVN.

Access from behind a firewall

If you are behind a firewall that blocks outgoing HTTP you will need to tell SVN about your web proxy server. An indication that this is your problem is that you will get a "svn: PROPFIND ... could not connect to server" message. Edit ~/.subversion/servers to add lines like: 

[groups]
dayabay = dayabay.ihep.ac.cn

[dayabay]
# BNL's proxy server address and port
http-proxy-host = 192.168.1.165
http-proxy-port = 3128

Alternatively you can just send all SVN traffic through the proxy: 

[global]
# BNL's proxy server address and port
http-proxy-host = 192.168.1.165
http-proxy-port = 3128


These examples show what is required for BNL users.

SVN Server-side Information

Users need not read this.

Path-Based Authorization

Both Apache and svnserve are capable of granting (or denying) permissions to users. Typically this is done over the entire repository: a user can read the repository (or not), and she can write to the repository (or not). It's also possible, however, to define finer-grained access rules. One set of users may have permission to write to a certain directory in the repository, but not others; another directory might not even be readable by all but a few special people.

Both servers use a common file format to describe these path-based access rules. In the case of Apache, one needs to load the mod_authz_svn module and then add the AuthzSVNAccessFile directive (within the httpd.conf file) pointing to your own rules-file. The Dayabay configuration for authenticated access. 

<Location /svn>
  DAV svn
  SVNParentPath /home/svn
  # our access control policy
  AuthzSVNAccessFile /etc/svn-authzaccess-file
  # only authenticated users may access the repository
  Require valid-user
  # how to authenticate a user
  AuthType Basic
  AuthName "Subversion repository"
  AuthUserFile /etc/svn-auth-file
</Location>

The following svn-authzaccess-file defines the rules for path-based access. 

[groups]
admin = tianxc, bv, caoj, patton, huangxt, tagg
developpers = antonyluk, blyth, bv, caoj, dengzy, etc.
users = dayabay, ntusync
[dybsvn:/]
@admin = rw      # the group of "admin" has the full read and write access on all the directories, 
                 # including create/delete top-level directories, in the dybsvn repository
@developers = r  # the groups of developers and users only have the read access.  
@users = r       # 
[dybsvn:/dybgaudi]
@admin = rw      # the group of "admin" has the full read and write access on all the directories, 
                 # including create/delete top-level directories, in the dybsvn repository
@developers = rw # the group of developers has the read and write access. 
@users = r       # the group of users has the read access.
...

If a new top-level directory is added into the dybsvn repository, the svn-authzaccess-file must be modified to enable the developers have the write permission. For example, the recent "relax" project, I need to add the following lines: 

@admin = rw      # implicit, can be omitted
@developers = rw # explicit, must be here
@users = r       # implicit, can be omitted

The svn-auth-file defines the password for each user, when a new user is added, please follow what I put down: 

htpasswd -m svn-auth-file newuser
New password: *******
Re-type new password: *******
Retrieved from "https://wiki.bnl.gov/dayabay/index.php?title=SVN_Repository&oldid=4425"