qlnx-psets Home Page

Setup and Use

To start using qlnx-psets, you need to start by installing cvsps, that you can find at:

http://www.cobite.com/cvsps

Download the package and install it by following the instructions given by the authors (and say thanks to them since cvsps really brings back order to the CVS caos). The whole process starts by rsync'ing the bkcvs CVS repository from kernel.org, that in my machines usually goes like:

$ cat bksync.sh
#!/bin/sh

rsync -vzrlt --delete rsync.kernel.org::pub/scm/linux/kernel/bkcvs /usr/src/


The shell script above fetches the bkcvs CVS repository into the /usr/src/bkcvs directory. Since rsync is based upon a smart protocol, updates of the local repository are typically fast and not bandwidth hungry. The qlnx-psets tool supports a few command line options, that can also be fed using a configuration file (to avoid retyping):

qlnx-psets.pl [--qroot DIR] [--root DIR] [--repo NAME] [--outdir DIR]
              [--cfgfile FILE] [--no-cvs-direct] [--no-bkcvs] [--hard-links] [--help]
              cmd [arg [, arg]]


Mapping between command line options and configurations file options are:


My example configuration file looks like:

# Maps to --qroot (defaults to environment QLNX_QROOT)
$qlnx_qroot = "/home/nobkup/qlnx-psets";

# Maps to --root (defaults to environment QLNX_ROOT)
$qlnx_root = "/usr/src/bkcvs";

# Maps to --repo (defaults to environment QLNX_REPO)
$qlnx_repo = "linux-2.5";

# Maps to --outdir (defaults to environment QLNX_OUTDIR)
$qlnx_outdir = ".";

# Maps to --no-cvs-direct (defaults to environment QLNX_CVSDIRECT)
$qlnx_cvsdirect = "--cvs-direct";

# Maps to --no-bkcvs (defaults to environment QLNX_BKCVS)
$qlnx_bkcvs = "--bkcvs";

# Maps to --hard-links (defaults to environment QLNX_USEHL)
$qlnx_usehl = 1;

The qlnx-psets.pl root directory (--qroot) is the place where the tool generate its own metadata to help and speedup the user command processing. Inside such directory, the following structure can be found:


The "caches" directory stores cached copies of the tree, to help cutting down the generation time of user configurations. Inside the "caches" directory, one or more directory will be stored by the user, with name equal to the patchset number they cache precedeed by the '#', or the version string they refer to. For example, my current "caches" contains only one cached copy "#18121", that corresponds to 2.6.6 (will see later how):

[davide@bigblue qlnx-psets]$ ll caches/
total 4
drwxrwxr-x 18 davide davide 4096 May 9 19:33 #18121

Inside cache directories we will find the typical content of the cached tree:

[davide@bigblue qlnx-psets]$ ll caches/#18121/
total 284
drwxrwxr-x 22 davide davide 4096 May 9 19:31 arch
-rw-r--r-- 1  davide davide 18691 May 9 19:32 COPYING
-rw-r--r-- 1  davide davide 85686 May 9 19:32 CREDITS
drwxrwxr-x 2  davide davide 4096 May 9 19:33 crypto
drwxrwxr-x 42 davide davide 4096 May 9 19:33 Documentation
drwxrwxr-x 44 davide davide 4096 May 9 19:33 drivers
drwxrwxr-x 53 davide davide 4096 May 9 19:33 fs
drwxrwxr-x 33 davide davide 4096 May 9 19:32 include
drwxrwxr-x 2  davide davide 4096 May 9 19:33 init
drwxrwxr-x 2  davide davide 4096 May 9 19:33 ipc
drwxrwxr-x 3  davide davide 4096 May 9 19:33 kernel
drwxrwxr-x 4  davide davide 4096 May 9 19:33 lib
-rw-r--r-- 1  davide davide 51590 May 9 19:32 MAINTAINERS
-rw-r--r-- 1  davide davide 36156 May 9 19:32 Makefile
drwxrwxr-x 2  davide davide 4096 May 9 19:33 mm
drwxrwxr-x 32 davide davide 4096 May 9 19:33 net
-rw-r--r-- 1  davide davide 13976 May 9 19:33 README
-rw-r--r-- 1  davide davide 2815 May 9 19:32 REPORTING-BUGS
drwxrwxr-x 7  davide davide 4096 May 9 19:33 scripts
drwxrwxr-x 3  davide davide 4096 May 9 19:33 security
drwxrwxr-x 15 davide davide 4096 May 9 19:33 sound
drwxrwxr-x 2  davide davide 4096 May 9 19:33 usr

I could rename the directory caches/#18121 to caches/2.6.6 and the effect would result to be the same. When used together with the bkcvs-generated data, caches should be retrieved directly from the CVS root, and not from kernel tarballs. This because the kernel tarballs is missing of some directories/files that patchsets wants to patch. Failing in doing so will result in errors from the patching operations. To do this you need to checkout from the CVS root by the date of the release you want to checkout. To do this, supposing we want to checkout release 2.6.5, we first search the logs for such release:

$ qlnx-psets.pl sl 'Tag: v2_6_5'

This will dump all log entries that contains such tag, and at the end you will see something like:

********************************
PatchSet 17248
Date: 2004/04/04 04:34:07
Author: torvalds
Branch: HEAD
Tag: v2_6_5
Log:
Linux 2.6.5

BKrev: 406f822foQTgmLYKWBzPlaKSfZ_q_g

Members:
        ChangeSet:1.17248->1.17249
        Makefile:1.423->1.424

This means that such tag has been created on 2004/04/04. So we can then use such date to checkout from CVS:

$ cd caches
$ cvs -d /usr/src/bkcvs co -D 2004-04-04 linux-2.5
$ mv linux-2.5 2.6.0

Other caches can simply be created with the qlnx-psets tool itself, by creating custom configuration file like:

$ cat 2.6.6.conf
@2.6.6

and by issuing:

$ qlnx-psets.pl cs 2.6.6.conf
$ mv linux-2.5 $QLNX_QROOT/caches/2.6.6

The qlnx-psets tool uses some heuristic to find out the best cache to use to generate a given user configuration. Care should be taken when using the --hard-links option because the generated copy uses hard link to the cached copy. The qlnx-psets tool automatically break the link when applies the patchsets, but the user should be careful when editing the built configuration. We will show later how to find mappings between patchset numbers and kernel revisions. The "patches" directory is the place where cvsps generated patch files wil be stored. File names inside the "patches" directory have the format:

PATCHSETNUM.patch

where PATCHSETNUM is the patchset that has to be used to migrate to version PATCHSETNUM. The "xpatches" directory stores patches that do not come from the CVS patch set, but that are instead coded by the user, and that the user wants to apply to his configuration. The "lastps" file is used by the tool to store the last patchset synced from the CVS repository, in order to cut update times. Only new patchset files will be generated when ussuing a 'cupdate' command. The "logfile" is a cached copy of the full cvsps log file, and it is used to speed up operations that might be time bound to the cvsps processing performance. The "migfile" is generated by the qlnx-psets tool, by parsing the "logfile" and detecting associations between patchset numbers and kernel versions. An example few lines of my "migfile" are:

2.6.5     17248
2.6.6-rc1 17705
2.6.6-rc2 17841
2.6.6-rc3 17985
2.6.6     18121
2.6.7-rc1 18806
2.6.7-rc2 19020
2.6.7-rc3 19355

We see here that, for example, the patchset 18121 correspond to the kernel version 2.6.6 (the one that is shown cached in the lines above). The "psets" file stores, one by line, all the patchsets contained inside the cvsps log file. To start using qlnx-psets, you can just edit the supplied configuration file and edit the variables to fit your setup. Typically you might want to set $qlnx_qroot, $qlnx_root, $qlnx_repo and $qlnx_outdir. As first-time caching speed up trick, you can edit (create actually) the "lastps" file containing the oldest patchset that you will ever request to remove. For example, if you know you configuration will never go back past version 2.6.2, you can set the "lastps" to 15325 (patchset version corresponding to 2.6.2). Then you can start issuing your first request to qlnx-psets, and this will be the one that will first create and sync its metadata with the CVS repository selected inside the configuration file:

$ ./qlnx-psets.pl --cfgfile qlnx.config cu

The command will take some time, because the qlnx-psets tool will ask cvsps to generate all patchsets starting from the "lastps" one up to the most current one (note that if you didn't set the "lastps" file, this can really take some time). The good news is, that new updates will always take a fraction of the time the first one took (depending on how many new patchsets have been generated inside the selected CVS repository).



This is a list of commands supported by the qlnx-psets tool:

cupdate
cu


migs [VER] ...
mg [VER] ...


logs PSET ...
lg PSET ...


getpatch PSET ...
gp PSET ...


searchlogs REGEX
sl REGEX


ckstart CFGFILE
cs CFGFILE


ckresume
cr


patch [+-]PSET ...
pa [+-]PSET ...


genpsets FROM TO
ge FROM TO