Cluster work

This is a list of work that needs to be done on the cluster/notes on work underway/notes on completed work.

Windows cluster nodes

Windows nodes can now join the cluster. They must be running Windows 10, because we rely on the "bash on windows 10" feature to run the main clusterpull.sh script.

Sadly this cannot call windows binaries at the moment, so we need to use outbash (from https://github.com/xilun/cbwin) in order to make it work.

We call into git bash to work there. This involves lots of hairy rewriting of the job strings with regexps.

I can rewrite the pipe for md5sum to use a fifo, but something is going wrong with it, so I've resorted to just using a simple file now.

I have successfully run simple gs jobs through the system. I need to check pcl/xps/mupdf and pdfwrite jobs next.

Set up Windows Cluster Nodes as VMs.

Currently I'm just using my own windows machine as a node. The plan is to run virtual nodes on the cluster machines using libkvm and virsh. I've installed libkvm on every machine in the office so far, but not done more than that.

Weekly/Nightly/Release tests

I've set up a system for doing non-user, not git-triggered jobs. I've called these 'auto' jobs.

Essentially, I have an 'auto' directory (casper:/home/regression/cluster/auto) in which I can have other directories. By convention I put nightly tests in 'nightly' and weekly in 'weekly'. Each test goes in its own directory inside here, so for instance: auto/weekly/debug-Z@.

Every 'auto' job has a jobdef.txt file in each such directory, that contains information about which jobs to run. At it's simplest this contains details of a single job to run. For instance:

  • product options <extras=-Z@> make

The possible entries on this line are:

product
the products to build
options
the options (to the cluster system) to use
make
the make target to use
beforeMake
any command to run before make
afterMake
any command to run after make
filter
any filters to use
rev
the git SHA/branch/tag to test (e.g. origin/ghostpdl-9.20)

Leaving any of these blank will just use a sensible default.

For options:

w32
Use 32bit wordsize build
office
Only run on machines in the office (where the bandwidth between them is presumably fast and suitable for copying large bitmaps)
relaxTimeout
Allow for a larger timeout on such jobs
extended
Run the extended set of tests (i.e. all the tests/devices that marcos used to test on overnights etc)
ufst
Run with ufst enabled
luratech
Run with luratech enabled

To start an auto job, use the 'enqueueAuto.pl' script, for example: ./enqueueAuto.pl auto/nightly/gs

The regression user has a set of crontab entries to call enqueueAuto.pl appropriately to do the nightly/weekly tests.

Test between 2 given points.

In order to release test we need to compare a given SHA/tag/branch (the proposed release) with another one (the previous release).

The cluster is normally set up to test each run against a 'previous' result. The md5 results from the new job are typically then kept as the baseline for the next job of this type.

For release runs that's no good, so we introduce a new 'ref' entry to the jobdef.txt file. For example, we might have the following 2 lines in a jobdef.txt file.

  • product <gs> ref <ghostpdl-9.19> options <extended>
  • product <gs> ref <ghostpdl-9.19> rev <ghostpdl-9.20> options <extended>

The first line causes a job to run that builds the 'ref' revision, and runs the tests to generate the baseline md5 sums - this is a 'reference generating' run.

The second line causes a job to run that generates the 'rev' revision, and compares it against the former. In this run, because ref and rev are both specified, we do NOT update the stored baselines. Thus we can repeatedly run this second line until we're happy without needing to rerun the first one.

Both these jobs only generate md5sums, not bitmaps. The hope is that from release to release most of the files should be unchanged, so the number of files that need to be examined as bitmaps should be small.

Consider pushing scan build onto the cluster

For every cluster push, or weekly?

Consider pushing coverage onto the cluster

Find out how to run the coverage tests.

-- Robin Watts - 2017-02-15

See also:

ClusterNodes, ClusterStructure, ClusterHowTo

Comments

bmpcmp output - next/back links on each page of diffs

-- Chris Liddell - 2017-03-10

Edit | Attach | Watch | Print version | History: r5 < r4 < r3 < r2 < r1 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r5 - 2017-09-16 - RobinWatts
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright 2014 Artifex Software Inc