Difference: RunningClusterJobs (1 vs. 8)

Revision 82018-06-26 - RobinWatts

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Running Cluster Jobs

Line: 73 to 73
 
-w
Set the "window" for the bmpcmp (default=1, sensible other values are 3, 5, 7 etc).
Changed:
<
<
-t
Set the "threshold" for the bmpcmp (default=1, value should be < 255, sensible values 8,16,32 or so).
>
>
-t
Set the "threshold" for the bmpcmp (default=1, value should be < 255, sensible values 8,16,33 or so). Do not use 32. 32 is a number with special meaning to the cluster, and it (and you) will be confused by what happens. Your job will go wrong. You will blame Robin. Robin will blame Marcos. Basically, no good can come of it.
  So, if a job produced 2000 diffs, you can run:

Revision 72016-12-20 - RobinWatts

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Running Cluster Jobs

The clusterpush.pl script allows you to run a cluster regression on code that has not been committed to git.

Changed:
<
<
Usage: clusterpush.pl [product] [user] [options]
>
>
Usage: clusterpush.pl [product] [user] [options]
 
Deleted:
<
<
 
Product

Indicates which products are to be tested:

Changed:
<
<
gs
pcl
xps
mupdf
mujstest 
>
>
  • gs
  • pcl
  • xps
  • mupdf
  • mujstest
  You can specify gs, pcl, and xps by themselves or in combation (i.e. clusterpush.pl gs pcl xps). mupdf and mujstest must be specified by themselves.
Changed:
<
<
If you do not specify a product gs pcl xps will be tested if you are in a ghostpdl directory and mupdf if in mupdf
>
>
If you do not specify a product gs pcl xps will be tested if you are in a ghostpdl directory and mupdf if in mupdf.
  Product can also be 'bmpcmp' (see below).
Changed:
<
<
user
>
>

user

  user is your cluster user name, if you don't specify one, clusterpush.pl uses, in order of preference, the value of one of the following environment variables: CLUSTER_USER, USER, and USERNAME This works as expected under Linux, Mac OS X, Msys, and probably cygwin.
Deleted:
<
<

options

 
Changed:
<
<
bmpcmp
indicates that a bitmap comparison of your latest cluster run to that of the last git commit should be performed (see below).
>
>

options

 
Changed:
<
<
lowres
hires
run only lowres (72 and 75 dpi) or hires (200 and 300 dpi) tests, by default both are run.
>
>
bmpcmp
Indicates that a bitmap comparison of your latest cluster run to that of the last git commit should be performed (see below).
 
Changed:
<
<

relaxtimeout
relaxes the timeout for each job, from 5 minutes to 30 minutes. Note that the total time for the cluster job is still limited to 2 hours.
>
>
lowres
Run only lowres (72 and 75 dpi) tests (by default both lowres and hires are run)
 
Changed:
<
<
filter=
limits jobs to those matching the specified parameter, for example filter=pgmraw will only test pgmraw files.
>
>
hires
Run only hires (200 and 300 dpi) tests (by default both lowres and hires are run)
 
Changed:
<
<
This option can be specified multiple times, in which case the filters are or'd together; i.e. filter=pgmraw filter=ppmraw will process both pgmraw and ppmraw files.
>
>
relaxtimeout
Relax the timeout for each job, from 5 minutes to 30 minutes. Note that the total time for the cluster job is still limited to 2 hours.
 
Added:
>
>
filter=...
Limit jobs to those matching the specified parameter, for example filter=pgmraw will only test pgmraw files.
This option can be specified multiple times, in which case the filters are or'd together; i.e. filter=pgmraw filter=ppmraw will process both pgmraw and ppmraw files.
 Separating multiple paramters seperated by a comma ands the filters together; i.e. filter=300,pbmraw processed only 300 dpi pbmraw files.
Changed:
<
<
extras=
add extra command line options to a cluster job, i.e. extras=-dEPSCROP.
>
>
extras=...
Add extra command line options to a cluster job, i.e. extras=-dEPSCROP.
 This option can be specified multiple times.
Added:
>
>
Some example invocations:

  • clusterpush.pl
  • clusterpush.pl gs pcl
  • clusterpush.pl filter=ppmraw.300.0
  • clusterpush.pl filter=ppmraw.300.0 filter=pgmraw.300.0
  • clusterpush.pl extras=-dEPSCROP
  • clusterpush.pl filter=png.300.0 extras=-dDownScaleFactor=3 extras=-dDownScaleETS=1
 

Bitmap Comparison (bmpcmp):

To use the feature run clusterpush.pl with the option bmpcmp

Line: 50 to 59
 
toolbin/localcluster/clusterpush.pl bmpcmp 
 
Changed:
<
<
This will queue a job that uses your most recent none-bmpcmp clusterpush job and generates bitmaps of the differences for the first 1000 files listed in the report. It puts these files on the internets
>
>
This will queue a job that tests only the files that showed as differences in your most recent non-bmpcmp clusterpush job. It will generate bitmaps of the differences for (by default) the first 1000 differences listed in the report. These files can be found by following the bmpcmp link to the right of the your username on the regression dashboard (or at http://www.ghostscript.com/~regression/$USER).

When the job is complete you will get an email telling you that it's done. Since some of the test files are customer confidential this directory is password protected.

You can add bmpcmp to a normal clusterpush (i.e. 'clusterpush gs bmpcmp' or 'clusterpush gs pcl xps bmpcmp'). This will queue both jobs in the correct sequence.

There are some options you can use to control the jobs:

-c
Test N files rather than 1000. (Still limited to 1000).

-s
Rather than starting with the first difference listed in the previous non-bmpcmps clusterpush results, start with the Nth difference.
 
Changed:
<
<
http://www.ghostscript.com/~regression/$USER
>
>
-w
Set the "window" for the bmpcmp (default=1, sensible other values are 3, 5, 7 etc).
 
Changed:
<
<
and then sends you an email telling you that it's done. Since some of the test files are customer confidential this directory is password protected.
>
>
-t
Set the "threshold" for the bmpcmp (default=1, value should be < 255, sensible values 8,16,32 or so).
 
Changed:
<
<

You can add bmpcmp to a normal clusterpush (i.e. 'clusterpush gs bmpcmp' or 'clusterpush gs pcl xps bmpcmp'). This will queueu both jobs in the correct sequence.

Aborting a cluster job:
>
>
So, if a job produced 2000 diffs, you can run:

  • clusterpush.pl bmpcmp

to see the first 1000 results, then once you've checked them:

  • clusterpush.pl bmpcmp -s1000

will test the second 1000 jobs.

By default bmpcmp is an exact bitmap equivalency test. By using -w and -t, you can make it a fuzzy test. -w allows for slight movements in features. -t allows for slight color changes.

Aborting a cluster job:

  If you want to abort a queued or currently running cluster job you canspecify abort as the clusterpush.pl option:
Changed:
<
<
clusterpush.pl abort
 
>
>
  • clusterpush.pl abort
 

misc.

Changed:
<
<

The order of the options is not important.

You need to run clusterpush.pl from the top level directory (the same
directory from where you normally run 'make'). Clusterpush prints an
error if you try to run it from a different directory.

Results will be sent via email after the run is completed, which takes
about 10 minutes, presuming the cluster isn't busy running other jobs.
Your code is compared to the most recent baseline, so for meaningful
results you will want to sync your code with head if recent changes to
head have resulted in output differences.

The first time you run clusterpush.pl it will take a long time to rsync
your source to the master machine and then from the master to the
nodes. After that things will go much quicker (unless, like me,
you accidently leave giant .ppm files in the current directory).

>
>
The order of the options is not important.

You need to run clusterpush.pl from the top level directory (the same directory from where you normally run 'make'). Clusterpush prints an error if you try to run it from a different directory.

Results will be sent via email after the run is completed, which takes about 10 minutes, presuming the cluster isn't busy running other jobs. User jobs will take priority over non-user jobs, so even if there are many non-user jobs queued, you only ever need to wait for the current one to finish before the user queue will be processed.

Your code is compared to the most recent baseline, so for meaningful results you will want to sync your code with head if recent changes to head have resulted in output differences.

The first time you run clusterpush.pl it will take a long time to rsync your source to the master machine and then from the master to the nodes. After that things will go much quicker (unless, like me,you accidently leave giant .ppm files in the current directory).

  -- Marcos Woehrmann - 2016-06-22

Revision 62016-06-22 - MarcosWoehrmann

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Running Cluster Jobs

Line: 17 to 17
 mupdf mujstest
Changed:
<
<
You can specify gs, pcl, and xps by themselves or in combation (i.e. clusterpush.pl gs pcl xps).
>
>
You can specify gs, pcl, and xps by themselves or in combation (i.e. clusterpush.pl gs pcl xps). mupdf and mujstest must be specified by themselves.
  If you do not specify a product gs pcl xps will be tested if you are in a ghostpdl directory and mupdf if in mupdf
Added:
>
>
Product can also be 'bmpcmp' (see below).
 
user
Changed:
<
<
user is your cluster user name, if you don't specify one, clusterpush.pl
uses, in order of preference, the value of one of the following
environment variables: CLUSTER_USER, USER, and USERNAME This works as expected under
Linux, Mac OS X, Msys, and probably cygwin.
>
>
user is your cluster user name, if you don't specify one, clusterpush.pl uses, in order of preference, the value of one of the following environment variables: CLUSTER_USER, USER, and USERNAME This works as expected under Linux, Mac OS X, Msys, and probably cygwin.
 

options

Changed:
<
<
bmpcmp

indicates that a bitmap comparison of your latest cluster run to that of the last git commit should be performed (see below).

lowres
hires

>
>
bmpcmp
indicates that a bitmap comparison of your latest cluster run to that of the last git commit should be performed (see below).
 
Changed:
<
<
run only lowres (72 and 75 dpi) or hires (200 and 300 dpi) tests, by default both are run.
>
>
lowres
hires
run only lowres (72 and 75 dpi) or hires (200 and 300 dpi) tests, by default both are run.
 
Changed:
<
<

relaxtimeout
>
>

relaxtimeout
relaxes the timeout for each job, from 5 minutes to 30 minutes. Note that the total time for the cluster job is still limited to 2 hours.
 
Changed:
<
<
relaxes the timeout for each job, from 5 minutes to 30 minutes. Note that the total time for the cluster job is still limited to 2 hours.

filter=

limits jobs to those matching the specified parameter, for example filter=pgmraw will only test pgmraw files.

>
>
filter=
limits jobs to those matching the specified parameter, for example filter=pgmraw will only test pgmraw files.
  This option can be specified multiple times, in which case the filters are or'd together; i.e. filter=pgmraw filter=ppmraw will process both pgmraw and ppmraw files.

Separating multiple paramters seperated by a comma ands the filters together; i.e. filter=300,pbmraw processed only 300 dpi pbmraw files.

Changed:
<
<
extras=

add extra command line options to a cluster job, i.e. extras=-dEPSCROP.

>
>
extras=
add extra command line options to a cluster job, i.e. extras=-dEPSCROP.
  This option can be specified multiple times.
Line: 58 to 50
 
toolbin/localcluster/clusterpush.pl bmpcmp 
 
Changed:
<
<
This will queue a job that uses your most recent none-bmpcmp clusterpush job and generates bitmaps of the differences for the first 1000 files listed in the report. It puts these files on the internets in the location
>
>
This will queue a job that uses your most recent none-bmpcmp clusterpush job and generates bitmaps of the differences for the first 1000 files listed in the report. It puts these files on the internets
  http://www.ghostscript.com/~regression/$USER
Changed:
<
<


and then sends you an email telling you that it's done. Since some of the test files are customer confidential this directory is password protected.
>
>
and then sends you an email telling you that it's done. Since some of the test files are customer confidential this directory is password protected.
 
You can add bmpcmp to a normal clusterpush (i.e. 'clusterpush gs bmpcmp' or 'clusterpush gs pcl xps bmpcmp'). This will queueu both jobs in the correct sequence.

Aborting a cluster job:
Changed:
<
<
If you want to abort a queued or currently running cluster job you can
specify abort as the clusterpush.pl option:
>
>
If you want to abort a queued or currently running cluster job you canspecify abort as the clusterpush.pl option:
 
clusterpush.pl abort
 

misc.

Changed:
<
<

The order of the options is not important.

You need to run clusterpush.pl from the top level directory (the same
directory from where you normally run 'make'). Clusterpush prints and
error if you try to run it from a different directory.

Results will be sent via email after the run is completed, which takes
about 10 minutes, presuming the cluster isn't busy. Your code is compared
to the most recent baseline, so for meaningful results you will want
to sync your code with head if recent changes to head have resulted in
regression differences.

The first time you run clusterpush.pl it will take a long time
to rsync your source to the master machine and then from the master to
the nodes. After that things will go much quicker (unless, like me,
you accidently leave giant .ppm files in the current directory).

Futhermore the first time clusterpush.pl is run each cluster node has to
transfer your source tree from the cluster master; some of the nodes have
relatively slow internet connections and may time out. The transfers
will complete, so the next time a clusterpush.pl is run everything will
proceed normally.
>
>

The order of the options is not important.

You need to run clusterpush.pl from the top level directory (the same
directory from where you normally run 'make'). Clusterpush prints an
error if you try to run it from a different directory.

Results will be sent via email after the run is completed, which takes
about 10 minutes, presuming the cluster isn't busy running other jobs.
Your code is compared to the most recent baseline, so for meaningful
results you will want to sync your code with head if recent changes to
head have resulted in output differences.

The first time you run clusterpush.pl it will take a long time to rsync
your source to the master machine and then from the master to the
nodes. After that things will go much quicker (unless, like me,
you accidently leave giant .ppm files in the current directory).

 
Changed:
<
<
-- Marcos Woehrmann - 2016-06-16
>
>
-- Marcos Woehrmann - 2016-06-22
 

Comments

Revision 52016-06-17 - MarcosWoehrmann

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Running Cluster Jobs

Line: 23 to 25
 user is your cluster user name, if you don't specify one, clusterpush.pl
uses, in order of preference, the value of one of the following
environment variables: CLUSTER_USER, USER, and USERNAME This works as expected under
Linux, Mac OS X, Msys, and probably cygwin.

options

Changed:
<
<
bmpcmp
>
>
bmpcmp
  indicates that a bitmap comparison of your latest cluster run to that of the last git commit should be performed (see below).
Changed:
<
<
lowres
hires
>
>
lowres
hires
  run only lowres (72 and 75 dpi) or hires (200 and 300 dpi) tests, by default both are run.
Changed:
<
<

relaxtimeout
>
>

relaxtimeout
  relaxes the timeout for each job, from 5 minutes to 30 minutes. Note that the total time for the cluster job is still limited to 2 hours.
Changed:
<
<
filter=
>
>
filter=
  limits jobs to those matching the specified parameter, for example filter=pgmraw will only test pgmraw files.
Line: 43 to 45
  Separating multiple paramters seperated by a comma ands the filters together; i.e. filter=300,pbmraw processed only 300 dpi pbmraw files.
Changed:
<
<
extras=
>
>
extras=

add extra command line options to a cluster job, i.e. extras=-dEPSCROP.

This option can be specified multiple times.

 

Bitmap Comparison (bmpcmp):

Revision 42016-06-16 - MarcosWoehrmann

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Running Cluster Jobs

Line: 29 to 27
  indicates that a bitmap comparison of your latest cluster run to that of the last git commit should be performed (see below).
Changed:
<
<
lowres:
hires:
>
>
lowres
hires
  run only lowres (72 and 75 dpi) or hires (200 and 300 dpi) tests, by default both are run.
Changed:
<
<

relaxtimeout:
>
>

relaxtimeout
  relaxes the timeout for each job, from 5 minutes to 30 minutes. Note that the total time for the cluster job is still limited to 2 hours.
Added:
>
>
filter=

limits jobs to those matching the specified parameter, for example filter=pgmraw will only test pgmraw files.

This option can be specified multiple times, in which case the filters are or'd together; i.e. filter=pgmraw filter=ppmraw will process both pgmraw and ppmraw files.

Separating multiple paramters seperated by a comma ands the filters together; i.e. filter=300,pbmraw processed only 300 dpi pbmraw files.

extras=

 

Bitmap Comparison (bmpcmp):

To use the feature run clusterpush.pl with the option bmpcmp

Revision 32016-06-16 - MarcosWoehrmann

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Running Cluster Jobs

Changed:
<
<
The clusterpush.pl script allows you to run a cluster regression on code that has not been committed to git. The current
Usage: clusterpush.pl [product] [options] [user]
>
>
The clusterpush.pl script allows you to run a cluster regression on code that has not been committed to git.
 
Changed:
<
<
Where product is one or more of gs, pcl, xps or mupdf or mujstest. If you do not specify a product the product(s) tested will be based on what directory you are 'cd'ed to when the clusterpush.pl script is run.
>
>
Usage: clusterpush.pl [product] [user] [options]
 
Changed:
<
<
options available are:
>
>
Product

Indicates which products are to be tested:

gs
pcl
xps
mupdf
mujstest 

You can specify gs, pcl, and xps by themselves or in combation (i.e. clusterpush.pl gs pcl xps).

If you do not specify a product gs pcl xps will be tested if you are in a ghostpdl directory and mupdf if in mupdf

user

user is your cluster user name, if you don't specify one, clusterpush.pl
uses, in order of preference, the value of one of the following
environment variables: CLUSTER_USER, USER, and USERNAME This works as expected under
Linux, Mac OS X, Msys, and probably cygwin.

options

bmpcmp

 
Changed:
<
<
bmpcmp: indicates that a bitmap comparison of your latest cluster run to that of the last git commit should be performed.
>
>
indicates that a bitmap comparison of your latest cluster run to that of the last git commit should be performed (see below).
 
Changed:
<
<
lowres:
hires: indicates to only perform a low-resolution orhigh-resolution cluster run; by default both are performed.

relaxtimeout: relaxes the timeout for each job, from 5 minutes to 30 minutes. Note that the total time for the cluster job is still limited to 2 hours.

user is your cluster user name, if you don't specify one, clusterpush.pl
uses, in order of preference, the value of one of the following
environment variables: CLUSTER_USER, USER, and USERNAME This works under
Linux, Mac OS X, Msys, and probably cygwin.

The order of the options is not important but some of the options
are mutually exclusive.

You need to run clusterpush.pl from the top level directory (the same
directory from where you normally run 'make'). Clusterpush prints and
error if you try to run it from a different directory.

Results will be sent via email after the run is completed, which takes
about 10 minutes, presuming the cluster isn't busy. Your code is compared
to the most recent baseline, so for meaningful results you will want
to sync your code with head if recent changes to head have resulted in
regression differences.

The first time you run clusterpush.pl it will take a long time
to rsync your source to the master machine and then from the master to
the nodes. After that things will go much quicker (unless, like me,
you accidently leave giant .ppm files in the current directory).

Futhermore the first time clusterpush.pl is run each cluster node has to
transfer your source tree from the cluster master; some of the nodes have
relatively slow internet connections and may time out. The transfers
will complete, so the next time a clusterpush.pl is run everything will
proceed normally.
Bitmap Comparison (bmpcmp):
>
>
lowres:
hires:

run only lowres (72 and 75 dpi) or hires (200 and 300 dpi) tests, by default both are run.


relaxtimeout:

relaxes the timeout for each job, from 5 minutes to 30 minutes. Note that the total time for the cluster job is still limited to 2 hours.

Bitmap Comparison (bmpcmp):

  To use the feature run clusterpush.pl with the option bmpcmp

toolbin/localcluster/clusterpush.pl bmpcmp 
 
Changed:
<
<
This will queue a job that uses your most recent none-bmpcmp clusterpush job and generates bitmaps of the differences for the first 1000 files listed in the report. It puts these files on the internets in the location
>
>
This will queue a job that uses your most recent none-bmpcmp clusterpush job and generates bitmaps of the differences for the first 1000 files listed in the report. It puts these files on the internets in the location
  http://www.ghostscript.com/~regression/$USER
Changed:
<
<


and then sends you an email telling you that it's done. Since some of the test files are customer confidential this directory is password protected.
>
>


and then sends you an email telling you that it's done. Since some of the test files are customer confidential this directory is password protected.
 
You can add bmpcmp to a normal clusterpush (i.e. 'clusterpush gs bmpcmp' or 'clusterpush gs pcl xps bmpcmp'). This will queueu both jobs in the correct sequence.

Aborting a cluster job:
Line: 32 to 54
 
Aborting a cluster job:

If you want to abort a queued or currently running cluster job you can
specify abort as the clusterpush.pl option:

Added:
>
>
 
clusterpush.pl abort
 
Added:
>
>

misc.


The order of the options is not important.

You need to run clusterpush.pl from the top level directory (the same
directory from where you normally run 'make'). Clusterpush prints and
error if you try to run it from a different directory.

Results will be sent via email after the run is completed, which takes
about 10 minutes, presuming the cluster isn't busy. Your code is compared
to the most recent baseline, so for meaningful results you will want
to sync your code with head if recent changes to head have resulted in
regression differences.

The first time you run clusterpush.pl it will take a long time
to rsync your source to the master machine and then from the master to
the nodes. After that things will go much quicker (unless, like me,
you accidently leave giant .ppm files in the current directory).

Futhermore the first time clusterpush.pl is run each cluster node has to
transfer your source tree from the cluster master; some of the nodes have
relatively slow internet connections and may time out. The transfers
will complete, so the next time a clusterpush.pl is run everything will
proceed normally.

  -- Marcos Woehrmann - 2016-06-16

Revision 22016-06-16 - MarcosWoehrmann

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Running Cluster Jobs

Changed:
<
<
The clusterpush.pl script allows you to run a cluster regression usingcode that has not been committed to git.
Usage: clusterpush.pl [product] [options] [user]
Where product is one or more of gs, pcl, xps or mupdf or mujstest.
If you do not specify a product the product(s) tested will be cased onwhat directory you are 'cd'ed to when the clusterpush.pl script is run.options available are:
bmpcmp: indicates that a bitmap comparison of your latest cluster run to that of the last git commit should be performed.

lowres:
hires: indicates to only perform a low-resolution orhigh-resolution cluster run; by default both are performed.

relaxtimeout: relaxes the timeout for each job, from 5 minutes to 30 minutes. Note that the total time for the cluster job is still limited to 2 hours.

user is your cluster user name, if you don't specify one, clusterpush.pl
uses, in order of preference, the value of one of the following
environment variables: CLUSTER_USER, USER, and USERNAME This works under
Linux, Mac OS X, Msys, and probably cygwin.

The order of the options is not important but some of the options
are mutually exclusive.

You need to run clusterpush.pl from the top level directory (the same
directory from where you normally run 'make'). Clusterpush prints and
error if you try to run it from a different directory.

Results will be sent via email after the run is completed, which takes
about 10 minutes, presuming the cluster isn't busy. Your code is compared
to the most recent baseline, so for meaningful results you will want
to sync your code with head if recent changes to head have resulted in
regression differences.

The first time you run clusterpush.pl it will take a long time
to rsync your source to the master machine and then from the master to
the nodes. After that things will go much quicker (unless, like me,
you accidently leave giant .ppm files in the current directory).

Futhermore the first time clusterpush.pl is run each cluster node has to
transfer your source tree from the cluster master; some of the nodes have
relatively slow internet connections and may time out. The transfers
will complete, so the next time a clusterpush.pl is run everything will
proceed normally.

bmpcmp mode:
To use the feature run clusterpush.pl with the option bmpcmp (i.e.
"toolbin/localcluster/clusterpush.pl bmpcmp"). This will queue a job
that looks at the most recent none-bmpcmp clusterpush job you ran and
>
>
The clusterpush.pl script allows you to run a cluster regression on code that has not been committed to git. The current
Usage: clusterpush.pl [product] [options] [user]

Where product is one or more of gs, pcl, xps or mupdf or mujstest. If you do not specify a product the product(s) tested will be based on what directory you are 'cd'ed to when the clusterpush.pl script is run.

options available are:

bmpcmp: indicates that a bitmap comparison of your latest cluster run to that of the last git commit should be performed.

lowres:
hires: indicates to only perform a low-resolution orhigh-resolution cluster run; by default both are performed.

relaxtimeout: relaxes the timeout for each job, from 5 minutes to 30 minutes. Note that the total time for the cluster job is still limited to 2 hours.

user is your cluster user name, if you don't specify one, clusterpush.pl
uses, in order of preference, the value of one of the following
environment variables: CLUSTER_USER, USER, and USERNAME This works under
Linux, Mac OS X, Msys, and probably cygwin.

The order of the options is not important but some of the options
are mutually exclusive.

You need to run clusterpush.pl from the top level directory (the same
directory from where you normally run 'make'). Clusterpush prints and
error if you try to run it from a different directory.

Results will be sent via email after the run is completed, which takes
about 10 minutes, presuming the cluster isn't busy. Your code is compared
to the most recent baseline, so for meaningful results you will want
to sync your code with head if recent changes to head have resulted in
regression differences.

The first time you run clusterpush.pl it will take a long time
to rsync your source to the master machine and then from the master to
the nodes. After that things will go much quicker (unless, like me,
you accidently leave giant .ppm files in the current directory).

Futhermore the first time clusterpush.pl is run each cluster node has to
transfer your source tree from the cluster master; some of the nodes have
relatively slow internet connections and may time out. The transfers
will complete, so the next time a clusterpush.pl is run everything will
proceed normally.

Bitmap Comparison (bmpcmp):

To use the feature run clusterpush.pl with the option bmpcmp

toolbin/localcluster/clusterpush.pl bmpcmp 
 

This will queue a job that uses your most recent none-bmpcmp clusterpush job and

 generates bitmaps of the differences for the first 1000 files listed in the report. It puts these files on the internets in the location
Deleted:
<
<
<http://www.ghostscript.com/~regression/$USER> and then sends you an email telling you that it's done. Since some of the test files are customer confidential this directory is password protected.

You can add bmpcmp to a normal clusterpush (i.e. 'clusterpush gs bmpcmp' or 'clusterpush gs pcl xps ls bmpcmp'). This will queueu both jobs in the correct sequence. abort mode:

 
Changed:
<
<
If you want to abort a queued or currently running cluster job you can specify abort to the clusterpush.pl option: 'clusterpush.pl abort'.
>
>
http://www.ghostscript.com/~regression/$USER



and then sends you an email telling you that it's done. Since some of the test files are customer confidential this directory is password protected.


You can add bmpcmp to a normal clusterpush (i.e. 'clusterpush gs bmpcmp' or 'clusterpush gs pcl xps bmpcmp'). This will queueu both jobs in the correct sequence.

Aborting a cluster job:

If you want to abort a queued or currently running cluster job you can
specify abort as the clusterpush.pl option:

clusterpush.pl abort
 
 -- Marcos Woehrmann - 2016-06-16

Comments

Revision 12016-06-16 - MarcosWoehrmann

Line: 1 to 1
Added:
>
>
META TOPICPARENT name="WebHome"

Running Cluster Jobs

The clusterpush.pl script allows you to run a cluster regression usingcode that has not been committed to git.
Usage: clusterpush.pl [product] [options] [user]
Where product is one or more of gs, pcl, xps or mupdf or mujstest.
If you do not specify a product the product(s) tested will be cased onwhat directory you are 'cd'ed to when the clusterpush.pl script is run.options available are:
bmpcmp: indicates that a bitmap comparison of your latest cluster run to that of the last git commit should be performed.

lowres:
hires: indicates to only perform a low-resolution orhigh-resolution cluster run; by default both are performed.

relaxtimeout: relaxes the timeout for each job, from 5 minutes to 30 minutes. Note that the total time for the cluster job is still limited to 2 hours.

user is your cluster user name, if you don't specify one, clusterpush.pl
uses, in order of preference, the value of one of the following
environment variables: CLUSTER_USER, USER, and USERNAME This works under
Linux, Mac OS X, Msys, and probably cygwin.

The order of the options is not important but some of the options
are mutually exclusive.

You need to run clusterpush.pl from the top level directory (the same
directory from where you normally run 'make'). Clusterpush prints and
error if you try to run it from a different directory.

Results will be sent via email after the run is completed, which takes
about 10 minutes, presuming the cluster isn't busy. Your code is compared
to the most recent baseline, so for meaningful results you will want
to sync your code with head if recent changes to head have resulted in
regression differences.

The first time you run clusterpush.pl it will take a long time
to rsync your source to the master machine and then from the master to
the nodes. After that things will go much quicker (unless, like me,
you accidently leave giant .ppm files in the current directory).

Futhermore the first time clusterpush.pl is run each cluster node has to
transfer your source tree from the cluster master; some of the nodes have
relatively slow internet connections and may time out. The transfers
will complete, so the next time a clusterpush.pl is run everything will
proceed normally.

bmpcmp mode:
To use the feature run clusterpush.pl with the option bmpcmp (i.e.
"toolbin/localcluster/clusterpush.pl bmpcmp"). This will queue a job
that looks at the most recent none-bmpcmp clusterpush job you ran and
generates bitmaps of the differences for the first 1000 files listed
in the report. It puts these files on the internets in the location
<http://www.ghostscript.com/~regression/$USER> and then sends you
an email telling you that it's done. Since some of the test files
are customer confidential this directory is password protected.

You can add bmpcmp to a normal clusterpush (i.e. 'clusterpush gs bmpcmp'
or 'clusterpush gs pcl xps ls bmpcmp'). This will queueu both jobs in
the correct sequence.
abort mode:

If you want to abort a queued or currently running cluster job you can
specify abort to the clusterpush.pl option: 'clusterpush.pl abort'.
-- Marcos Woehrmann - 2016-06-16

Comments


<--/commentPlugin-->
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright 2014 Artifex Software Inc