Difference: UpdatingReleaseBaselines (1 vs. 2)

Revision 22020-03-25 - RobinWatts

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Updating Release Baselines

Line: 16 to 16
 
  • Cluster test the commit
  • Spot that this changes multiple files, and that an update to the baselines is justified.
  • Finish testing (examining bmpcmps as normal).
Changed:
<
<
  • Push to golden.
>
>
  • Don't push to golden yet.
  Now we need to update the baselines to that commit.

  • Identify the previous baseline commit. This will either be a tag of the form ghostpdl-<version>rc<n>, or it'll be of the form ghostpdl-<version>-test-base-<m>.
  • Tag your new commit as ghostpdl-<version>-test-base-<m+1> (where m is 0 if the previous tag was an rc one).
Added:
>
>
  • Push that tag to golden. Note that this means your commit is not on golden/master yet!
 
  • Arrange a release test between the previous tag and your new one.
  • Check the bitmaps.
Changed:
<
<
If the bitmaps are acceptable, job done. If not:
>
>
If the bitmaps are acceptable:
 
Changed:
<
<
  • Remove the new tag.
  • Either remove the commit (by git surgery, or by reverting) OR do a fix commit.
  • Repeat this process from the start.
>
>
  • Push to golden/master. If you need to rebase, so be it. This may mean that the commit that ends up on golden/master is not the same SHA as the commit that you tested, but that doesn't matter. Any subsequent testing will still be based upon the tag you pushed earlier which is NOT updated.
  • Job done.

If the bitmaps are not acceptable:

  • Remove the new tag from golden.
  • Repeat this process from the start (pushing the same "new" tag to golden as you did before).
  -- Robin Watts - 2020-03-19

Revision 12020-03-19 - RobinWatts

Line: 1 to 1
Added:
>
>
META TOPICPARENT name="WebHome"

Updating Release Baselines

Sometimes, during development, we'll make a commit that changes large numbers of diffs (for instance, a change that affects linewidths, or color rendition, or image positions). Previously we have just accepted such diffs, which means that when we do a release test at the end of the cycle, we have to contend with many hundreds of pages of diffs. The sheer volume of diffs can cause unrelated regressions to be swamped and missed.

We, therefore, adopt a new procedure of doing interim 'release tests' in the cluster, so as to update our baseline of what has been checked.

This page outlines the procedure to be followed:

Procedure

A typical course of events will be:

  • Prepare a commit
  • Cluster test the commit
  • Spot that this changes multiple files, and that an update to the baselines is justified.
  • Finish testing (examining bmpcmps as normal).
  • Push to golden.

Now we need to update the baselines to that commit.

  • Identify the previous baseline commit. This will either be a tag of the form ghostpdl-<version>rc<n>, or it'll be of the form ghostpdl-<version>-test-base-<m>.
  • Tag your new commit as ghostpdl-<version>-test-base-<m+1> (where m is 0 if the previous tag was an rc one).
  • Arrange a release test between the previous tag and your new one.
  • Check the bitmaps.

If the bitmaps are acceptable, job done. If not:

  • Remove the new tag.
  • Either remove the commit (by git surgery, or by reverting) OR do a fix commit.
  • Repeat this process from the start.

-- Robin Watts - 2020-03-19

Comments

<--/commentPlugin-->
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright 2014 Artifex Software Inc