Updating Release Baselines

Sometimes, during development, we'll make a commit that changes large numbers of diffs (for instance, a change that affects linewidths, or color rendition, or image positions). Previously we have just accepted such diffs, which means that when we do a release test at the end of the cycle, we have to contend with many hundreds of pages of diffs. The sheer volume of diffs can cause unrelated regressions to be swamped and missed.

We, therefore, adopt a new procedure of doing interim 'release tests' in the cluster, so as to update our baseline of what has been checked.

This page outlines the procedure to be followed:

Procedure

A typical course of events will be:

  • Prepare a commit
  • Cluster test the commit
  • Spot that this changes multiple files, and that an update to the baselines is justified.
  • Finish testing (examining bmpcmps as normal).
  • Push to golden.

Now we need to update the baselines to that commit.

  • Identify the previous baseline commit. This will either be a tag of the form ghostpdl-<version>rc<n>, or it'll be of the form ghostpdl-<version>-test-base-<m>.
  • Tag your new commit as ghostpdl-<version>-test-base-<m+1> (where m is 0 if the previous tag was an rc one).
  • Arrange a release test between the previous tag and your new one.
  • Check the bitmaps.

If the bitmaps are acceptable, job done. If not:

  • Remove the new tag.
  • Either remove the commit (by git surgery, or by reverting) OR do a fix commit.
  • Repeat this process from the start.

-- Robin Watts - 2020-03-19

Comments

Edit | Attach | Watch | Print version | History: r4 < r3 < r2 < r1 | Backlinks | Raw View | Raw edit | More topic actions...
Topic revision: r1 - 2020-03-19 - RobinWatts
 
  • Edit
  • Attach
This site is powered by the TWiki collaboration platform Powered by PerlCopyright 2014 Artifex Software Inc