• Don't push to golden yet.

• Push that tag to golden. Note that this means your commit is not on golden/master yet!

• Push to golden/master. If you need to rebase, so be it. This may mean that the commit that ends up on golden/master is not the same SHA as the commit that you tested, but that doesn't matter. Any subsequent testing will still be based upon the tag you pushed earlier which is NOT updated.
• Job done.

If the bitmaps are not acceptable:

• Remove the new tag from golden.
• Repeat this process from the start (pushing the same "new" tag to golden as you did before).

Updating Release Baselines

Sometimes, during development, we'll make a commit that changes large numbers of diffs (for instance, a change that affects linewidths, or color rendition, or image positions). Previously we have just accepted such diffs, which means that when we do a release test at the end of the cycle, we have to contend with many hundreds of pages of diffs. The sheer volume of diffs can cause unrelated regressions to be swamped and missed.

We, therefore, adopt a new procedure of doing interim 'release tests' in the cluster, so as to update our baseline of what has been checked.

This page outlines the procedure to be followed:

Procedure

A typical course of events will be:

• Prepare a commit
• Cluster test the commit
• Spot that this changes multiple files, and that an update to the baselines is justified.
• Finish testing (examining bmpcmps as normal).
• Push to golden.

Now we need to update the baselines to that commit.

• Identify the previous baseline commit. This will either be a tag of the form ghostpdl-<version>rc<n>, or it'll be of the form ghostpdl-<version>-test-base-<m>.
• Tag your new commit as ghostpdl-<version>-test-base-<m+1> (where m is 0 if the previous tag was an rc one).
• Arrange a release test between the previous tag and your new one.
• Check the bitmaps.

If the bitmaps are acceptable, job done. If not:

• Remove the new tag.
• Either remove the commit (by git surgery, or by reverting) OR do a fix commit.
• Repeat this process from the start.

-- Robin Watts - 2020-03-19

Comments

<--/commentPlugin-->