Building Memento builds.

GS on Unix

make memento (or {gs,gcpl6,gxps}memento etc). Binaries go into membin.

GS on Windows

Select the "Memento" configuration from the drop down and build as normal. Binaries go in membin

MuPDF on Unix

make build=memento. Binaries go into build/memento/.

MuPDF on Windows

Select the "Memento" configuration from the drop down and build as normal. Binaries go in platform/win32/Memento/.

New projects

Build the debug version of your project with "-DMEMENTO" and ensure that memento.h is included in every single C file built.

Running Memento builds.

Just run builds exactly as normal, and they will run as normal but (at about half speed).

When the program exits, you will get a list of all the leaked blocks, together with some helpful statistics. You can get the same list of allocated blocks at any point during program execution by calling Memento_listBlocks();

How does Memento work?

Really simply. Every call to malloc/free/realloc etc is redirected through Memento.

Every call to malloc/free/realloc counts as an 'allocation event'.

On each event Memento increments a counter. Every block is tagged with the current counter on allocation. Every so often during program execution, the heap is checked for consistency. By default this happens after 1024 events, then after 2048 events, then after 4096 events, etc. This can be changed by setting MEMENTO_PARANOIA or at runtime by using: Memento_setParanoia(int level).

0 turns off such checking, 1 sets checking to happen on every event,

any positive number n sets checking to happen once every n events, and any negative number n sets checking to happen after -n events, then after -2n events etc.

The default paranoia level is therefore -1024.

Memento keeps blocks around for a while after they have been freed, and checks them as part of these heap checks to see if they have been written to (or are freed twice etc).

A given heap block can be checked for consistency (it's 'pre' and 'post' guard blocks are checked to see if they have been written to) by calling Memento_checkBlock(void *blockAddress);

A check of all the memory can be triggered by calling Memento_check(); (or Memento_checkAllMemory(); if you'd like it to be quieter).

A good place to breakpoint is Memento_breakpoint, as this will then trigger your debugger if an error is detected. This is done automatically for debug windows builds.

If a block is found to be corrupt, information will be printed to the console, including the address of the block, the size of the block, the type of corruption, the number of the block and the event on which it last passed a check for correctness.

If you rerun, and call Memento_paranoidAt(int event); with this number the the code will wait until it reaches that event and then start checking the heap after every allocation event. Assuming it is a deterministic failure, you should then find out where in your program the error is occurring (between event x-1 and event x).

Then you can rerun the program again, and call Memento_breakAt(int event); and the program will call Memento_Breakpoint() when event x is reached, enabling you to step through.

Memento_find(address) will tell you what block (if any) the given address is in.

An example:

Suppose we have a gs invocation that crashes with memory corruption.

  • Build with -DMEMENTO.
  • In your debugger put a breakpoint on Memento_breakpoint.
  • Run the program. It will stop when Memento initialises (on the first allocation in the program).
  • Execute Memento_setParanoia(1); (In VS use Ctrl-Alt-Q). (Note #1)
  • Continue execution.
  • It will detect the memory corruption on the next allocation event after it happens, and stop in Memento_breakpoint. The console should show something like:
    • Freed blocks:
    • 0x172e610(size=288,num=1415) index 256 (0x172e710) onwards corrupted
    • Block last checked OK at allocation 1457. Now 1458.
  • This means that the block became corrupted between allocation 1457 and 1458 - so if we rerun and stop the program at 1457, we can then step through, (possibly with a data breakpoint at the block address + 256) and see when it occurs.
  • So restart the program from the beginning. When we stop after initialisation execute Memento_breakAt(1457); (and maybe Memento_setParanoia(1), or Memento_setParanoidAt(1457))
  • (In VS you can 'execute' a function while stopped at a breakpoint by hitting Alt-Ctrl-Q to get the QuickWatch window into which you type the function call (for instance "Memento_breakAt(1457)" and hit return).
  • (In gdb you do 'call Memento_breakAt(1457)').
  • Continue execution until we hit Memento_breakpoint.
  • Now you can step through and watch the memory corruption happen.

*Note #1: Using Memento_setParanoia(1) can cause your program to run very slowly. You may instead choose to use Memento_setParanoia(100) (or some other figure). This will only exhaustively check memory on every 100th allocation event. This trades speed for the size of the average allocation event range in which detection of memory corruption occurs. You may (for example) choose to run once checking every 100 allocations and discover that the corruption happens between events X and X+100. You can then rerun using Memento_paranoidAt(X), and it'll only start exhaustively checking when it reaches X.

* More than one memory allocator?

If you have more than one memory allocator in the system (like for instance the ghostscript chunk allocator, that builds on top of the standard malloc and returns chunks itself), then there are some things to note:

  • If the secondary allocator gets its underlying blocks from calling malloc, then those will be checked by Memento, but 'subblocks' that are returned to the secondary allocator will not. There is currently no way to fix this other than trying to bypass the secondary allocator. One way I have found to do this with the chunk allocator is to tweak its idea of a 'large block' so that it puts every allocation in its own chunk. Clearly this negates the point of having a secondary allocator, and is therefore not recommended for general use.

  • Again, if the secondary allocator gets its underlying blocks from calling malloc (and hence Memento) leak detection should still work (but whole blocks will be detected rather than subblocks).

  • If on every allocation attempt the secondary allocator calls into Memento_failThisEvent(), and fails the allocation if it returns true then more useful features can be used; firstly memory squeezing will work, and secondly, Memento will have a "finer grained" paranoia available to it.

Usage with C++:

Memento has some experimental code in it to trap new/delete (and new[]/delete[] if required) calls.

In order for this to work, either:

1) Build memento.c with the c++ compiler.


2) Build memento.c as normal with the C compiler, then from any one of your .cpp files, do:

#define MEMENTO_CPP_EXTRAS_ONLY #include "memento.c"

In the case where MEMENTO is not defined, this will not do anything.

Both Windows and GCC provide separate new[] and delete[] operators for arrays. Apparently some systems do not. If this is the case for your system, define MEMENTO_CPP_NO_ARRAY_CONSTRUCTORS.

Getting better stack backtraces

At various points, Memento wants to give you stack backtraces. On pretty much all platforms other than Windows, there are limits to what the standard C runtime can give you - in particular it will give you file, but not function/line details for each frame in the callstack.

Memento will therefore make use of if it is installed on your system. It probably won't be installed by default, but it's really simple to build it yourself, and only needs to be done once per machine:

  • git clone git://
  • cd libbacktrace
  • ./configure
  • make

This leaves the built .so as .libs/ If this only builds .a and not .so files, do:

  • make clean # seems to be necessary, otherwise some symbols are missing in final .so.
  • ./configure --enable-shared
  • make

Memento will look for this on LD_LIBRARY_PATH, or in /opt/lib/, or in /lib/, or in /usr/lib/, or in /usr/local/lib/. I recommend using /opt/lib/ as this won't conflict with anything that you get via a package manager like apt.

  • sudo mkdir /opt
  • sudo mkdir /opt/lib
  • sudo cp .libs/ /opt/lib/

Memory Squeezing with Memento

On Unix, with single threaded programs.

To memory squeeze fast, Memento uses fork. Unfortunately fork goes horribly wrong with multi-threaded programs. Accordingly, we can only use the fast memory squeezer with single threaded programs.

Even more unfortunately, pthreads gets confused by fork, and so any use of pthread mutexes, semaphores etc, even in single threaded programs will break.

To work around this in gs, the gx_ functions that deal with threading and locking etc are nobbled, in that they check for Memento_squeezing() and don't do anything. (Previously we relied on MEMENTO_SQUEEZE_BUILD being defined at build time, but this is no longer required).

Then select the allocation event you would like to start squeezing at.

For instance:

  • ./
  • make memento
  • MEMENTO_SQUEEZEAT=1 membin/gs -sDEVICE=ppmraw -o /dev/null examples/tiger.eps > squeeze 2>&1 &

This will then run away in the background testing each possible exit route from the code, with the results going into squeeze.

You can watch for progress using:

  • tail -f squeeze | grep "Memory squeezing @"

A script to make the results of this slightly more readable can be found in the ghostpdl.git repo as toolbin/

Use it as cat squeeze | perl | head -1000000 > squeeze.html

The "| head -1000000" is to avoid giving your browser a hernia.

and then view the results in a browser. Red numbers are SEGVs, oranges are leaks, green are passes. (Adding -q removes the passes to save space).

The best single line (bash) command I've come up with is:

  • MEMENTO_SQUEEZEAT=1 membin/gs -sDEVICE=ppmraw -o /dev/null examples/tiger.eps 2>&1 | tee >(perl toolbin/ | gzip -9 -c > squeeze.html.gz) | grep "Memory squeezing @"

-- Robin Watts - 2017-02-02

In the .html file generated by toolbin/, one can search for the three different results with:

  • grep -w ok
  • grep -w segv
  • grep -w leak


You can test multiple squeeze regions by setting MEMENTO_SQUEEZES to one or more comma-separated ranges.

To simplify testing around known allocation numbers we support some limited addition and subtraction.

Each range is a begin value optionally followed by .. and an end value. The end value can start with '+' in which case it is relative to the begin value. A value is a number optionally followed by '+' or '-' and a second number.

        <value>             - evaluates to <value>..<value>+1.
        <value>..+<number>  - evaluates to <value>..<value>+<number>

    <number>: [0-9]+

E.g.: MEMENTO_SQUEEZES=115802-5..+10,115867-5..+10,119928-5..+10 will test the 10 adjacent allocation values around each of 115802, 115867 and 119928, which is useful to check one hasn't broken some previous memento fixes.


Edit | Attach | Watch | Print version | History: r13 < r12 < r11 < r10 < r9 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r13 - 2020-06-15 - JulianSmith
This site is powered by the TWiki collaboration platform Powered by PerlCopyright 2014 Artifex Software Inc