• Looks like PyMuPDF has fairly elaborate support for redactions. Makes use of pdf_redact_page() and then writes on top of the redaction?

• • Added Document::lookup_metadata() method overload that returns std::string.
  • added global const std::vector<std::string> metadata_keys.
  • Changed Outline iteration to include depth information.

• • Added python wrapper for PdfDocument::page_write() out-params
  • Using improved scheme for wrapping functions/methods with out-params - instead of trying to use SWIG's typemaps, which are very clumy in the context of mupdfwrap.py and seemingly more designed for custom-written .i files, we now use simple auto-generated C functions to package up out-params into a struct, then extract into a tuple in auto-generated Python.
  • Provide two wrappers for mupdf.Buffer.buffer_extract() - return raw C (size, data) values or return a Python bytes. The former can be used to construct a mupdf.Stream constructor (doesn't seem possible to convert a Python bytes back into (size, data)). [This allows us to mimic PyMuPDF-Utilities/demo/pdf-converter.py.]

• PyMuPDF can copy a TOC into a PdfDocument.

• • added python wrapper for PdfDocument::page_write() out-params
  • Using improved scheme for wrapping functions/methods with out-params - instead of trying to use SWIG's typemaps, which are very clumy in the context of mupdfwrap.py and seemingly more designed for custom-written .i files, we now use simple C functions to package up out-params into a struct.
  • Make mupdf.Buffer.buffer_extract() return raw C (data, size) values; don't convert into a Python bytes, because can't find a way to convert a bytes back into (data, size) suitable for passing to mupdf.Stream constructor. [This allows us to mimic PyMuPDF-Utilities/demo/pdf-converter.py].

• Comparison with PyMuPDF.
• No surprises so far.
• For more details see: #Comparison_with_PyMuPDF

Comparison with PyMuPDF

• Am writing equivalent code to some example programmes in https://github.com/pymupdf/PyMuPDF-Utilities.
• Method names are usually different, because PyMuPDF uses its own names instead of basing names on the underlying MuPDF API.
• Have made various additions/fixes to mypdfwrap.py (for details see: https://git.ghostscript.com/?p=user/julian/mupdf.git;a=summary)
  • added Document::lookup_metadata() method overload that returns std::string.
  • added const std::vector<std::string> metadata_keys
  • changed Outline iteration to include depth information
  • Fixed ref-counting in Page::load_links().
  • fixed Page::search_page() to return std::vector.
• PyMuPDF has more information about links - fitz.LINK_GOTO, LINK_GOTOR, fitz.LINK_LAUNCH, fitz.LINK_URI.
• PyMuPDF has abstraction for writing image files which calls fz_save_pixmap_as_png() or fz_save_pixmap_as_pnm() etc, depending on the filename.

PyMuPDF: https://github.com/pymupdf/PyMuPDF-Utilities/blob/master/demo/demo.py

Equivalent code using mupdfwrap:

#! /usr/bin/env python3

import mupdf

import os
import sys


assert len(sys.argv) == 7
filename, page_num, zoom, rotate, output, needle = sys.argv[1:]
page_num = int(page_num)
zoom = int(zoom)
rotate = int(rotate)

document = mupdf.Document(filename)

print('')
print(f'Document {filename} has {document.count_pages()} pages.')
print('')
print(f'Metadata Information:')
print(f'mupdf.metadata_keys={mupdf.metadata_keys}')
for key in mupdf.metadata_keys:
    value = document.lookup_metadata(key)
    print(f'    {key}: {value!r}')
print('')

outline = mupdf.Outline(document)
for o in outline:
    print(f'    {" "*4*o.m_depth}{o.m_depth}: {o.m_outline.title()}')

if page_num > document.count_pages():
    raise SystemExit(f'page_num={page_num} is out of range - {filename} has {document.count_pages()} pages')

page = document.load_page(page_num)
links = page.load_links()
if links:
    print(f'Links on page {page_num}:')
    for link in links:
        if link.m_internal:
            print(f'    extern={mupdf.is_external_link(link.uri())}: {link.uri()}')
else:
    print(f'No links on page {page_num}')

trans = mupdf.Matrix.scale(zoom / 100.0, zoom / 100.0).pre_rotate(rotate)

pixmap = page.new_pixmap_from_page(trans, mupdf.Colorspace(mupdf.Colorspace.Fixed_RGB), alpha=False)

def save_pixmap(path):
    suffix = os.path.splitext(path)[1]
    if 0: pass
    elif suffix == '.pam':   pixmap.save_pixmap_as_pam(path)
    elif suffix == '.pbm':   pixmap.save_pixmap_as_pbm(path)
    elif suffix == '.pcl':   pixmap.save_pixmap_as_pcl(path, append=0, options=mupdf.PclOptions())
    elif suffix == '.pclm':  pixmap.save_pixmap_as_pclm(path, append=0, options=mupdf.PclmOptions())
    elif suffix == '.pdfocr':pixmap.save_pixmap_as_pdfocr(path, append=0, options=mupdf.PdfocrOptions())
    elif suffix == '.pkm':   pixmap.save_pixmap_as_pkm(path)
    elif suffix == '.png':   pixmap.save_pixmap_as_png(path)
    elif suffix == '.pnm':   pixmap.save_pixmap_as_pnm(path)
    elif suffix == '.ppm':   pixmap.save_pixmap_as_ppm(path)
    elif suffix == '.ps':    pixmap.save_pixmap_as_ps(path, append=0)
    elif suffix == '.psd':   pixmap.save_pixmap_as_psd(path)
    elif suffix == '.pwg':   pixmap.save_pixmap_as_pwg(path, append=0, pwg=mupdf.PwgOptions())
    else:
        raise Exception(f'Unrecognised output format: {path}')
save_pixmap(output)
hit_quads = page.search_page(needle, max=16)
print(f'search text {needle!r} found {len(hit_quads)} on the page')
for hit_quad in hit_quads:
    pixmap.invert_pixmap_rect(hit_quad.rect_from_quad().irect_from_rect())
save_pixmap(f'dl-{output}')

• MuPDFWrap

• The C++ API is built by mupdf:scripts/mupdfwrap.py. It requires clang-6 or clang-7, and python-clang.

• Building the Python API requires swig-3 or swig-4.

• We generate C++ wrapper functions for most fz_ and pdf_ functions. These wrapper convert fz_ exceptions into C++ exceptions, and use auto-generated per-thread fz_context's.
• We generate C++ class wrappers for most fz_ and pdf_ structs.
• We auto-detect fz_*() and pdf_*() fns suitable for wrapping as constructors, methods or static methods.

• Wrapper class constructors and methods provide access to 1270 fz_*() and pdf_*() fns, out of a total of 1513 wrapped fz_*() and pdf_*() functions. Most of the omitted functions don't take struct args, e.g. fz_strlcpy().

• We use clang to extract doxygen-style comments, and propagate them into generated header files.
• If swig is version 4+, we tell it to propagate comments into the generated mupdf.py.

• https://ghostscript.com/~julian/mupdf/include/html/index.html
• https://ghostscript.com/~julian/mupdf/platform/c++/include/html/index.html

• https://ghostscript.com/~julian/mupdf/build/shared-release/mupdf.html

Auto-generated C++ and Python APIs for mupdf.

C++

• C++ API is generated by mupdf:scripts/mupdfwrap.py, which uses python-clang.
• We require clang-6 or clang-7.
• We generate C++ class wrappers for all fz structs.

• We provide access via wrapper classes to 690 fz_*() fns.
• There are a total of 890 fz_*() functions. Most of the omitted functions don't take struct args, e.g. fz_strlcpy(). Of the remaining, some use fz_* enums (which we don't yet wrap); see https://ghostscript.com/~julian/mupdf/platform/c++/fn_usage.txt for more information.

Python

• Python API is generated by running SWIG on the C++ API's header files.
• Python API is enough to allow implementation of mutool in Python - see mupdf:scripts/mutool.py and mupdf:scripts/mutool_draw.py.

General

• We work on nuc1 and peeved and jules-laptop.
• We require:
  • python-clang (version 6 or 7)
  • python3-dev (version 3.6 or later)
  • swig (version 3 or 4)

mutool.py

mudpdf:scripts/mutool*.py are a Python re-implementation of the mutool application.

They do not use threads, or the include/mupdf/pdf/ functionality.

• We provide access via wrapper classes to 686 fz_*() fns.
• There are a total of 888 fz_*() functions. Most of the omitted functions don't take struct args, e.g. fz_strlcpy(). Of the remaining, some use fz_* enums (which we don't yet wrap); see https://ghostscript.com/~julian/mupdf/platform/c++/fn_usage.txt for more information.
• We work with swig-3 or swig-4.
• We work with clang-6 or clang-7.
• We work on peeved and jules-laptop.

• https://ghostscript.com/~julian/doxygen/mupdf/index.html
• https://ghostscript.com/~julian/doxygen/mupdfcpp/index.html

And pydoc html representation of the generated mupdf.py API:

• https://ghostscript.com/~julian/pydoc/mupdf.html

Status

As of 2020-4-24:

• We generate class wrappers for all fz structs.
• We auto-detect fz_*() fns suitable for wrapping as constructors, methods or static methods.
• Some generated classes have auto-generated support for iteration.
• We add various custom wrappers for fz_*() fns.
• We add various custom methods/constructors.
• We provide access via wrapper classes to 715 fz_*() fns.
• There are a total of 935 fz_*() functions. Most of the omitted functions don't take struct args, e.g. fz_strlcpy(). Of the remaining, some use fz_* enums (which we don't yet wrap); see https://ghostscript.com/~julian/mupdf/platform/c++/fn_usage.txt for more information.

Comments

• We use clang to extract doxygen-style comments, and propogate them into generated header files.
• If swig is version 4+, we tell it to propogate comments into generated mupdf.py.

Files

• https://ghostscript.com/~julian/mupdf/platform/c++/fn_usage.txt

• https://ghostscript.com/~julian/mupdf/platform/c++/
• https://ghostscript.com/~julian/mupdf/platform/python

• https://git.ghostscript.com/?p=user/julian/mupdf.git;a=blob;f=scripts/mupdfwrap.py;hb=HEAD

Integration with mupdf git.

    mupdf/
        build/
            shared/
                libmupdf.so  [generated file]
                libmupdfcpp.so [generated file, implements C++ API]
        platform/
            c++/
                Makefile
                implementation/
                    *.cpp [generated files]
                include/
                    mupdf/
                        *.h [generated files]
            python/
                Makefile
                mudf.py [generated file, implements Python API]
                _mupdf.so [generated file, implements Python API internals]

• https://git.ghostscript.com/?p=user/julian/mupdf.git;a=summary
• https://git.ghostscript.com/?p=user/julian/julian-tools.git

To build, ensure that julian-tools is checked out next to mupdf, then:

    cd mupdf/platform/python
    make
    make test

The makefiles build things by running ../../../julian-tools/mupdfwrap.py.

Have also added crude support for building mupdf as a shared object, which is required for integrating with the SWIG python module. At the moment, mupdfwrap.py drives this to build mupdf.so.

See:

• https://git.ghostscript.com/?p=user/julian/julian-tools.git;a=blob;f=mutool.py;hb=HEAD
• https://git.ghostscript.com/?p=user/julian/julian-tools.git;a=blob;f=mutool_draw.py;hb=HEAD

mutool.py

Python implementation of mutool. as of 2020-4-1, this only implements some of the "trace" sub-command. See:

* https://git.ghostscript.com/?p=user/julian/julian-tools.git;a=blob;f=mutool.py;hb=HEAD

• https://ghostscript.com/~julian/mupdf_test/test.py.html

• https://ghostscript.com/~julian/mupdf_test/test.py.txt
• https://ghostscript.com/~julian/mupdf_test/test_out.txt

• [Old] Using lower-case and underscores: https://ghostscript.com/~julian/mupdf_test0/

• See test_mupdfcpp_swig() function for example python code that uses the generated API.
• See "Todo:" section in comment near top, for status of various todo items.

• Using lower-case and underscores: https://ghostscript.com/~julian/mupdf_test0/

• https://git.ghostscript.com/?p=user/julian/julian-tools.git;a=blob;f=mupdfwrap.py;hb=HEAD

• https://git.ghostscript.com/?p=user/julian/julian-tools.git;a=blob;f=test_mupdfpy.py.py;hb=HEAD

• https://ghostscript.com/~julian/mupdf_test/

Auto-generated mupdf C++ wrappers

Article text.

Standard wrappers:

• https://ghostscript.com/~julian/mupdf_wrap.cpp.txt
• https://ghostscript.com/~julian/mupdf_wrap.h.txt

Using class template for exceptions:

• https://ghostscript.com/~julian/mupdf_wrapt.cpp.txt
• https://ghostscript.com/~julian/mupdf_wrapt.h.txt

These were generated by:

• https://git.ghostscript.com/?p=user/julian/julian-tools.git;a=blob;f=jclang.py;hb=HEAD

-- Julian Smith - 2020-03-04

Comments

<--/commentPlugin-->