Search code examples
node.jspdfmd5checksumpdftk

Why does repeated bursting of a multi-page PDF into individual pages via pdftk change the md5 checksum of those pages?

I'm using pdftk and doing some testing and finding that bursting a multipage PDF file into separate single page PDF files, and then generating an md5 hash checksum (digital fingerprint) for each of those single page PDFs results in a different hash every time I do the burst. This is the result even if it's the exact same file with no changes.

My test process is:

  1. Decompress test.pdf (a simple text-only PDF that contains 10 pages)
  2. Using pdftk, burst (split) test.pdf into 10 separate PDF files (1 page per file)
  3. Generate md5 hash checksum for each of the 10 single-page PDF files
  4. Record the 10 hash checksums
  5. Repeat steps 1-4
  6. Note that all hashes differ

Side note: generating a checksum on the PDF after decompression yields the exact same checksum upon repetition.

I'm using node.js and its crypto module for this exercise.

My question is: Why do the checksums differ upon repetition? I would think that the resulting 10 single-page files are exactly the same as the last time they were created. Their parent document (and thus the individual pages themselves) has not changed at all.

Solution

  • According to the PDF spec, whenever a PDF creator writes out a modified PDF, it should update the key named /ModDate in the /Info array of metadata entries.

    Also, it will (likely) change the document UUID in the PDF's XMP metadata structure to a new ID.

    So, when you want to use MD5 (or any similar method) to check for 'stable results' in your PDF generation processes (think of unit tests or whatever), you should do one of these two things before applying your MD5-summing:

    • either 'normalize' your PDF output to always write the same ModDate and UUID into the files (if your PDF-generating facilities allows you to tweak it that way),
    • or run an edit (you can use sed) over the files that normalizes the /ModDate (and possibly also the /CreationDate) and UUID entries of the files.

    Update: Since you seem to be familiar with pdftk already, you should be able to dump a metadata text file (like Ezra showed):

    pdftk in.pdf dump_data output data.txt

    or (in case you need it):

    pdftk in.pdf dump_data_utf8 output data.utf8.txt

    Then edit the data*.txt files to make them suite your needs: change the PDF UUIDs (pdftk calls them PdfID0 / PdfID1) to easily recognizable values (00000... and fffff...), change the dates to another easily recognizeable one. Then update your files with these metadata values:

    pdftk in.pdf update_info data.txt output in-updated.pdf \
      &&  mv in-updated.pdf in.pdf

    or 

    pdftk in.pdf update_info data.utf8.txt output in-updated.utf8.pdf \
      &&  mv in-updated.utf8.pdf in.pdf

    Only then run your Md5 checksumming and see if it works (or needs some more fine-tuning).