Search code examples
rxmlxpathopenxmldocx

Extracting structured content from Word XML

I want to extract the text content from Word documents while preserving the document structure. This question comes closest and provides the starting point. Here is a dummy word document. I've unzipped it and included the document.xml below:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<w:document xmlns:wpc="http://schemas.microsoft.com/office/word/2010/wordprocessingCanvas" xmlns:mo="http://schemas.microsoft.com/office/mac/office/2008/main" xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006" xmlns:mv="urn:schemas-microsoft-com:mac:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:r="http://schemas.openxmlformats.org/officeDocument/2006/relationships" xmlns:m="http://schemas.openxmlformats.org/officeDocument/2006/math" xmlns:v="urn:schemas-microsoft-com:vml" xmlns:wp14="http://schemas.microsoft.com/office/word/2010/wordprocessingDrawing" xmlns:wp="http://schemas.openxmlformats.org/drawingml/2006/wordprocessingDrawing" xmlns:w10="urn:schemas-microsoft-com:office:word" xmlns:w="http://schemas.openxmlformats.org/wordprocessingml/2006/main" xmlns:w14="http://schemas.microsoft.com/office/word/2010/wordml" xmlns:w15="http://schemas.microsoft.com/office/word/2012/wordml" xmlns:wpg="http://schemas.microsoft.com/office/word/2010/wordprocessingGroup" xmlns:wpi="http://schemas.microsoft.com/office/word/2010/wordprocessingInk" xmlns:wne="http://schemas.microsoft.com/office/word/2006/wordml" xmlns:wps="http://schemas.microsoft.com/office/word/2010/wordprocessingShape" mc:Ignorable="w14 w15 wp14"><w:body><w:p w14:paraId="09AE8A7D" w14:textId="338F199D" w:rsidR="00D8348A" w:rsidRPr="00987637" w:rsidRDefault="00987637" w:rsidP="00987637"><w:pPr><w:pStyle w:val="Title"/><w:rPr><w:sz w:val="40"/><w:szCs w:val="40"/></w:rPr></w:pPr><w:r w:rsidRPr="00987637"><w:rPr><w:sz w:val="40"/><w:szCs w:val="40"/></w:rPr><w:t xml:space="preserve">The </w:t></w:r><w:r w:rsidR="00A6772E"><w:rPr><w:sz w:val="40"/><w:szCs w:val="40"/></w:rPr><w:t>Example Docx</w:t></w:r><w:bookmarkStart w:id="0" w:name="_GoBack"/><w:bookmarkEnd w:id="0"/></w:p><w:p w14:paraId="0803A50C" w14:textId="77777777" w:rsidR="00987637" w:rsidRDefault="00987637"/><w:p w14:paraId="0EDF635A" w14:textId="77777777" w:rsidR="00987637" w:rsidRDefault="00987637" w:rsidP="00987637"><w:pPr><w:pStyle w:val="Heading2"/></w:pPr><w:r><w:t>Introduction</w:t></w:r></w:p><w:p w14:paraId="2AF32AB2" w14:textId="77777777" w:rsidR="00987637" w:rsidRDefault="00987637"/><w:p w14:paraId="4DB9AE94" w14:textId="6C08452A" w:rsidR="00987637" w:rsidRPr="00987637" w:rsidRDefault="00C52D6C" w:rsidP="00987637"><w:r><w:t>This is the introduction para.</w:t></w:r><w:r w:rsidR="00A6772E" w:rsidRPr="00A6772E"><w:t xml:space="preserve"> </w:t></w:r><w:r w:rsidR="00A6772E"><w:t>Awesome.</w:t></w:r></w:p><w:p w14:paraId="676B3B48" w14:textId="77777777" w:rsidR="00987637" w:rsidRPr="00987637" w:rsidRDefault="00987637" w:rsidP="00987637"/><w:p w14:paraId="17CE18FE" w14:textId="77777777" w:rsidR="00987637" w:rsidRDefault="00987637" w:rsidP="00987637"><w:pPr><w:pStyle w:val="Heading2"/></w:pPr><w:r><w:t>Chapter 1</w:t></w:r></w:p><w:p w14:paraId="58A187F6" w14:textId="77777777" w:rsidR="00987637" w:rsidRDefault="00987637"/><w:p w14:paraId="2815D3B5" w14:textId="58856A78" w:rsidR="00C52D6C" w:rsidRPr="00987637" w:rsidRDefault="00C52D6C" w:rsidP="00C52D6C"><w:r><w:t xml:space="preserve">This is the </w:t></w:r><w:r><w:t xml:space="preserve">chapter 1 intro </w:t></w:r><w:r><w:t>para.</w:t></w:r><w:r><w:t xml:space="preserve"> </w:t></w:r><w:r w:rsidR="00A6772E"><w:t>Awesome.</w:t></w:r></w:p><w:p w14:paraId="7010E2CD" w14:textId="77777777" w:rsidR="00987637" w:rsidRDefault="00987637" w:rsidP="00987637"/><w:p w14:paraId="52D9B766" w14:textId="77777777" w:rsidR="00987637" w:rsidRDefault="00987637" w:rsidP="00987637"><w:pPr><w:pStyle w:val="Heading3"/></w:pPr><w:r><w:t>Chapter 1.1</w:t></w:r></w:p><w:p w14:paraId="055DB7AC" w14:textId="77777777" w:rsidR="00987637" w:rsidRDefault="00987637" w:rsidP="00987637"/><w:p w14:paraId="22A2C473" w14:textId="1D3FE489" w:rsidR="00A6772E" w:rsidRPr="00987637" w:rsidRDefault="00A6772E" w:rsidP="00A6772E"><w:r><w:t xml:space="preserve">This is </w:t></w:r><w:r><w:t>section 1.1</w:t></w:r><w:r><w:t xml:space="preserve"> </w:t></w:r><w:r><w:t>text</w:t></w:r><w:r><w:t>. Awesome.</w:t></w:r></w:p><w:p w14:paraId="1F60E111" w14:textId="77777777" w:rsidR="00987637" w:rsidRDefault="00987637" w:rsidP="00987637"/><w:p w14:paraId="62A6FEE6" w14:textId="77777777" w:rsidR="00987637" w:rsidRDefault="00987637" w:rsidP="00987637"><w:pPr><w:pStyle w:val="Heading3"/></w:pPr><w:r><w:t xml:space="preserve">Chapter </w:t></w:r><w:r><w:t>1.2</w:t></w:r></w:p><w:p w14:paraId="3B8DA076" w14:textId="77777777" w:rsidR="00987637" w:rsidRPr="00987637" w:rsidRDefault="00987637" w:rsidP="00987637"/><w:p w14:paraId="43519FB7" w14:textId="311F726C" w:rsidR="00987637" w:rsidRDefault="00A6772E" w:rsidP="00987637"><w:r><w:t>This is section 1.</w:t></w:r><w:r><w:t>2</w:t></w:r><w:r><w:t xml:space="preserve"> text. Awesome.</w:t></w:r></w:p><w:p w14:paraId="2D6908F2" w14:textId="77777777" w:rsidR="00987637" w:rsidRDefault="00987637" w:rsidP="00987637"/><w:p w14:paraId="3189011C" w14:textId="77777777" w:rsidR="00987637" w:rsidRDefault="00987637" w:rsidP="00987637"><w:pPr><w:pStyle w:val="Heading2"/></w:pPr><w:r><w:t>Chapter 2</w:t></w:r></w:p><w:p w14:paraId="6FFC56CD" w14:textId="77777777" w:rsidR="00987637" w:rsidRDefault="00987637" w:rsidP="00987637"/><w:p w14:paraId="3E9AE930" w14:textId="329881E3" w:rsidR="00987637" w:rsidRPr="00987637" w:rsidRDefault="00A6772E" w:rsidP="00987637"><w:r><w:t xml:space="preserve">This is </w:t></w:r><w:r><w:t>the c</w:t></w:r><w:r><w:t xml:space="preserve">hapter </w:t></w:r><w:r><w:t>2</w:t></w:r><w:r><w:t xml:space="preserve"> </w:t></w:r><w:r><w:t xml:space="preserve">text. </w:t></w:r><w:r><w:t>Awesome.</w:t></w:r></w:p><w:p w14:paraId="57F0B3DF" w14:textId="77777777" w:rsidR="00987637" w:rsidRDefault="00987637"/><w:p w14:paraId="48315F42" w14:textId="77777777" w:rsidR="00987637" w:rsidRDefault="00987637"/><w:sectPr w:rsidR="00987637" w:rsidSect="00751697"><w:pgSz w:w="11900" w:h="16840"/><w:pgMar w:top="1440" w:right="1440" w:bottom="1440" w:left="1440" w:header="720" w:footer="720" w:gutter="0"/><w:cols w:space="720"/><w:docGrid w:linePitch="360"/></w:sectPr></w:body></w:document>

This reads in the xml:

library(xml2)
library(magrittr)
doc = read_xml('document.xml')

I'm unclear about next steps to extract the content from this. I tried:

xml_find_all(doc, "/w:body/w:p/w:hyperlink/w:r/w:t") %>% xml_text()

But this returns nothing. Grateful for assistance. My goal is to export the text content in some format that preserves the document structure - perhaps a csv with columns reflecting the tree from Title through Heading 1 etc to Normal.

----------------------------------------\

Edit:

The example doc structure is as I see it:

Title
-- Heading 2
-- -- para
-- -- Chapter 1
-- -- -- para
-- -- -- Chapter 1.1
-- -- -- -- para
-- -- -- Chapter 1.2
-- -- -- -- para
-- -- Chapter 2
-- -- -- para

To be honest I'm flexible on output (json is fine) but it needs to reflect this approx doc structure.

Solution

  • You're likely severely underestimating the complexity of the task.

    XPath is for selection. It would be a simple matter to write an XPath expression to extract a set of nodes from your OOXML document. For example, this XPath 2.0 expression,

    //w:p[w:pPr/w:pStyle/@w:val='Heading2']/string()

    will return for your document:

    Introduction
Chapter 1
Chapter 2

    If you want to do more than select, if you want to transform, then you'll want to step up to XSLT. You'll also have to delve into 5K+ pages of OOXML specification to handle everything that can be done in a Word document. Note that indentation levels, since they can come from styles, can be particularly tricky to cover in general.

    XPath is amazingly powerful for selection, however. I'll leave you with one more XPath 2.0 expression,

    //w:p[normalize-space()]/normalize-space()

    which for your document will return,

    The Example Docx
Introduction
This is the introduction para. Awesome.
Chapter 1
This is the chapter 1 intro para. Awesome.
Chapter 1.1
This is section 1.1 text . Awesome.
Chapter 1.2
This is section 1. 2 text. Awesome.
Chapter 2
This is the c hapter 2 text. Awesome.

    And that seems tantalizingly close to what you want for nearly no development effort. You might be able to recover the hierarchy with less effort than the general case if you restrict your input space, but to handle the general case is extremely complex.