Search code examples
xpathxerces-cxqilla

How can I extract a node with XPath 2 from a HTML document (Xerces/Xqilla)


I want to extract specific nodes from a html document with XPATH2 using the xerces and xqilla libraries, but apparantly I'm unable to construct a valid XPATH expression or my code is wrong somewhere.

My current code:

#include <iostream>
#include <string>

#include <xercesc/dom/DOMImplementation.hpp>
#include <xercesc/dom/DOMImplementationRegistry.hpp>
#include <xercesc/dom/DOMConfiguration.hpp>

#include <xercesc/dom/DOMXPathExpression.hpp>
#include <xercesc/dom/DOMXPathResult.hpp>
#include <xercesc/dom/DOMLSParser.hpp>
#include <xercesc/dom/DOMDocument.hpp>
#include <xercesc/dom/DOMLSSerializer.hpp>
#include <xercesc/dom/DOMLSOutput.hpp>

#include <xercesc/sax/SAXParseException.hpp>
#include <xercesc/sax/ErrorHandler.hpp>

#include <xercesc/framework/StdOutFormatTarget.hpp>
#include <xercesc/framework/MemBufInputSource.hpp>
#include <xercesc/framework/Wrapper4InputSource.hpp>

#include <xercesc/util/XMLString.hpp>

#include <xqilla/xqilla-dom3.hpp>
#include <xercesc/parsers/AbstractDOMParser.hpp>
using namespace std;


const char document[] = { 0x3c, 0x21, 0x44, 0x4f, 0x43, .....,  0x6c, 0x3e, 0x0a, 0x00 };

int main() {

    // init xerces and xqilla engines
    XQillaPlatformUtils::initialize();

    // retrieve xqilla DOMImpl.
    xercesc::DOMImplementation* xqilla_impl
        = xercesc::DOMImplementationRegistry::getDOMImplementation(X("XPath2 3.0"));

    {
        // create DOMLSParser
        AutoRelease<xercesc::DOMLSParser> parser(xqilla_impl->createLSParser(xercesc::DOMImplementationLS::MODE_SYNCHRONOUS, 0));
        xercesc::DOMConfiguration *config = parser->getDomConfig();
        config->setParameter(xercesc::XMLUni::fgXercesScannerName, xercesc::XMLUni::fgWFXMLScanner);

        // retrieve lesson page:
        string str(document);

        xercesc::Wrapper4InputSource* wrapper =
                new xercesc::Wrapper4InputSource(
                new xercesc::MemBufInputSource((XMLByte*) str.c_str(), (XMLSize_t) str.length(), "index.html", false));


        // create DOM structure:
        xercesc::DOMDocument* dom = parser->parse(wrapper);

        AutoRelease<xercesc::DOMXPathExpression> expression(
            dom->createExpression(xercesc::XMLString::transcode("html/head"), 0)
        );

        AutoRelease<xercesc::DOMXPathResult> result(expression->evaluate(
            dom, xercesc::DOMXPathResult::ITERATOR_RESULT_TYPE, 0
        ));

        cout << result->iterateNext() << endl; // output is always 0
    }

    XQillaPlatformUtils::terminate();

    return 0;
}

What do I have to change?

EDIT:

The HTML files I want to look at are a large collection of files, so I made a small sample file to test my programm and/or XPATH expressions:

  <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
  <html xmlns="http://www.w3.org/1999/xhtml" >
  <head>
    <title>Some title</title>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
    <meta name="title" content="Some title" />
    <meta name="keywords" content="Keywords" />
    <meta name="description" content="A short description" />
  </head>
  <body>
    <p>
      Lorem ipsum dolor sit amet, consetetur sadipscing elitr,
      sed diam nonumy <b>eirmod <u>tempor</u> invidunt ut</b> labore et dolore<br />
      magna aliquyam erat, sed diam voluptua. At vero eos et accusam
      et justo duo dolores et ea rebum. Stet clita kasd gubergren,<br />
      no sea takimata sanctus est Lorem ipsum dolor sit amet.
    </p>
  </body>
</html>

Even with this, my program does not find any node with my XPATH expression.



I was able to find 2 suboptimal solutions for my problem

  • 1. solution:
    Use an XPath expression, that does not care about namespaces like '*:html/*:head/*:title/text()'.
  • 2. solution:
    Turn off DOM namespaces in the parser:
    config->setParameter(xercesc::XMLUni::fgDOMNamespaces, false);

I would be happier, if had found a way to manually set a custom prefix for an unnamed DOM namespace or if I had an XPath expression, where I can explicitly specify an empty prefix, but now at least I can process my documents.


Solution

  • The html and head elements are in a namespace, whereas your XPath is looking for elements in no namespace. Use "h:html/h:head" and bind the "h" prefix to the XHTML namespace. I don't know precisely how to do this binding with the XQilla API, but there'll be some method to do it.