Efficient merging of multiple, large xml files into one

Question

I searched the web and I searched stackoverflow up and down. No solution. Although I found solutions how to do this within pure xslt here.

But the problem is that the resulting xml will be several hundred MB large. So I must do this with SAX in Java. (please no xslt solution, although I tagged it with xslt ;-))

Let me explain with more detail. I have several multiple xml files (preferable InputSteam's) which should be parsed. The files or InputStream's looks like

inputstream1

<root>
  <doc>
    <tag>test1</tag>
  </doc>
  <doc>
    <tag>test2</tag>
  </doc>
  ...
</root>

inputstream2

<root>
  <doc>
    <tag>test3</tag>
  </doc>
  <doc>
    <tag>test4</tag>
  </doc>
  ...
</root>

inputstream1+inputstream2+...+inputstreamN = resulting xml. It will look like

<root>
  <doc>
    <tag>test1</tag>
  </doc>
  <doc>
    <tag>test2</tag>
  </doc>
  ...
   <doc>
    <tag>test3</tag>
  </doc>
  <doc>
    <tag>test4</tag>
  </doc>
  ...
</root>

Do someone has a solution or a link for this? Is this possible via implementing a custom InputSource or should I use a custom ContentHandler? Or is this possible with joost/stx?

The nice thing if I could use a ContentHandler would be that I could apply some minor transformations (I already implemented this). But then the problem is that I don't know a way to pass multiple files or InputStream's as InputSource:

XMLReader xmlReader = XMLReaderFactory.createXMLReader();
xmlReader.setContentHandler(customHandler);
xmlReader.parse(getInputSource()); // only one InputStream possible

or should I parse the InputStreams directly within my ContentHandler?

Answer 1

I finally managed this via the following snippet:

  finalHandler = new StreamResult(new OutputStreamWriter(System.out));
  // customHandler extends DefaultHandler
  CustomTransformerHandler customHandler = new CustomTransformerHandler(
         finalHandler);
  customHandler.startDocumentExplicitly();
  InputStream is = null;
  while ((is = customHandler.createNextInputStream()) != null) {
    // multiple inputStream parsing
    XMLReader myReader = XMLReaderFactory.createXMLReader();
    myReader.setContentHandler(customHandler);
    myReader.parse(new InputSource(is));
  }
  customHandler.endDocumentExplicitly();

The important part was to leave the startDocument and endDocument methods empty. All other methods (characters, startElement, endElement) will be redirected to the finalHandler. The customHandler.createNextInputStream method returns null if all inputstreams are read.