My XQuery code should load a number of xml files from MarkLogic database, parse them, and output all results in single xml. How do I load all files in a specific directory in XQuery? For example, I have a MarkLogic database which has a folder "test", and this folder has different XML files. I could load each file using doc() function, but of course I can't just write doc("1.xml"), doc("2.xml") and so on because I don't know what files are available on the server. For now I can see only 1 solution to my problem:
Doing a search on a REST interface in a manner like someserver/v1/search?uri=test/ , loading it's results with the doc() function, parsing them, and then loading each document in a loop.
But perhaps there is another correct way to this problem? I am just starting to know MarkLogic and XQuery, so I could really miss something important. Maybe there is a more direct way to search for documents and load them than what I've proposed?
You could use something like:
<my-results>{
for $doc in xdmp:directory("test/", "infinity") (: use depth = '1' for no recursion :)
return my:format-results(my:parse($doc))
}</my-results>
or:
<my-results>{
for $uri in cts:uri-match("test/", "document")
let $doc := fn:doc($uri)
return my:format-results(my:parse($doc))
}</my-results>
But if your database contains a lot of XML files, the resulting XML could become very big, causing resource and timeout issues. It would be better to use the CPF framework to process content one by one in the background, and use paged searching to gather processing results that you could either store in a separate directory (one results doc per input file), or within the properties of the input file itself using something like xdmp:document-set-property.
HTH!