Search code examples
amazon-s3marklogic

Copy data from Marklogic to Amazon S3


Is there a way to copy/move data from Marklogic Server to Amazon S3? I don't want all data to be moved, but certain documents pertaining to a particular collection or some other logic. I can do xdmp:save() and that works for few thousand documents, but I have got few million records and this method won't work out well in that case, is there a better and robust way that can be used to copy data over? Can I use MLCP for this or use a spawn module to run it over a task server and get this work done? I am running on ML-8 hosted on AWS.

Any suggestion would help immensely.

Regards Amit


Solution

  • I used mlcp export for making the change, and it works quite well with the collection filter and does the trick for me. I have not tried the CORB2 yet, but will give it a try as well when time permits

    mlcp export -host {host} -port {port} -username {username} -password {password} -output_file_path {S3 path} -collection_filter {collection name to be moved}