How to fetch document from database and pass it to transformation using dmsdk?

Question

I need to fetch document from database and apply transformation using dmsdk. I successfully fetched documents from database using below line of codes:

        QueryManager queryManager=client.newQueryManager();
        StructuredQueryBuilder sqb = queryManager.newStructuredQueryBuilder();
        StructuredQueryDefinition query =sqb.collection("test");

It returns URI's of the documents. But my transformation accepts json objects as a input. I need to pass json objects instead of uri.

My Transformation:

xquery version "1.0-ml";

module namespace test =
  "http://marklogic.com/rest-api/transform/deepan";

declare function test:transform(
  $context as map:map,
  $params as map:map,
  $content as document-node()
) as document-node()
{
    let $jsoncont := xdmp:from-json-string($content)

    let $inputval := "fname,lname"
    let $orig-value := map:get($jsoncont, "value")

    let $jscode := "var simple = require('/wdsUtils.sjs');
                    var content, input;
                    simple.createUri(content,input);"

    let $uri := xdmp:javascript-eval($jscode,('content',$orig-value,'input',$inputval))
    let $_ := map:put($content, "uri",$uri)
    let $_ := map:put($content, "value",$orig-value)
    return $content  

};

My dmsdk code:

    static String HOST = "localhost";
    static int PORT = 8136;
    static String USER = "admin";
    static String PASSWORD = "admin";
    private static DatabaseClient client = 
            DatabaseClientFactory.newClient(
                HOST, PORT, new DigestAuthContext(USER, PASSWORD));

    public static void loadData(String txName)
    {
        QueryManager queryManager=client.newQueryManager();
        StructuredQueryBuilder sqb = queryManager.newStructuredQueryBuilder();
        StructuredQueryDefinition query =sqb.collection("test");
        DataMovementManager dmm = client.newDataMovementManager();
        QueryBatcher batcher = dmm.newQueryBatcher(query);

        batcher.withConsistentSnapshot();
        ServerTransform txform = new ServerTransform(txName);

        ApplyTransformListener transformListener = new ApplyTransformListener()
                .withTransform(txform)
                .withApplyResult(ApplyResult.REPLACE);

        batcher.onUrisReady(transformListener)          
                .onQueryFailure( exception -> exception.printStackTrace() );

        dmm.startJob(batcher);
    }

    public static void main(String[] args) 
    {
        loadData("deepan");
    }

Exception:

01:09:10.819 [main] WARN com.marklogic.client.datamovement.ApplyTransformListener - Error: com.marklogic.client.FailedRequestException: Local message: failed to apply resource at internal/apply-transform: Bad Request. Server Message: XDMP-ARGTYPE: (err:XPTY0004) fn:doc(fn:doc("/one.json")) -- arg1 is not of type xs:string* in batch with urs ([/one.json, /three.json])
01:09:10.821 [pool-1-thread-2] DEBUG com.marklogic.client.impl.OkHttpServices - Query uris with structured query <query xmlns="http://marklogic.com/appservices/search" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:search="http://marklogic.com/appservices/search" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><collection-query><uri>test</uri></collection-query></query>
01:09:10.821 [pool-1-thread-2] DEBUG com.marklogic.client.impl.OkHttpServices - Getting internal/uris as text/uri-list
01:09:10.823 [pool-1-thread-1] DEBUG com.marklogic.client.impl.OkHttpServices - Query uris with structured query <query xmlns="http://marklogic.com/appservices/search" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:search="http://marklogic.com/appservices/search" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><collection-query><uri>test</uri></collection-query></query>
01:09:10.824 [pool-1-thread-1] DEBUG com.marklogic.client.impl.OkHttpServices - Getting internal/uris as text/uri-list
01:09:10.830 [pool-1-thread-2] DEBUG com.marklogic.client.impl.OkHttpServices - Posting internal/apply-transform
01:09:10.852 [pool-1-thread-2] WARN com.marklogic.client.datamovement.ApplyTransformListener - Error: com.marklogic.client.FailedRequestException: Local message: failed to apply resource at internal/apply-transform: Bad Request. Server Message: XDMP-ARGTYPE: (err:XPTY0004) fn:doc(fn:doc("/two.json")) -- arg1 is not of type xs:string* in batch with urs ([/two.json])

Answer 1

The xdmp:from-json-string() expects a string, but the $content parameter to the transform is a document-node(), not a string.

Try xdmp:from-json() instead of xdmp:from-json-string() to convert the JSON node to a map if you need a mutable structure.

Also, I'm wondering whether the xdmp:javascript-eval() is necessary. You should be able to call the function from XQuery with something along the lines of

let $uri := xdmp:apply(
    xdmp:function(xs:QName("createUri"), "/wdsUtils.sjs"),
    $orig-value,
    $inputval)

It might not be necessary to convert the $content JSON node to a map (which becomes an object literal in JavaScript) depending on what the createUri() function does.

The map:put() operations won't work on a node. Instead, consider converting a map to a JSON node with something along the lines of

return xdmp:to-json(map:entry("uri",$uri)=>map:with("value",$orig-value))

Hoping that helps,