MarkLogic XML Queries

Question

We would like to add some scripts to our monitoring environment, for which we would need some help with the XML queries for

1. Blockings

2. Total number of connections

3. Highest cpu consuming ML ingestion process

4. Long Running Process

5. Stats - cpu, memoryetc

6. Any forests down

We are creating a self service tool to monitor above

Answer 1

There are a number of different functions that can be used to obtain most of the information that you are looking for: xdmp:host-status(), xdmp:server-status(), and xdmp:forest-status(). I'll post a few specific code examples below.

There are also Manage REST endpoints available, if you want to plot some of the information over time to get an idea of rates and trends, and to be able to see things at different granularity (minutely, hourly, daily).

1. Blockings

Using the code from this MarkLogic knowledgebase article on locks, you can list all of the transactions that are blocked and waiting on other transaction locks:

declare namespace server ="http://marklogic.com/xdmp/status/server";

declare variable $host := xdmp:host();

for $server in xdmp:servers()
return
  for $request in xdmp:server-status($host,$server)//server:request-status
  let $transaction-id := $request/server:transaction-id
  let $start-time := $request/server:start-time/text()
  order by $request/server:start-time ascending
  return
  (
    "Server : "||xdmp:server-name($server),
    "Transaction : "||$transaction-id,
    "Started at : "||$start-time,
    try{ xdmp:transaction-locks($host, $transaction-id) }catch($e){}
  )

2. Total number of connections

You can obtain the list of requests for a given Appserver on a given Host from xdmp:server-status and add them up:

sum(xdmp:server-status(xdmp:hosts(), xdmp:servers())/*:request-statuses/*:request-status)

However, you might want to just look at the request-rate (which reports the requests/sec) for each appserver.

3. Highest cpu consuming ML ingestion process

There isn't a mechanism to obtain CPU consumption per transaction/request. It isn't something that is metered and measured down at that level.

4. Longest running process

   head( for $request in xdmp:server-status(xdmp:hosts(), xdmp:servers())/:request-statuses/:request-status order by $request/*:start-time/xs:dateTime(.) ascending return $request )

5. Stats - cpu, memory, etc

You can obtain metrics from xdmp:host-status. There are various cpu, memory and other host status fields.

xdmp:host-status(xdmp:hosts())

6. Any forests down

You can check xdmp:forest-status() and decide what criteria to apply. For instance, to list the primary forests that are not open:

import module namespace admin = "http://marklogic.com/xdmp/admin"   at "/MarkLogic/admin.xqy";
declare namespace f = "http://marklogic.com/xdmp/status/forest";

for $forest in admin:get-forest-ids(admin:get-configuration())
let $status := xdmp:forest-status($forest)
let $forestName := $status/f:forest-name/text()
let $forestState := $status/f:state/text()
let $isMaster := $status/f:master-forest/text() eq $forest 
let $isDown := $forestState ne "open"   
where $isMaster and $isDown 
return $forestName||": "||$forestState