Search code examples
azure-cosmosdbgremlintinkerpop3azure-cosmosdb-gremlinapi

Optimize this query to get lineage diagram in graph using gremlin


I have my data set up in cosmos DB gremlin API. I am trying to write a query that can starting at a given vertex, gives all the edges with labels and properties in both directions, and the vertices with their properties, with a loop count of lets say 2. I am running the following queries.

Data to be setup

g.addV('item').property('id','1').property('name', 'item1')
g.addV('item').property('id','2').property('name', 'item2')
g.addV('item').property('id','3').property('name', 'item3')
g.addV('item').property('id','4').property('name', 'item4')
g.addV('item').property('id','5').property('name', 'item5')
g.addV('item').property('id','6').property('name', 'item6')
g.addV('item').property('id','7').property('name', 'item7')

g.addV('report').property('id','8').property('name', 'report1')
g.addV('report').property('id','9').property('name', 'report2')
g.addV('folder').property('id','10').property('name', 'folder1')
g.addV('server').property('id','11').property('name', 'server1')

g.V('1').addE('hasParent').to(g.V('2'))
g.V('1').addE('hasParent').to(g.V('3'))
g.V('2').addE('hasParent').to(g.V('4'))
g.V('5').addE('hasParent').to(g.V('6'))
g.V('4').addE('hasParent').to(g.V('5'))
g.V('7').addE('hasParent').to(g.V('5'))

g.V('8').addE('hasParent').to(g.V('9'))
g.V('8').addE('hasParent').to(g.V('9'))
g.V('8').addE('belongsTo').to(g.V('10'))
g.V('10').addE('belongsTo').to(g.V('11'))

query: g.V('id','1').repeat(bothE().dedup().otherV()).times(2).path()

response:

[
  {
    "labels": [
      [],
      [],
      [],
      [],
      []
    ],
    "objects": [
      {
        "id": "1",
        "label": "item",
        "type": "vertex",
        "properties": {
          "name": [
            {
              "id": "1|name",
              "value": "item1"
            }
          ]
        }
      },
      {
        "id": "a6b02073-68f8-4d53-8af0-54739eaaeaa0",
        "label": "hasParent",
        "type": "edge",
        "inVLabel": "item",
        "outVLabel": "item",
        "inV": "2",
        "outV": "1"
      },
      {
        "id": "2",
        "label": "item",
        "type": "vertex",
        "properties": {
          "name": [
            {
              "id": "2|name",
              "value": "item2"
            }
          ]
        }
      },
      {
        "id": "c8722748-3899-4a2c-afe4-6f13b6c1f8d5",
        "label": "hasParent",
        "type": "edge",
        "inVLabel": "item",
        "outVLabel": "item",
        "inV": "4",
        "outV": "2"
      },
      {
        "id": "4",
        "label": "item",
        "type": "vertex",
        "properties": {
          "name": [
            {
              "id": "4|name",
              "value": "item4"
            }
          ]
        }
      }
    ]
  },
  {
    "labels": [
      [],
      [],
      [],
      [],
      []
    ],
    "objects": [
      {
        "id": "1",
        "label": "item",
        "type": "vertex",
        "properties": {
          "name": [
            {
              "id": "1|name",
              "value": "item1"
            }
          ]
        }
      },
      {
        "id": "a6b02073-68f8-4d53-8af0-54739eaaeaa0",
        "label": "hasParent",
        "type": "edge",
        "inVLabel": "item",
        "outVLabel": "item",
        "inV": "2",
        "outV": "1"
      },
      {
        "id": "2",
        "label": "item",
        "type": "vertex",
        "properties": {
          "name": [
            {
              "id": "2|name",
              "value": "item2"
            }
          ]
        }
      },
      {
        "id": "a6b02073-68f8-4d53-8af0-54739eaaeaa0",
        "label": "hasParent",
        "type": "edge",
        "inVLabel": "item",
        "outVLabel": "item",
        "inV": "2",
        "outV": "1"
      },
      {
        "id": "1",
        "label": "item",
        "type": "vertex",
        "properties": {
          "name": [
            {
              "id": "1|name",
              "value": "item1"
            }
          ]
        }
      }
    ]
  },
  {
    "labels": [
      [],
      [],
      [],
      [],
      []
    ],
    "objects": [
      {
        "id": "1",
        "label": "item",
        "type": "vertex",
        "properties": {
          "name": [
            {
              "id": "1|name",
              "value": "item1"
            }
          ]
        }
      },
      {
        "id": "2f0c5890-3fa5-4d7d-8ada-910e5419750f",
        "label": "hasParent",
        "type": "edge",
        "inVLabel": "item",
        "outVLabel": "item",
        "inV": "3",
        "outV": "1"
      },
      {
        "id": "3",
        "label": "item",
        "type": "vertex",
        "properties": {
          "name": [
            {
              "id": "3|name",
              "value": "item3"
            }
          ]
        }
      },
      {
        "id": "2f0c5890-3fa5-4d7d-8ada-910e5419750f",
        "label": "hasParent",
        "type": "edge",
        "inVLabel": "item",
        "outVLabel": "item",
        "inV": "3",
        "outV": "1"
      },
      {
        "id": "1",
        "label": "item",
        "type": "vertex",
        "properties": {
          "name": [
            {
              "id": "1|name",
              "value": "item1"
            }
          ]
        }
      }
    ]
  }
]

This is giving me the desired output, but has a lot of repetitions, i am guessing because it prints the path as it traverses. How can I modify this query to return the smallest possible json output? Just return the vertices and edges once. I am using cosmosDB gremlin APIs if that makes any difference.

Response needed: just a list of vertices and edges needed for painting this graph.


Solution

  • Your example as-is doesn't produce the same output that I see in your question - I get just one path:

    gremlin> g.V('id','1').repeat(bothE().dedup().otherV()).times(2).path()
    ==>[v[1],e[11][1-hasParent->2],v[2],e[13][2-hasParent->4],v[4]]
    

    I'm not sure if that's some mistake in your data script but I think I get the gist of what you are asking from your output. I can recreate a situation of duplication by doing an emit() on the repeat() to include all paths in the output:

    gremlin> g.V('id','1').repeat(bothE().dedup().otherV()).emit().times(2).path()
    ==>[v[1],e[11][1-hasParent->2],v[2]]
    ==>[v[1],e[12][1-hasParent->3],v[3]]
    ==>[v[1],e[11][1-hasParent->2],v[2],e[13][2-hasParent->4],v[4]]
    

    Now I have some duplication of vertices/edges in the resulting output. If you just want a unique list of all those objects, just unfold() each Path and dedup():

    gremlin> g.V('id','1').repeat(bothE().dedup().otherV()).emit().times(2).path().unfold().dedup()
    ==>v[1]
    ==>e[11][1-hasParent->2]
    ==>v[2]
    ==>e[12][1-hasParent->3]
    ==>v[3]
    ==>e[13][2-hasParent->4]
    ==>v[4]
    

    This is a bit of an interesting pattern in a sense because it provides an easy way to produce a subgraph for Gremlin Language Variants and graph systems that don't support subgraph() step.