Why is JanusGraph not dropping a discarded index?
I'm performance-evaluating graph-databases as of their latest releases that meet our needs.
I need to drop to remake the index for JMH testing: speed, stability, and scale.
package org.janusgraph.core.schema;
public enum SchemaAction {
// ...
DROP_INDEX(Collections.singleton(SchemaStatus.DISCARDED)),
// ...
}
Please note that on-topic, answers to this question posed here, and furthering comments to this specified setup, are always appreciated.
This intended for a persistent backend JanusGraph server; not an 'inmemory' gremlin server.
So this data will not be cleared between events, logins, or restarts.
This Cassandra-Server is for a persistent hosting service.
This not a POC of a test POC within a Gremlin-Console sample.
Further evaluation will be applied upon the enabled index so no suggestions of clearing, deleting, or inmemory please.
I was expecting the index to drop away so I could recreate the index with
janusGraphManagement.buildIndex("_id", Vertex.class).addKey(propertyKey).buildCompositeIndex();
But what actually happened was
Exception in thread "main" java.lang.IllegalStateException: Vertex with id 521 was removed.
I tried seeing if this was another one of those implicit / automatic processes JanusGraph sometimes has.
buildIndex() automatically does its own get() / call(), SchemaStatus.REGISTERED, and SchemaStatus.ENABLED.
But when I comment out the janusGraphManagement.commit(), JanusGraph says it didn't drop the index.
Exception in thread "main" java.lang.IllegalArgumentException: An index with name '_id' has already been defined
janusGraphManagement.commit()...
2023-05-16 11:00:01,455 [INFO] [Main.main] :: _id: DISCARDED
Exception in thread "main" java.lang.IllegalStateException: Vertex with id 521 was removed.
at org.janusgraph.core.InvalidElementException.removedException(InvalidElementException.java:57)
at org.janusgraph.graphdb.vertices.AbstractVertex.verifyAccess(AbstractVertex.java:93)
at org.janusgraph.graphdb.vertices.AbstractVertex.query(AbstractVertex.java:140)
at org.janusgraph.graphdb.types.vertices.JanusGraphSchemaVertex.getDefinition(JanusGraphSchemaVertex.java:97)
at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.expireSchemaElement(StandardJanusGraphTx.java:1632)
at org.janusgraph.graphdb.database.management.ManagementSystem.commit(ManagementSystem.java:252)
at Test18.main(Test18.java:54)
...
janusGraphManagement.commit()...
2023-05-16 10:19:28,787 [INFO] [Main.main] :: _id: DISCARDED
2023-05-16 10:19:28,798 [INFO] [Main.main] :: SchemaAction.DROP_INDEX index.getIndexStatus(propertyKey): DISCARDED
Exception in thread "main" java.lang.IllegalArgumentException: An index with name '_id' has already been defined
at com.google.common.base.Preconditions.checkArgument(Preconditions.java:220)
at org.janusgraph.graphdb.database.management.ManagementSystem.checkIndexName(ManagementSystem.java:661)
at org.janusgraph.graphdb.database.management.ManagementSystem.createCompositeIndex(ManagementSystem.java:728)
at org.janusgraph.graphdb.database.management.ManagementSystem.access$300(ManagementSystem.java:130)
at org.janusgraph.graphdb.database.management.ManagementSystem$IndexBuilder.buildCompositeIndex(ManagementSystem.java:824)
at Test18.main(Test18.java:35)
// ...
public static void main(String[] args) throws InterruptedException, ExecutionException {
JanusGraph janusGraph = JanusGraphFactory.build().set("storage.backend", "cql").set("storage.hostname", "localhost:9042").open();
GraphTraversalSource g = janusGraph.traversal();
g.V().drop().iterate();
janusGraph.tx().commit();
JanusGraphManagement janusGraphManagement = janusGraph.openManagement();
PropertyKey propertyKey = janusGraphManagement.getOrCreatePropertyKey("_id");
logger.info(janusGraphManagement.getGraphIndex("_id") + ":\t" + janusGraphManagement.getGraphIndex("_id").getIndexStatus(propertyKey).toString());
// ...
if (SchemaStatus.DISCARDED.equals(janusGraphManagement.getGraphIndex("_id").getIndexStatus(propertyKey))){
JanusGraphIndex janusGraphIndex = janusGraphManagement.getGraphIndex("_id");
janusGraphManagement = janusGraph.openManagement();
janusGraphManagement.updateIndex(janusGraphIndex, SchemaAction.DROP_INDEX).get();
janusGraphManagement.commit();
logger.info("SchemaAction.DROP_INDEX index.getIndexStatus(propertyKey):\t" + janusGraphIndex.getIndexStatus(propertyKey));
}
if (!janusGraphManagement.containsGraphIndex("_id")) {
janusGraphManagement = janusGraph.openManagement();
janusGraphManagement.buildIndex("_id", Vertex.class).addKey(propertyKey).buildCompositeIndex();
janusGraphManagement.commit();
JanusGraphIndex janusGraphIndex = janusGraphManagement.getGraphIndex("_id");
logger.info("index.getIndexStatus(propertyKey):\t" + janusGraphIndex.getIndexStatus(propertyKey));
}
// ...
Ended-up with how JanusGraphManagement handles updateIndex().
JanusGraphIndex will be "considered" the new SchemaStatus when asking with getIndexStatus.
This won't be true when awaiting with awaitGraphIndexStatus().status() .
The ManagementSystem only sees the same as after janusGraphManagement.commit().
Then, a new janusGraphManagement = janusGraph.openManagement() will have to be made to compare that they say the janusGraphIndex says the same with both of them.
But, when performing a SchemaAction.DROP_INDEX this will not be true.
janusGraphIndex will maintain its status as DISCARDED until janusGraphManagement.commit() has occurred. This is a differing behavior to the rest of the SchemaAction handling.
This can be verified with the sample below.
2023-05-16 12:32:33,880 [INFO] [Main.main] :: buildCompositeIndex
janusGraphIndex.getIndexStatus(propertyKey): ENABLED
2023-05-16 12:32:34,013 [INFO] [Main.main] :: index.getIndexStatus(propertyKey): ENABLED
2023-05-16 12:32:34,020 [INFO] [Main.main] :: SchemaAction.DISABLE_INDEX
janusGraphIndex.getIndexStatus(propertyKey): DISABLED
2023-05-16 12:32:34,177 [INFO] [o.j.g.o.j.IndexRemoveJob.Thread-19] :: Index _id metrics: success-tx: 5 doc-updates: 0 succeeded: 0
...
2023-05-16 12:32:34,288 [INFO] [o.j.g.o.j.IndexRemoveJob.Thread-1] :: Index _id metrics: success-tx: 21 doc-updates: 0 succeeded: 0
2023-05-16 12:32:34,401 [INFO] [o.j.g.d.m.ManagementSystem.Thread-1] :: Index update job successful for [_id]
2023-05-16 12:32:34,409 [INFO] [o.j.g.d.m.GraphIndexStatusWatcher.main] :: All 1 key(s) on index _id have status(es) [DISCARDED]
2023-05-16 12:32:34,411 [INFO] [Main.main] :: SchemaAction.DISCARD_INDEX
janusGraphIndex.getIndexStatus(propertyKey): DISCARDED
2023-05-16 12:32:34,415 [INFO] [Main.main] :: SchemaAction.DROP_INDEX
janusGraphIndex.getIndexStatus(propertyKey): DISCARDED
2023-05-16 12:32:34,521 [INFO] [Main.main] :: SchemaAction.DROP_INDEX
janusGraphIndex: null
Process finished with exit code 0
// ...
JanusGraphManagement janusGraphManagement = janusGraph.openManagement();
PropertyKey propertyKey = janusGraphManagement.getOrCreatePropertyKey("_id");
JanusGraphIndex janusGraphIndex;
if (!janusGraphManagement.containsGraphIndex("_id")) {
janusGraphManagement.buildIndex("_id", Vertex.class).addKey(propertyKey).buildCompositeIndex();
janusGraphIndex = janusGraphManagement.getGraphIndex("_id");
logger.info("buildCompositeIndex \njanusGraphIndex.getIndexStatus(propertyKey):\t" + janusGraphIndex.getIndexStatus(propertyKey));
janusGraphManagement.commit();
}
janusGraphManagement = janusGraph.openManagement();
janusGraphIndex = janusGraphManagement.getGraphIndex("_id");
if (SchemaStatus.ENABLED.equals(janusGraphIndex.getIndexStatus(propertyKey))) {
logger.info("index.getIndexStatus(propertyKey):\t" + janusGraphIndex.getIndexStatus(propertyKey));
janusGraphManagement.updateIndex(janusGraphIndex, SchemaAction.DISABLE_INDEX).get();
logger.info("SchemaAction.DISABLE_INDEX \njanusGraphIndex.getIndexStatus(propertyKey):\t" + janusGraphIndex.getIndexStatus(propertyKey));
janusGraphManagement.commit();
}
if (SchemaStatus.DISABLED.equals((janusGraphIndex.getIndexStatus(propertyKey)))) {
janusGraphManagement = janusGraph.openManagement();
janusGraphIndex = janusGraphManagement.getGraphIndex("_id");
janusGraphManagement.updateIndex(janusGraphIndex, SchemaAction.DISCARD_INDEX).get();
ManagementSystem.awaitGraphIndexStatus(janusGraph, "_id").status(SchemaStatus.DISCARDED).call();
logger.info("SchemaAction.DISCARD_INDEX \njanusGraphIndex.getIndexStatus(propertyKey):\t" + janusGraphIndex.getIndexStatus(propertyKey));
janusGraphManagement.commit();
}
if (SchemaStatus.DISCARDED.equals((janusGraphIndex.getIndexStatus(propertyKey)))) {
janusGraphManagement = janusGraph.openManagement();
janusGraphIndex = janusGraphManagement.getGraphIndex("_id");
janusGraphManagement.updateIndex(janusGraphIndex, SchemaAction.DROP_INDEX).get();
for (int i=0; i<15; i++)
if (!janusGraphManagement.containsGraphIndex("_id"))
break;
logger.info("SchemaAction.DROP_INDEX \njanusGraphIndex.getIndexStatus(propertyKey):\t" + janusGraphIndex.getIndexStatus(propertyKey));
janusGraphManagement.commit();
}
janusGraphManagement = janusGraph.openManagement();
janusGraphIndex = janusGraphManagement.getGraphIndex("_id");
logger.info("SchemaAction.DROP_INDEX \njanusGraphIndex:\t" + janusGraphIndex);
// ...