I'm going to integrate my Neo4j application with Elasticsearch. I'm going to try the following Neo4j plugin from GraphAware https://github.com/graphaware/neo4j-to-elasticsearch
Could you please tell does it support Neo4j 3.1.3 version? Also, what version of ES should be used there ? Does it support ES 5.3.0 for example ?
Right now I have installed Elasticsearch 5.3.0 locally. Also I have added 2 jars:
graphaware-neo4j-to-elasticsearch-3.1.3.45.7.jar
graphaware-server-community-all-3.1.3.46.jar
to my Neo4j CE 3.1.3\plugins\ folder.
Also, I have added the following config (as descrbed here https://github.com/neo4j-contrib/neo4j-elasticsearch#example) lines into my database configuration(Neo4j Community Edition\neo4j.conf):
elasticsearch.host_name=http://localhost:9200
elasticsearch.index_spec=decisions:Decision(name,description)
But when I'm inserting the data through Spring Data Neo4j project nothing happens.. The Elasticsearch indexes still empty.
What am I doing wrong and how to make it working ?
UPDATED
I made some progress on this:
This is my Neo4j.conf:
dbms.unmanaged_extension_classes=com.graphaware.server=/graphaware
com.graphaware.runtime.enabled=true
com.graphaware.module.ES.2=com.graphaware.module.es.ElasticSearchModuleBootstrapper
com.graphaware.module.UIDM.uuidProperty=ID()
com.graphaware.module.ES.node=hasLabel('Decision')
com.graphaware.module.ES.relationship=(false)
com.graphaware.module.ES.uri=localhost
com.graphaware.module.ES.port=9200
com.graphaware.module.ES.index=neo4j-index
In my application I have an entity: Decision. Decision is a complex type with a big parent hierarchy:
Decision extends Commentable extends Votable extends Flaggable ... and so on.
Right now I'm adding ~60 Decision nodes to my Neo4j database but I'm unable to see all of them at ES. I can see only the first one with a duplication per parent class in hierarchy.
This what what I have now:
$ curl -XGET 'http://127.0.0.1:9200/neo4j-index-node/_search?pretty=true&q=*:*'
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0{
"took" : 1,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"hits" : {
"total" : 8,
"max_score" : 1.0,
"hits" : [ {
"_index" : "neo4j-index-node",
"_type" : "Decision",
"_id" : "null",
"_score" : 1.0,
"_source" : {
"avgVotesWeight" : 0.0,
"updateDate" : "2017-04-19T09:17:36.883Z",
"nameSlug" : "antivirus-software-for-windows",
"lowerName" : "antivirus software for windows",
"description" : "This is a non-exhaustive list of notable antivirus and Internet Security software, in the form of comparison tables, according to their platform (e.g. desktop and server, mobile, etc.)",
"totalDislikes" : 0,
"totalChildDecisions" : 59,
"totalCriteria" : 19,
"multiVotesAllowed" : false,
"totalCharacteristics" : 8,
"totalComments" : 0,
"totalFlags" : 0,
"likeSum" : 0.0,
"totalVotes" : 665,
"name" : "Antivirus software for Windows",
"totalLikes" : 0,
"createDate" : "2017-04-19T09:17:20.347Z"
}
}, {
"_index" : "neo4j-index-node",
"_type" : "Authorable",
"_id" : "null",
"_score" : 1.0,
"_source" : {
"avgVotesWeight" : 0.0,
"updateDate" : "2017-04-19T09:17:36.883Z",
"nameSlug" : "antivirus-software-for-windows",
"lowerName" : "antivirus software for windows",
"description" : "This is a non-exhaustive list of notable antivirus and Internet Security software, in the form of comparison tables, according to their platform (e.g. desktop and server, mobile, etc.)",
"totalDislikes" : 0,
"totalChildDecisions" : 59,
"totalCriteria" : 19,
"multiVotesAllowed" : false,
"totalCharacteristics" : 8,
"totalComments" : 0,
"totalFlags" : 0,
"likeSum" : 0.0,
"totalVotes" : 665,
"name" : "Antivirus software for Windows",
"totalLikes" : 0,
"createDate" : "2017-04-19T09:17:20.347Z"
}
}, {
"_index" : "neo4j-index-node",
"_type" : "Subscribable",
"_id" : "null",
"_score" : 1.0,
"_source" : {
"avgVotesWeight" : 0.0,
"updateDate" : "2017-04-19T09:17:36.883Z",
"nameSlug" : "antivirus-software-for-windows",
"lowerName" : "antivirus software for windows",
"description" : "This is a non-exhaustive list of notable antivirus and Internet Security software, in the form of comparison tables, according to their platform (e.g. desktop and server, mobile, etc.)",
"totalDislikes" : 0,
"totalChildDecisions" : 59,
"totalCriteria" : 19,
"multiVotesAllowed" : false,
"totalCharacteristics" : 8,
"totalComments" : 0,
"totalFlags" : 0,
"likeSum" : 0.0,
"totalVotes" : 665,
"name" : "Antivirus software for Windows",
"totalLikes" : 0,
"createDate" : "2017-04-19T09:17:20.347Z"
}
}, {
"_index" : "neo4j-index-node",
"_type" : "BaseEntity",
"_id" : "null",
"_score" : 1.0,
"_source" : {
"avgVotesWeight" : 0.0,
"updateDate" : "2017-04-19T09:17:36.883Z",
"nameSlug" : "antivirus-software-for-windows",
"lowerName" : "antivirus software for windows",
"description" : "This is a non-exhaustive list of notable antivirus and Internet Security software, in the form of comparison tables, according to their platform (e.g. desktop and server, mobile, etc.)",
"totalDislikes" : 0,
"totalChildDecisions" : 59,
"totalCriteria" : 19,
"multiVotesAllowed" : false,
"totalCharacteristics" : 8,
"totalComments" : 0,
"totalFlags" : 0,
"likeSum" : 0.0,
"totalVotes" : 665,
"name" : "Antivirus software for Windows",
"totalLikes" : 0,
"createDate" : "2017-04-19T09:17:20.347Z"
}
}, {
"_index" : "neo4j-index-node",
"_type" : "Votable",
100 7842 100 7842 0 0 7842 0 0:00:01 --:--:-- 0:00:01 7658kl",
"_score" : 1.0,
"_source" : {
"avgVotesWeight" : 0.0,
"updateDate" : "2017-04-19T09:17:36.883Z",
"nameSlug" : "antivirus-software-for-windows",
"lowerName" : "antivirus software for windows",
"description" : "This is a non-exhaustive list of notable antivirus and Internet Security software, in the form of comparison tables, according to their platform (e.g. desktop and server, mobile, etc.)",
"totalDislikes" : 0,
"totalChildDecisions" : 59,
"totalCriteria" : 19,
"multiVotesAllowed" : false,
"totalCharacteristics" : 8,
"totalComments" : 0,
"totalFlags" : 0,
"likeSum" : 0.0,
"totalVotes" : 665,
"name" : "Antivirus software for Windows",
"totalLikes" : 0,
"createDate" : "2017-04-19T09:17:20.347Z"
}
}, {
"_index" : "neo4j-index-node",
"_type" : "Flaggable",
"_id" : "null",
"_score" : 1.0,
"_source" : {
"avgVotesWeight" : 0.0,
"updateDate" : "2017-04-19T09:17:36.883Z",
"nameSlug" : "antivirus-software-for-windows",
"lowerName" : "antivirus software for windows",
"description" : "This is a non-exhaustive list of notable antivirus and Internet Security software, in the form of comparison tables, according to their platform (e.g. desktop and server, mobile, etc.)",
"totalDislikes" : 0,
"totalChildDecisions" : 59,
"totalCriteria" : 19,
"multiVotesAllowed" : false,
"totalCharacteristics" : 8,
"totalComments" : 0,
"totalFlags" : 0,
"likeSum" : 0.0,
"totalVotes" : 665,
"name" : "Antivirus software for Windows",
"totalLikes" : 0,
"createDate" : "2017-04-19T09:17:20.347Z"
}
}, {
"_index" : "neo4j-index-node",
"_type" : "Likeable",
"_id" : "null",
"_score" : 1.0,
"_source" : {
"avgVotesWeight" : 0.0,
"updateDate" : "2017-04-19T09:17:36.883Z",
"nameSlug" : "antivirus-software-for-windows",
"lowerName" : "antivirus software for windows",
"description" : "This is a non-exhaustive list of notable antivirus and Internet Security software, in the form of comparison tables, according to their platform (e.g. desktop and server, mobile, etc.)",
"totalDislikes" : 0,
"totalChildDecisions" : 59,
"totalCriteria" : 19,
"multiVotesAllowed" : false,
"totalCharacteristics" : 8,
"totalComments" : 0,
"totalFlags" : 0,
"likeSum" : 0.0,
"totalVotes" : 665,
"name" : "Antivirus software for Windows",
"totalLikes" : 0,
"createDate" : "2017-04-19T09:17:20.347Z"
}
}, {
"_index" : "neo4j-index-node",
"_type" : "Commentable",
"_id" : "null",
"_score" : 1.0,
"_source" : {
"avgVotesWeight" : 0.0,
"updateDate" : "2017-04-19T09:17:36.883Z",
"nameSlug" : "antivirus-software-for-windows",
"lowerName" : "antivirus software for windows",
"description" : "This is a non-exhaustive list of notable antivirus and Internet Security software, in the form of comparison tables, according to their platform (e.g. desktop and server, mobile, etc.)",
"totalDislikes" : 0,
"totalChildDecisions" : 59,
"totalCriteria" : 19,
"multiVotesAllowed" : false,
"totalCharacteristics" : 8,
"totalComments" : 0,
"totalFlags" : 0,
"likeSum" : 0.0,
"totalVotes" : 665,
"name" : "Antivirus software for Windows",
"totalLikes" : 0,
"createDate" : "2017-04-19T09:17:20.347Z"
}
} ]
}
}
Why all of the Decision's sub-classes(like Votable, Likeable) are also duplicated there? As you can see from my config I have tried to exclude everything except:
com.graphaware.module.ES.node=hasLabel('Decision')
My goal is to index only Decision nodes at ES.
Also, why the only single decision from ~60 is indexed there ? I think "_id" : "null" at the Elasticsearch JSON output is related to this issue. What am I doing wrong ?
It seems our documentation has been weird about the usage of the internal id as document id :
Replace this line :
com.graphaware.module.UIDM.uuidProperty=ID()
By this
com.graphaware.module.ES.keyProperty=ID()
However if you use the uuid module as well, it is best to use "uuid" instead of the internal ID
I tested on community 3.1.3 with the versions mentioned and it is working.
{
"took" : 1,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"hits" :{
"_index" : "neo4j-index-node",
"_type" : "Decision",
"_id" : "2",
"_score" : 1.0,
"_source":{"id":123,"title":"Ibiza"}
}, {
"_index" : "neo4j-index-node",
"_type" : "Decision",
"_id" : "3",
"_score" : 1.0,
"_source":{"id":123,"title":"Weird Decision"}
} ]
}
}
If your nodes have both labels Decision and Votable, then you have to exclude them in the configuration like this :
com.graphaware.module.ES.node=hasLabel('Decision') && !hasLabel('Votable') && !hasLabel('BaseEntity')