I am transferring my rdb to neo4j 2.0.3 using neography. At the beginning my each transfer query used to take very less time, but now(size of the graphml file is around 1.3gbs) the insertion is taking 3-5xtime and often throws this error for certain queries.
org.neo4j.server.rest.repr.RepresentationExceptionHandlingIterable.exceptionOnHasNext(RepresentationExceptionHandlingIterable.java:50)
org.neo4j.helpers.collection.ExceptionHandlingIterable$1.hasNext(ExceptionHandlingIterable.java:46)
org.neo4j.helpers.collection.IteratorWrapper.hasNext(IteratorWrapper.java:42)
org.neo4j.server.rest.repr.ListRepresentation.serialize(ListRepresentation.java:71)
org.neo4j.server.rest.repr.Serializer.serialize(Serializer.java:75)
org.neo4j.server.rest.repr.MappingSerializer.putList(MappingSerializer.java:61)
org.neo4j.server.rest.repr.CypherResultRepresentation.serialize(CypherResultRepresentation.java:83)
org.neo4j.server.rest.repr.MappingRepresentation.serialize(MappingRepresentation.java:41)
org.neo4j.server.rest.repr.OutputFormat$1.write(OutputFormat.java:160)
org.neo4j.server.guard.GuardingRequestFilter.doFilter(GuardingRequestFilter.java:68)
org.neo4j.server.rest.security.SecurityFilter.doFilter(SecurityFilter.java:112)
org.neo4j.server.guard.GuardingRequestFilter.doFilter(GuardingRequestFilter.java:68)
java.lang.Thread.run(Thread.java:745)
and
org.neo4j.kernel.guard.Guard$Timeout.check(Guard.java:132) org.neo4j.kernel.guard.Guard.check(Guard.java:43) org.neo4j.kernel.InternalAbstractGraphDatabase$5.getNodeByIdOrNull(InternalAbstractGraphDatabase.java:745) org.neo4j.kernel.impl.core.NodeManager.getNodeById(NodeManager.java:301) org.neo4j.kernel.InternalAbstractGraphDatabase.getNodeById(InternalAbstractGraphDatabase.java:1121) org.neo4j.cypher.internal.spi.v2_0.TransactionBoundQueryContext$NodeOperations.getById(TransactionBoundQueryContext.scala:157) org.neo4j.cypher.internal.spi.v2_0.TransactionBoundQueryContext$$anonfun$getNodesByLabel$1.apply(TransactionBoundQueryContext.scala:131) org.neo4j.cypher.internal.spi.v2_0.TransactionBoundQueryContext$$anonfun$getNodesByLabel$1.apply(TransactionBoundQueryContext.scala:131) org.neo4j.cypher.internal.helpers.JavaConversionSupport$$anon$2.next(JavaConversionSupport.scala:33) scala.collection.Iterator$$anon$11.next(Iterator.scala:328) scala.collection.Iterator$$anon$13.next(Iterator.scala:372) scala.collection.Iterator$$anon$14.hasNext(Iterator.scala:389) scala.collection.Iterator$$anon$13.hasNext(Iterator.scala:371) scala.collection.Iterator$$anon$14.hasNext(Iterator.scala:388) scala.collection.Iterator$class.foreach(Iterator.scala:727) scala.collection.AbstractIterator.foreach(Iterator.scala:1157) scala.collection.generic.Growable$class.$plus$plus$eq(Growable.scala:48) scala.collection.mutable.ListBuffer.$plus$plus$eq(ListBuffer.scala:176) scala.collection.mutable.ListBuffer.$plus$plus$eq(ListBuffer.scala:45) scala.collection.TraversableOnce$class.to(TraversableOnce.scala:273) scala.collection.AbstractIterator.to(Iterator.scala:1157) scala.collection.TraversableOnce$class.toList(TraversableOnce.scala:257) scala.collection.AbstractIterator.toList(Iterator.scala:1157)
I am importing data of movies from rdb to neo4j by looping over each row of rdb and creating node for a movie (if not existing) and its related tags (if not existing) and then creating relationships between them, and similar such actions.
How can I make the performance better and resolve this error?
This is my neo4j-wrapper.conf
While searching, I landed on http://docs.neo4j.org/chunked/stable/configuration.html which gives a detail performance and configuration tips. But it seems complex to begin with. Are there any to do's and don't's to use neo4j with ruby for better performance.
Please help me out.
This gives a nice background of scaling up...