Search code examples
mysqletlorientdbgraph-visualization

OrientDB - Why does ETL need some time before beginning to load


As you can see from the console output:

BEGIN ETL PROCESSOR
+ extracted 0 records (0 records/sec) - 0 records -> loaded 0 vertices (0 vertic
es/sec) Total time: 1055ms [0 warnings, 0 errors]
+ extracted 0 records (0 records/sec) - 0 records -> loaded 0 vertices (0 vertic
es/sec) Total time: 2055ms [0 warnings, 0 errors]
+ extracted 0 records (0 records/sec) - 0 records -> loaded 0 vertices (0 vertic
es/sec) Total time: 3389ms [0 warnings, 0 errors]
+ extracted 0 records (0 records/sec) - 0 records -> loaded 0 vertices (0 vertic
es/sec) Total time: 4389ms [0 warnings, 0 errors]
+ extracted 0 records (0 records/sec) - 0 records -> loaded 0 vertices (0 vertic
es/sec) Total time: 5389ms [0 warnings, 0 errors]
+ extracted 0 records (0 records/sec) - 0 records -> loaded 0 vertices (0 vertic
es/sec) Total time: 6389ms [0 warnings, 0 errors]
+ extracted 0 records (0 records/sec) - 0 records -> loaded 0 vertices (0 vertic
es/sec) Total time: 7488ms [0 warnings, 0 errors]
+ extracted 0 records (0 records/sec) - 0 records -> loaded 0 vertices (0 vertic
es/sec) Total time: 8488ms [0 warnings, 0 errors]
+ extracted 0 records (0 records/sec) - 0 records -> loaded 0 vertices (0 vertic
es/sec) Total time: 9658ms [0 warnings, 0 errors]
+ extracted 236 records (234 records/sec) - 236 records -> loaded 235 vertices (
233 vertices/sec) Total time: 10663ms [0 warnings, 0 errors]

You can my ETL config here: https://groups.google.com/forum/#!topic/orient-database/JbMrhqSGXGM

So how it comes that in the first 10 seconds it does seem that he is doing nothing.

FYI: in the MySQL table are ~250.00 entries, just colums was taken from there


Solution

  • It has something to do with the way you access the database. There can be plocal and remote and some more. You should test them out which one works best on a local massive import.