Search code examples
solrdih

How to import multiple files into Solr using DIH

I have some data stored in file, and every line represent one record. So, I am using LineEntityProcessor and FileDataSource. Tt's ok to import one file, but my data is stored in multiple files. Is there any way to import more then one file into Solr at the same time?

Solution

  • You would want to use nested entities with outside entity being FileListEntityProcessor and your entity(entities) being inside that and using the variables exposed by the outer entity.

    You will also need to set rootEntity attribute to false on that outside entity, so a document is generated for each inner entity.

    Basically, you need one more level of entities wrapped around your existing definition.

    Or, if you have slightly different files and only a couple of those, you can just have several entity definitions side-by-side and DIH will run them all.