Search code examples
elasticsearchfoselasticabundle

How to store reusable data in elastic search


Since I don't want to call the API every time I need certain data (like an array of 1000 rows) I would like to store that array in ElasticSearch so I can easily get this array without the need to call the api. I'm using FOS Elastic Bundle. Is this even possible to make and if it is how?

What I would do:

-I have a function that gets this data from database
-I would like to save this data in ES after calling php bin/console fos:elastica:populate
-use this array in controller to return it to the view and use it there.

Solution

  • I would suggest that you define a type with a mapping that can cover a single row in your database. After that, when you have fetched the 1000 rows from the database, you can index those 1000 rows in a single bulk index call in form of 1000 documents: https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html You can then fetch these 1000 documents for use in controller.

    Alternatively, you can define a mapping with a nested property. This nested property should be identical to a row in your database. Using this, you can create a single document with 1000 rows worth of your data inside the nested property like an array. After that, you can fetch this single document.

    Which of these strategies is better will depend on your requirement. The second is a heavier indexing process while first is relatively heavy fetch process. In my experience with ElasticSearch, it is better to have lighter indexing requests to ensure data consistency. Depending on your data, you can create the 1000 documents with IDs in a certain pattern and with the IDs known, fetching these documents becomes very efficient.