I have an Elasticsearch cluster on AWS ec2 instance. It is t3.small with 2 vcores and 2 GB ram. I have installed Elasticsearch & kibana. For extensions, I have installed Heartbeat and Metricbeat. The database I'm working with is mongo DB and all my data is no-SQL. I feed my engine from my MongoDB cluster which is in my local machine with a script. I feed my engine and run the queries from my app and also from the console. So far so good. Everything is fine. Well, the cluster is always yellow it is not green.
The problem starts after hitting multiple requests on the engine. After 50 or 60 search queries the data just disappears. Well somehow my engine is forcefully dumping my indices and it's not being able to restore those data ( obviously I have no snapshot and no restore point ) and I keep getting lose those data. I have to manually feed the engine again and again. Well at first I had 1 GB ram so I thought upgrading would fix the issue but after upgrading to 2 GB ram it didn't stop. Well, now the data stays there for some more time.
So here are my DB configs.
Testing purposes this is not the production mode.
So I need to know
Elastic documentation leaves much to be desired, but if, by "engine" you are referring to the engines that show up in the Enterprise Search/App Search tab (Open a deployment from the Elastic main dashboard, navigate to Overview link under the Search section, and from there to the App Search link under the Enterprise Search section), then this might help.
When an Engine is created, if you select the search engine type as being Elasticsearch index-based, and you select an index that does not start with search-, then it will use your selected index, but create an alias with the name search-yourindexname-alias. If yourindexname gets reindexed, that alias may disappear, and the engine will look like there are no records there. The solution should be to re-create the disappeared alias on yourindexname.