This definitely is a fundamental knowledge gap for me but when people say in memory what do they usually mean?
For example, in my mind it means a running machine and you have some process like redis that starts up on that machine where the data is stored but I'm not sure if that's correct. I'm thinking they don't mean you spin up a web service and have a data structure that stores data or maybe that also is in memory?
Thanks in advance for any clarity.
In memory i.e. data loaded to RAM, vs. data stored on storage devices e.g. SSD, HDD.
For example, some databases loads everything to RAM, some other databases load indexes to RAM and some load indexes and enables swapping if they don't fit (on expense of performance when this happens) and there multiple other options too. The point is different databases can be in-memory, partially in-memory or not in-memory.
If you read an entire file to a buffer/string, it's in memory, if you go through it's stream, processing only the current record, then the file does not need to fit in memory.
When you keep something in memory, you can access it faster (the CPU actually loads from it's cache the fastest, then from the RAM, then from storage (where flash is much faster than magnetic) and even with these speed depends on a variety of technologies in play.
In general, RAM is more expensive than storage and it is usually volatile (as in does not keep state when powered down). If you want to store a lot of data in memory, your hardware bills will be higher, however, you can achieve a lower latency.
In memory can refer to what's loaded to RAM in your application or in a database you are sizing (allocating/pricing HW for).