When measuring the throughput of a system that inherently caches (on the server side), should I turn the cache off to measure the worst-case, first load time, or is it irrelevant, as the great average is not really influenced by that one not cached call?
The actual system operates with image data (DICOM).
Here's what I think:
If you want to test the system as it will work in production, use the same setup as in production.
If you want to test a standalone component, feel free to disable any related components to make it separated.
For a load test it might be reasonable to disable the cache, but this is purely just for measurements. It won't reflect the real characteristics of the system.