Content Search Web-Parts supports a caching feature as described here: https://support.office.com/en-us/article/Make-pages-load-faster-with-caching-in-the-Content-Search-Web-Part. This should bring enhanced performance.
When configuring caching as described in the article the Content Search Web-Part does not return any results.
After saving and publishing the Content Search Web-Part does not return any result. This is reproducable in multiple tenants.
If you want to enable caching for a security group, this security group requires permissions on the site collection (min. read permission). In my case "Everyone, except external users" had no read permissions on the site collection.