In MESI protocol, if a cacheline needs to be loaded into a cache, the CPU will issue a PrRd. Depending whether the cacheline is already in another cache, a BusRd is issued. And other caches will then see the BusRd and check if they have a valid copy. If yes, this cache will send value.
Now intel CAT (Cache Allocation Technology) provides a way to isolate LLC cache usage to different CPUs. For example, CPU1 uses the first 8 ways and CPU2 uses the next 8 ways. My question is: if now CPU1 needs to load a cacheline which is in CPU2's cache, will CPU2 send that copy instead of loading from main memory?
Yes. CAT is not a form of NUMA, the address space is still shared. It's just a micro-architectural feature that helps you control the cache occupancy so threads will have less interferences with each other (or have access to more caching opportunities, depending on how you allocate the masks).
If you don't return data from the other thread partition you will lose your coherence (what if the line is modified? you can't return stale data from the memory in that case). Think of it like this - each thread can lookup from the entire cache, but allocate only to his partition (this could easily be implemented by hacking the LRU and victim selection). This way you get full control over private lines, and only shared lines will be placed in the partition of whichever thread accessed them first. Close enough to get what QoS was needed for.
One open implementation question could be this - what happens what you allocate a line in one partition, but continue using it only by the other thread. Will it eventually get migrated to the other partition? My guess is no, simply because it's too much of a hassle to detect and organize.