If I understand correctly, Pytorch's API is stable between patch versions, so its downstream packages should still work with staggered patch versions. For instance, torchaudio 2.0.2 and torch 2.0.1 should be compatible. But all 3 of these packages require an exactly matching pytorch version, e.g. torch==2.0.2.
Is there a reason why Pytorch does this?
Because they integrate PyTorch on C++ level. Each library has extension module written in C++ which links against libtorch. For example, TorchVision implements image I/O using libjpegturbo and libpng, which converts the common image format into PyTorch tensor.
TorchAudio has audio/video processor, TorchText has sentencepiece and other NLP-related utilities written in C++.
The problem is that libtorch does not have stable ABI (not API), and PyTorch does not have stable ABI policy for C++ interface.
If you attempt to import these libraries with unmatched PyTorch, you get somewhat cryptic error message like undefined symbol: _znk3c104type14issubtypeofextest10shared_ptris0_epso.
This essentially means that "this C++ function of PyTorch found at built time was not found in the runtime PyTorch. I cannot proceed."