I'm confused as to how these two classes communicate with each other because I don't see them subclassing one or the other. How does UICollectionViewLayout communicate its information to a UICollectionView exactly?
A UICollectionView has a strong reference to its layout, through its collectionViewLayout property.
A UICollectionViewLayout has an unretained reference to its collection view, through its collectionView property. When you set the collection view's layout, the collection view sets the layout's collectionView property back to the collection view.
Most communication between the two objects happens by the collection view sending messages to the layout and receive return values back from those messages. Many of the messages are publicly declared in the UICollectionViewLayout class, and you can implement your own layout by making a subclass of UICollectionViewLayout and implementing/overriding methods to respond to those messages.