So it seems that in the spirit of having documentation all over the place, docs.microsoft has once again taken the cake.
Looking into sending push notifications, I came across these two pages:
Notice how there's no reference from either page to the other.
So I tried searching Google, doing similar searches to the title of this question, only to find little more than the above two pages.
Which brings me here, with the following question: What is the relationship (and/or difference) between App Center Push and Azure Notification Hub?
They don't have a reference from either to the other because they are two completely unrelated solutions (except for the fact they solve the same problem).
Both allow you to setup a cross-platform push notification infrastructure for your mobile app, but App Center is free, specifically built for iOS, Android and Windows notifications and requires less setup, but gives you a bit less control over how the notifications look. However it makes it easier to segment notifications based on analytics data. Notifications Hub is a Azure service, so you need a Azure subscription and you need to build a backend to make it work. It takes a more work but you have more complete control over the notifications.
The awesome James Montemagno made a nice summary article comparing the two options.