This question is bit on the Reverse Side of Connecting via SSMS from Azure VM to on premise SQL Server.
Can Hybrid connection Manager be used to configure connectivity between Onpremise and Azure such that i can use SSMS from Azure VM to connect to OnPremise SQL Server?
I don't think you can use Hybrid connection Manager to connect between On-premise and Azure network. As far as I know, the HCM is a relay agent and included in a partition of hybrid connection feature. The hybrid connection feature consists of two outbound calls to Azure Service Bus Relay.
For example, if you want web apps in the app service to access the DB in the on-premise network. There is a connection from a library on the host where your app is running in App Service. There is also a connection from the HCM to Service Bus Relay. See how it works. Also, The new Hybrid Connections capability of Relay is a secure, open-protocol evolution based on HTTP and WebSockets. In this case, you could not create a Hybrid Connection from Azure VM to Service Bus Relay via SSMS.
So, If you want to connect the on-premise SQL server from Azure VM via SSMS, you need to set up a VPN connection between Azure and your on-premise network. Read more details about VPN gateway.