Please suggest if we can access on-premises MS SQL Server Database from Azure Web App behind ASE.
My understanding is that we just need to get sql port opened on firewall for connectivity from ASE subnet to on-premises subnet.
You could but there are other ways like Hybrid connections:
Within App Service, Hybrid Connections can be used to access application resources in other networks. It provides access from your app to an application endpoint. It does not enable an alternate capability to access your application. As used in App Service, each Hybrid Connection correlates to a single TCP host and port combination. This means that the Hybrid Connection endpoint can be on any operating system and any application, provided you are accessing a TCP listening port. The Hybrid Connections feature does not know or care what the application protocol is, or what you are accessing. It is simply providing network access.
https://learn.microsoft.com/en-us/azure/app-service/app-service-hybrid-connections
Other choice is VPN or Expressroute
https://learn.microsoft.com/en-us/azure/app-service/environment/network-info