I am "learning" about the development of ASP.NET Web APIs with security provided by OAUTH2 and OpenId. So I have started with the Web API Template. The Web API Template has its own "Individual Account" user management mechanism which can issue tokens via /Token. However getting Refresh Tokens with this mechanism is a little trickier.
This got me thinking, should I be really looking to provide this functionality by using IdentityServer instead. If so, as I understand it, as I am using ASP.NET 4.7/MVC5, and not ASP.Core, then I should use IdentityServer3 and not version 4.
Also there is all of the login functionality required when the Access Tokens and Refresh Tokens are expired.
It's unlikely that anyone will be able to definitively tell you whether to use Identity Server or not. You will need to make that decision yourself as you understand the problem domain the most.
If you do decide to offload authentication to a central authority provider (like Identity Server) then I can tell you that there is no hard dependencies that would force you to use Identity Server 3 or 4 as they both ultimately conform to oauth2/oidc protocol. If you have ability to host .Net Core apps then I can recommend you to use Identity Server 4 as that implementation is significantly more extensible and offers extreme flexibility.