We have an API, we planning to publish this API in the Azure marketplace under developer service. For authentication, we are planning to use developer service SSO (Single sign-on) in API. I googled this but can't find any useful link, please suggest some links
Azure AD B2C became generally available recently and it somehow offers what you are looking for. You can use local Azure identity and/or social account for your end users to authenticate with SSO support. https://azure.microsoft.com/en-us/services/active-directory-b2c/
With that being said and although you can secure your Web API with AAD b2C as in this guide, your customers won't be able to call this WebAPI from their own applications as the calling app must be registered in the same Azure AD directory with the same application Id. This is a limitation that is mentioned here:
Note: Azure AD B2C currently supports only web APIs that are accessed by their own well-known clients. For instance, your complete app may include an iOS app, an Android app, and a back-end web API. This architecture is fully supported. Allowing a partner client, such as another iOS app, to access the same web API is not currently supported. All of the components of your complete app must share a single application ID.
If the above limitation is OK with you, then you might also be interested in this Azure article to list your app in the Azure AD application gallery.