I want to implement a custom client secret validator in my IdentityServer4 host. My question is, how do I register it with the identity server builder in my startup.cs file? There is a method to register a regular secret validator but I cannot find one for adding the ClientSecretValidator.
A 'secret validator' within the context of IdentityServer4 refers to a validator for client secrets (see the description of the 'AddSecretValidator' extension method under 'Additional Services'). This is probably what you're looking for unless you'd like to implement the code for finding the secret in the request, getting the client, and validating the secret it in the same class.
The implementation of IClientSecretValidator just orchestrates the usage of the ISecretParser (the default implementation of this interface will find the client secret on the incoming request), the IClientStore (the chosen implementation of this interface will retrieve the configured client so the actual secret can be retrieved), and the ISecretValidator (the default implementation of this interface will validate the secret in the request against the 'parsedSecret' retrieved from the client).
To answer your initial question, you can't add your own ClientSecretValidator wit the builder, although you can replace the existing one. To do this, you'd have to do a call similar to services.Replace(ServiceDescriptor.Transient<IClientSecretValidator, CustomClientSecretValidator>());
More information on replacing services that are already registered in ASP.NET Core's dependency injection system can be found here