ASP.NET Core Middleware for gRPC Service that routes requests to a different gRPC service

Summary

I am using C# with .NET 6 (or 8) trying to create basically a more-capable gRPC proxy service that can receive a gRPC request, wrap that gRPC inside of another one, and then forward that gRPC(gRPC) along to a different gRPC service that is equipped to handle it. It's all a bit weird I will admit.

What I've tried and the problems I've encountered

I've tried a couple of different approaches for this but I am currently trying to use a Middleware that will do the following:

Intercept (not be confused with gRPC interceptors) the original gRPC
Wrap the original gRPC in another gRPC
Change the context.Request.Path to point to the service I really want it to be processed by
Call await _next(context) and eventually ASP.NET will call the service I switched it out to

Unfortunately, by the time my Middleware is called the context.Features.Get<IEndpointFeature>() feature is set which points to the original gRPC service that I no longer want to process this request. Or if I don't define a matching service for the received gRPC, I get an IEndpointFeature as follows:

What I would like to do is change the IEndpointFeature.Endpoint to point to an Endpoint for my other gRPC service but I am not sure how to create one properly in my Middleware. I have looked at some of the gRPC for .NET source and it seems like all of the code I'd need is marked internal so I'd have to re-implement a whole bunch of logic for this myself. It seems like there should be an easier way to either create my own Endpoint instance or to get my Middleware to run before the built-in Middleware that is setting this IEndpointFeature in the first place.

Note that if I simply wanted to re-route the request as-is, that may be easier but I am actually needing to manipulate the gRPC request when it is received as well.

This is what I have in my Program.cs where you can see I don't explicitly enable any other routing middleware... I suspect the middleware that sets this IEndpointFeature may be part of the builder.Services.AddGrpc() call:

using service.Services;

WebApplicationBuilder builder = WebApplication.CreateBuilder(args);

// Additional configuration is required to successfully run gRPC on macOS.
// For instructions on how to configure Kestrel and gRPC clients on macOS, visit https://go.microsoft.com/fwlink/?linkid=2099682

// Add services to the container.
builder.Services.AddGrpc();

WebApplication app = builder.Build();

app.UseMiddleware<MyRouteChangingMiddleware>();

// Configure the HTTP request pipeline.
app.MapGrpcService<GreeterServiceV1>();  // This is the 'original' gRPC service
app.MapGrpcService<GreeterServiceV2>();  // This is service I want to re-route to
app.MapGet("/", () => "Communication with gRPC endpoints must be made through a gRPC client. To learn how to create a client, visit: https://go.microsoft.com/fwlink/?linkid=2086909");

app.Run();

ASP.NET Core Rewrite Middleware (update 10/26/2023): I found the RewriteMiddleware in the ASP.NET Core source which has the following comment/code in it that seems like it is eluding to the same problem I am encountering:

                // An endpoint may have already been set. Since we're going to re-invoke the middleware pipeline we need to reset
                // the endpoint and route values to ensure things are re-calculated.
                context.SetEndpoint(endpoint: null);
                var routeValuesFeature = context.Features.Get<IRouteValuesFeature>();
                if (routeValuesFeature is not null)
                {
                    routeValuesFeature.RouteValues = null!;
                }
                return _options.BranchedNext(context);

So I updated my own middleware to follow this example trying to re-invoke the middleware pipeline but unfortunately, after clearing the route and IEndpointFeature it does not seem to re-invoke whichever Microsoft middleware is responsible for setting those features and I end up with an "Unimplemented" response returned to my client.

Solution

I ended up figuring out a different way to redirect the incoming request to a different service endpoint without re-running the entire middleware pipeline. However, it still did not end up using this approach because I needed to also modify the gRPC service response in my middleware. Since I was calling _next(), my understanding is that I should not be writing to the response stream myself. Instead, I need to use a terminal middleware that does not call _next() but instead writes the response itself.

Still, I figure this may help others if their use-case is different so I'll share what I did to redirect the incoming gRPC request to a different endpoint than what it normally would've been sent to...

First, I used an extension method to inject my middleware (called TunnelingMiddleware) into my service container -- this is almost a direct copy from the RewriteMiddleware I mentioned in my question:

public static class TunnelingMiddlewareExtensions
{
    public static IApplicationBuilder UseTunneling(this IApplicationBuilder app)
    {
        ArgumentNullException.ThrowIfNull(app);
        return AddTunnelingMiddleware(app);
    }

    private static IApplicationBuilder AddTunnelingMiddleware(IApplicationBuilder app)
    {
        // Only use this path if there's a global router (in the 'WebApplication' case).
        if (app.Properties.TryGetValue(RerouteHelper.GlobalRouteBuilderKey, out var routeBuilder) && routeBuilder is not null)
        {
            return app.Use(next =>
            {
                RequestDelegate newNext = RerouteHelper.Reroute(app, routeBuilder, next);
                Channel<TunneledRequest> tunneledRequestChannel = app.ApplicationServices.GetRequiredService<Channel<TunneledRequest>>();
                ILoggerFactory loggerFactory = app.ApplicationServices.GetRequiredService<ILoggerFactory>();
                ILogger<TunnelingMiddleware> logger = loggerFactory.CreateLogger<TunnelingMiddleware>();

                return new TunnelingMiddleware(next, tunneledRequestChannel, logger, app).InvokeAsync;
            });
        }

        return app.UseMiddleware<TunnelingMiddleware>();
    }
}

Notice this passes in the IApplicationBuilder app to the TunnelingMiddleware constructor.

My TunnelingMiddleware class and constructor then looked like this:

public class TunnelingMiddleware
{
    private readonly RequestDelegate _next;
    private readonly Channel<TunneledRequest> _tunneledRequestChannel;
    private readonly ILogger<TunnelingMiddleware> _logger;
    private readonly Endpoint _tunnelServiceEndpoint;

    public TunnelingMiddleware(RequestDelegate next,
        Channel<TunneledRequest> tunneledRequestChannel,
        ILogger<TunnelingMiddleware> logger,
        IApplicationBuilder applicationBuilder)
    {
        _next = next;
        _tunneledRequestChannel = tunneledRequestChannel;
        _logger = logger;

        // This is the relevant portion here... You have to get the EndpointDataSource and then look through the endpoints in there to find your desired endpoint.
        EndpointDataSource dataSource = applicationBuilder.ApplicationServices.GetRequiredService<EndpointDataSource>();

        // In my case it a service I called TunnelServiceV1 with a method called SendRequestThroughTunnel
        IEnumerable<Endpoint> tunnelEndpoints = dataSource.Endpoints
            .Where((endpoint) =>
            {
                GrpcMethodMetadata? metadata = endpoint.Metadata.GetMetadata<GrpcMethodMetadata>();
                if (metadata != null &&
                    metadata.ServiceType == typeof(TunnelServiceV1) &&
                    string.Equals(metadata.Method.Name, "SendRequestThroughTunnel", StringComparison.OrdinalIgnoreCase))
                {
                    return true;
                }
                else
                {
                    return false;
                }
            });

        // I save this off and use it later in InvokeAsync:
        _tunnelServiceEndpoint = tunnelEndpoints.First();
    }

Then later in my InvokeAsync method:

    public async Task InvokeAsync(HttpContext context)
    {
            // Redirect to the Tunnel service's SendRequestThroughTunnel endpoint.
            context.Request.Path = "/tunnel.v1.RequestTunnel/SendRequestThroughTunnel";

            IHttpRequestFeature? httpRequestFeature = context.Features.Get<IHttpRequestFeature>();
            if (httpRequestFeature != null)
            {
                httpRequestFeature.RawTarget = "/tunnel.v1.RequestTunnel/SendRequestThroughTunnel";
            }

            // Set the endpoint to my Tunnel service endpoint I found in the constructor
            context.SetEndpoint(endpoint: _tunnelServiceEndpoint);

            // Proceed to next middleware or endpoint delegate...
            await _next(context);
    }