GRPC Permission Denied

Question

I am building application in Asp Net Core web API with microservice architecture. I want to make request/response call from one microservice to another using gRPC.

the problem is that the following code works on localhost but I can't deploy it on AWS.

This is my code for gRPC Server:

Proto file

syntax = "proto3";

option csharp_namespace = "Infrastructure";

//package CompanyManagment;

service CompanyProtoService{
    rpc GetCompanyLogos(CompanyLogoRequest) returns (stream CompanyLogoResponse);
}


message CompanyLogoRequest {
    repeated string CompanyId = 1;
}

message CompanyLogoResponse {
    string Logo = 1;
}

Proto Service

public class CompanyService : CompanyProtoService.CompanyProtoServiceBase
{
    private readonly ICompanyRepository _companyRepository;
    public CompanyService(ICompanyRepository companyRepository) 
    {
        _companyRepository = companyRepository;
    }

    public override async Task GetCompanyLogos(CompanyLogoRequest request, IServerStreamWriter<CompanyLogoResponse> responseStream, ServerCallContext context)
    {
        List<CompanyEntity> companies = await _companyRepository.GetCompanies();

        Dictionary<string, string> companyLogos = companies
            .GroupBy(company => company.Id.ToString())
            .ToDictionary(group => group.Key, group => group.First().LogoUrl!);

        List<string> logos = request.CompanyId.ToList()
            .Select(companyId => companyLogos.TryGetValue(companyId, out string logo) ? logo : "DefaultLogo.webp")
            .ToList();

        foreach (var logo in logos) 
        {
            CompanyLogoResponse response = new() { Logo = logo};
            await responseStream.WriteAsync(response);
        }
    }
}

Program.cs

    public static IServiceCollection AddInfrastructure(
        this IServiceCollection services,
        ConfigurationManager configuration) 
    {

        services.AddGrpc();

        return services;
    }


var app = builder.Build();
{
    app.UseCors(builder =>
    {
        builder.AllowAnyOrigin().AllowAnyMethod().AllowAnyHeader();
    });

    app.UseExceptionHandler(HttpContextItemKeys.Error);

    app.UseHttpsRedirection();

    app.UseResponseCompression();

    app.UseAuthentication();
    app.UseAuthorization();

    
    app.UseRouting();

    app.UseEndpoints(endpoints =>
    {
        endpoints.MapGrpcService<CompanyService>();
    });

    app.MapControllers();

    app.Run();
}

In project csproj I also have

<ItemGroup>
    <Protobuf Include="Protos\Company.proto" GrpcServices="Server" />
  </ItemGroup>

Now Client Side, this is my code of request.

            var channel = GrpcChannel.ForAddress("URL HERE"); 

            var client = new CompanyProtoService.CompanyProtoServiceClient(channel);

            CompanyLogoRequest recommendedCompanyLogoRequest = new() { CompanyId = { recomended.Select(x => x.CompanyId.ToString()).ToList() } };

            using (var call = client.GetCompanyLogos(recommendedCompanyLogoRequest))
            {
                while (await call.ResponseStream.MoveNext())
                {
                    recomendedLogos.Add(call.ResponseStream.Current.Logo);
                }

My main problem when I test the code from local host everything works fine but when I deploy it on AWS and try to reach the gRPC server from client side with real URL it give me the error:

Status(StatusCode="PermissionDenied", Detail="Bad gRPC response. HTTP status code: 403")'

I don't know what am I missing my DevOps told me that everything is correct in his side and the problem is in back-end. do you have any suggestions what can be problem?

I have tried everything googled it and can't find similar problem. also when I changed the real url to some random one, the error changed to "The gRPC server does not exists on this URL". so from that I guess that my URL is correct. but can't find the solution to give access to client.

Answer 1

I think I had a very similar issue about a year ago (PermissionDenied error), and solved it by setting up TLS with valid certificates. Localhost worked fine, in our network it only worked after setting up encryption. I think you need some special configuration to NOT use encryption with gRPC.

Here it says in the Microsoft Documentation: The ASP.NET Core gRPC template and samples use TLS by default. and: When an HTTP/2 endpoint is configured without TLS, the endpoint's ListenOptions.Protocols must be set to HttpProtocols.Http2. HttpProtocols.Http1AndHttp2 can't be used because TLS is required to negotiate HTTP/2. Without TLS, all connections to the endpoint default to HTTP/1.1, and gRPC calls fail. Also checkout this Github issue: gRPC doesn't work without HTTPS. It mentions that gRPC uses https per default, an that you need to switch it off if you don't want to use encryption, with the following line on the client-side:

AppContext.SetSwitch("System.Net.Http.SocketsHttpHandler.Http2UnencryptedSupport", true);

The same code snippet is also mentioned in the Microsoft Documentation in the article Call insecure gRPC services with .NET Core client. Other option of course would be to use HTTPS and set up encryption with valid certificates. Hope this helps.

EDIT: Maybe you can configure the Kestrel on the server side to not use HTTP2 and TLS, here is a good documentation on how to configure the Kestrel, f.e.:

builder.WebHost.ConfigureKestrel(opt =>
{
    int port = 5001; // your port here
    opt.ListenAnyIP(port, listenOptions =>
    {
        // configure Kestrel for HTTP2, not the default HTTPS2 (depends if you want to use HTTP1 or HTTP2)
        listenOptions.Protocols = HttpProtocols.Http2;
    });
});

Also checkout this github repon issue on configuring the Kestrel with HTTPS.