Search code examples
kubernetesudpcloudflarerancherrancher-desktop

Got "failed to sufficiently increase receive buffer size" error for cloudflared


I have a local Kubernetes created by Rancher Desktop. I have deployed a named Cloudflared Tunnel based on this tutorial.

I recently started to get error:

failed to sufficiently increase receive buffer size (was: 208 kiB, wanted: 2048 kiB, got: 416 kiB). See https://github.com/lucas-clemente/quic-go/wiki/UDP-Receive-Buffer-Size for details.

Note this does not affect the actual function of Cloudflared Tunnel, which is more like a warning. However, I do hope to fix it.

I have read the content in the link. However, this is running in a pod, so I am not sure how to fix it.

Below is full log:

2023-03-18 00:27:51.450Z 2023-03-18T00:27:51Z INF Starting tunnel tunnelID=c9aa4140-fee8-4862-a479-3c1faacbd816
2023-03-18 00:27:51.450Z 2023-03-18T00:27:51Z INF Version 2023.3.1
2023-03-18 00:27:51.450Z 2023-03-18T00:27:51Z INF GOOS: linux, GOVersion: go1.19.7, GoArch: arm64
2023-03-18 00:27:51.451Z 2023-03-18T00:27:51Z INF Settings: map[config:/etc/cloudflared/config/config.yaml cred-file:/etc/cloudflared/creds/credentials.json credentials-file:/etc/cloudflared/creds/credentials.json metrics:0.0.0.0:2000 no-autoupdate:true]
2023-03-18 00:27:51.453Z 2023-03-18T00:27:51Z INF Generated Connector ID: a2d07b8a-3343-4b28-bbb5-a0cc951d5093
2023-03-18 00:27:51.453Z 2023-03-18T00:27:51Z INF Initial protocol quic
2023-03-18 00:27:51.456Z 2023-03-18T00:27:51Z INF ICMP proxy will use 10.42.0.32 as source for IPv4
2023-03-18 00:27:51.456Z 2023-03-18T00:27:51Z INF ICMP proxy will use fe80::3c91:31ff:fe74:68ee in zone eth0 as source for IPv6
2023-03-18 00:27:51.456Z 2023-03-18T00:27:51Z WRN The user running cloudflared process has a GID (group ID) that is not within ping_group_range. You might need to add that user to a group within that range, or instead update the range to encompass a group the user is already in by modifying /proc/sys/net/ipv4/ping_group_range. Otherwise cloudflared will not be able to ping this network error="Group ID 65532 is not between ping group 1 to 0"
2023-03-18 00:27:51.456Z 2023-03-18T00:27:51Z WRN ICMP proxy feature is disabled error="cannot create ICMPv4 proxy: Group ID 65532 is not between ping group 1 to 0 nor ICMPv6 proxy: socket: permission denied"
2023-03-18 00:27:51.460Z 2023-03-18T00:27:51Z INF Starting Hello World server at 127.0.0.1:34545
2023-03-18 00:27:51.460Z 2023-03-18T00:27:51Z INF Starting metrics server on [::]:2000/metrics
2023-03-18 00:27:51.462Z 2023/03/18 00:27:51 failed to sufficiently increase receive buffer size (was: 208 kiB, wanted: 2048 kiB, got: 416 kiB). See https://github.com/lucas-clemente/quic-go/wiki/UDP-Receive-Buffer-Size for details.
2023-03-18 00:27:51.592Z 2023-03-18T00:27:51Z INF Connection ca329025-1f06-4f36-a8b2-27eda979345d registered with protocol: quic connIndex=0 ip=198.41.192.107 location=LAX
2023-03-18 00:27:51.760Z 2023-03-18T00:27:51Z INF Connection a25fdab3-adff-4be5-8eb3-c22d593dfbc5 registered with protocol: quic connIndex=1 ip=198.41.200.193 location=SJC
2023-03-18 00:27:52.670Z 2023-03-18T00:27:52Z INF Connection ef583d03-d123-4e8e-b8ad-37eed817d2da registered with protocol: quic connIndex=2 ip=198.41.200.113 location=SJC
2023-03-18 00:27:53.684Z 2023-03-18T00:27:53Z INF Connection 25609514-8c37-451e-b4ac-1fb9fba2b9b8 registered with protocol: quic connIndex=3 ip=198.41.192.37 location=LAX

Solution

  • My cloudflared pod is running under hm-cloudflared namesapce.

    So I can get the node name by:

    kubectl get pods -o wide -n hm-cloudflared
    
    NAME                           READY   STATUS             RESTARTS       AGE   IP           NODE                   NOMINATED NODE   READINESS GATES
    cloudflared-7cdf78df46-x5fb7   0/1     CrashLoopBackOff   13 (93s ago)   26m   10.42.0.82   lima-rancher-desktop   <none>           <none>
    

    Once getting the node name that the pod is running in, then you can ssh into the Kubernetes node by kubectl-plugins:

    # Install kubectl-plugins
    git clone https://github.com/luksa/kubectl-plugins $HOME/kubectl-plugins
    export PATH=$PATH:$HOME/kubectl-plugins
    
    # SSH into the Kubernetes node by kubectl-plugins
    kubectl ssh node lima-rancher-desktop
    

    Inside the Kubernetes node, based on https://github.com/quic-go/quic-go/wiki/UDP-Receive-Buffer-Size#non-bsd, I can increase the UDP receive buffer size by:

    sysctl -w net.core.rmem_max=2500000
    

    This command would increase the maximum receive buffer size to roughly 2.5 MB.

    Now just restart the cloudflared pod, the the issue should be gone! Hopefully it helps save some people time in future!