Search code examples
goapache-kafkaconfluent-kafka-go

Unable to connect to Confluent Kafka using segmentio's kafka-go


I am able to connect to the confluent kafka cluster using the confluent cli but I am unable to do using the segmentio's kafka-go library. I get the below error.

with SASL: SASL handshake failed: EOF

Here is my function in go

package consumer

import (
    "context"
    "fmt"
    "log"
    "os"
    "time"

    "github.com/segmentio/kafka-go"
    "github.com/segmentio/kafka-go/sasl/plain"
)
func Consume(ctx context.Context) {
    // create a new logger that outputs to stdout
    // and has the `kafka reader` prefix
    l := log.New(os.Stdout, "kafka reader: ", 0)
    mechanism := plain.Mechanism{
        Username: "my-api-key",
        Password: "my-api-secret",
    }

    dialer := &kafka.Dialer{
        Timeout:       10 * time.Second,
        DualStack:     true,
        SASLMechanism: mechanism,
    }

    r := kafka.NewReader(kafka.ReaderConfig{
        Brokers: []string{brokerAddress}, // brokerAddress given in confluent cloud cluster settings. 
        Topic:   []string{"steps"}[0],
        // assign the logger to the reader
        Logger: l,
        Dialer: dialer,
    })
    for {
        // the `ReadMessage` method blocks until we receive the next event
        msg, err := r.ReadMessage(ctx)
        if err != nil {
            panic("could not read message " + err.Error())
        }
        // after receiving the message, log its value
        fmt.Println("received: ", string(msg.Value))
    }
}

I tried generating new keys, use my account username and password, reduce the partitions but nothing works.


Solution

  • it seems your server's TLS version is not accepted, you can force go-kafka to accept it using MinVersion:

    dialer := &kafka.Dialer{
            Timeout:       10 * time.Second,
            DualStack:     true,
            SASLMechanism: mechanism,
            TLS: &tls.Config{
                MinVersion: tls.VersionTLS12,
            },
        }