Search code examples
kubernetesminiok3s

Minio deployment using kubernetes doesn't work as expected

I'm experimenting with kubernetes and a minio deployment. I have a k3s 4 node cluster, each one with 4 50GB disk. Following the instructions here I have done this:

  1. First I installed krew in order to install the minio and the directpv operators.

  2. I installed those two without a problem.

  3. I formatted every Available hdd in the node using kubectl directpv drives format --drives /dev/vd{b...e} --nodes k3s{1...4}

  4. I then proceed to make the deployment, first I create the namespace with kubectl create namespace minio-tenant-1, and then I actually create the tenant with:

    kubectl minio tenant create minio-tenant-1 --servers 4 --volumes 8 --capacity 10Gi --storage-class direct-csi-min-io --namespace minio-tenant-1

  5. The only thing I need to do then is expose the port to access, which I do with: kubectl port-forward service/minio 443:443 (I'm guessing it should be a better way to achieve this, as the last command isn't apparently permanent, maybe using a LoadBalancer or NodePort type services in the kubernetes cluster).

So far so good, but I'm facing some problems:

  • When I try to create an alias to the server using mc the prompt answer me back with:

mc: Unable to initialize new alias from the provided credentials. Get "https://127.0.0.1/probe-bucket-sign-9aplsepjlq65/?location=": x509: cannot validate certificate for 127.0.0.1 because it doesn't contain any IP SANs

I can surpass this with simply adding the --insecure option, but I don't know why it throws me this error, I guess is something how k3s manage the TLS auto-signed certificates.

  • Once created the alias (I named it test) of the server with the --insecure option I try to create a bucket, but the server always answer me back with:

    mc mb test/hello

    mc: <ERROR> Unable to make bucket \test/hello. The specified bucket does not exist.

So... I can't really use it... Any help will be appreciated, I need to know what I'm doing wrong.

Solution

  • Guided by information at the Minio documentation. You have to generate a public certificate. First of all generate a private key use command:

    certtool.exe --generate-privkey --outfile NameOfKey.key

    After that create a file called cert.cnf with content below:

    # X.509 Certificate options
#
# DN options

# The organization of the subject.
organization = "Example Inc."

# The organizational unit of the subject.
#unit = "sleeping dept."

# The state of the certificate owner.
state = "Example"

# The country of the subject. Two letter code.
country = "EX"

# The common name of the certificate owner.
cn = "Sally Certowner"

# In how many days, counting from today, this certificate will expire.
expiration_days = 365

# X.509 v3 extensions

# DNS name(s) of the server
dns_name = "localhost"

# (Optional) Server IP address
ip_address = "127.0.0.1"

# Whether this certificate will be used for a TLS server
tls_www_server

    Run certtool.exe and specify the configuration file to generate a certificate:

    certtool.exe --generate-self-signed --load-privkey NameOfKey.key --template cert.cnf --outfile public.crt

    And the end put the public certificate into:

    ~/.minio/certs/CAs/