Terraform error connection refused: wrongly using localhost

Question

Have successfully been using terraform for about a month. Used terraform apply to create many resources on Azure (i.e. azurerm_kubernetes_cluster, kubernetes_service).

I am suddenly getting the below error regarding my kubernetes service.

╷
│ Error: Get "http://localhost/api/v1/namespaces/default/services/<service name>": dial tcp [::1]:80: connect: connection refused
│ 
│   with kubernetes_service.<service name>,
│   on main.tf line 132, in resource "kubernetes_service" "<service name>":
│  132: resource "kubernetes_service" "<service name>" {
│ 
╵

I can't figure out why suddenly this URL is referencing localhost, should be Azure. I am unsure what could have changed this.

• I am using the correct kubectl context.
• kubectl config view returns correct cluster details
• kubectl cluster-info returns correct azure endpoints
• Verified the service is available in Azure Portal
• Updated terraform to latest version
• Ran apply with debug:

2023-11-05T16:14:31.939-0500 [DEBUG] provider.terraform-provider-kubernetes_v2.23.0_x5: 2023/11/05 16:14:31 [INFO] Checking service <service name>
2023-11-05T16:14:31.940-0500 [DEBUG] provider.terraform-provider-kubernetes_v2.23.0_x5: 2023/11/05 16:14:31 [DEBUG] Kubernetes API Request Details:
2023-11-05T16:14:31.940-0500 [DEBUG] provider.terraform-provider-kubernetes_v2.23.0_x5: ---[ REQUEST ]---------------------------------------
2023-11-05T16:14:31.940-0500 [DEBUG] provider.terraform-provider-kubernetes_v2.23.0_x5: GET /api/v1/namespaces/default/services/<service name> HTTP/1.1
2023-11-05T16:14:31.940-0500 [DEBUG] provider.terraform-provider-kubernetes_v2.23.0_x5: Host: localhost
2023-11-05T16:14:31.940-0500 [DEBUG] provider.terraform-provider-kubernetes_v2.23.0_x5: User-Agent: HashiCorp/1.0 Terraform/1.6.3
2023-11-05T16:14:31.940-0500 [DEBUG] provider.terraform-provider-kubernetes_v2.23.0_x5: Accept: application/json, */*
2023-11-05T16:14:31.940-0500 [DEBUG] provider.terraform-provider-kubernetes_v2.23.0_x5: Accept-Encoding: gzip
2023-11-05T16:14:31.940-0500 [DEBUG] provider.terraform-provider-kubernetes_v2.23.0_x5
2023-11-05T16:14:31.940-0500 [DEBUG] provider.terraform-provider-kubernetes_v2.23.0_x5
2023-11-05T16:14:31.940-0500 [DEBUG] provider.terraform-provider-kubernetes_v2.23.0_x5: -----------------------------------------------------
2023-11-05T16:14:31.942-0500 [DEBUG] provider.terraform-provider-kubernetes_v2.23.0_x5: 2023/11/05 16:14:31 [DEBUG] Received error: &url.Error{Op:"Get", URL:"http://localhost/api/v1/namespaces/default/services/<service name>", Err:(*net.OpError)(0x14001036a50)}
2023-11-05T16:14:31.944-0500 [ERROR] provider.terraform-provider-kubernetes_v2.23.0_x5: Response contains error diagnostic: diagnostic_summary="Get \"http://localhost/api/v1/namespaces/default/services/<service name>\": dial tcp [::1]:80: connect: connection refused" tf_req_id=ab63a5a7-5bab-fc9a-c4f2-c7b102614920 tf_resource_type=kubernetes_service @caller=github.com/hashicorp/[email protected]/tfprotov5/internal/diag/diagnostics.go:55 tf_rpc=ReadResource diagnostic_detail="" tf_proto_version=5.3 tf_provider_addr=registry.terraform.io/hashicorp/kubernetes @module=sdk.proto diagnostic_severity=ERROR timestamp=2023-11-05T16:14:31.943-0500
2023-11-05T16:14:31.944-0500 [ERROR] vertex "kubernetes_service.<service name>" error: Get "http://localhost/api/v1/namespaces/default/services/<service name>": dial tcp [::1]:80: connect: connection refused
2023-11-05T16:14:31.944-0500 [ERROR] vertex "kubernetes_service.<service name> (expand)" error: Get "http://localhost/api/v1/namespaces/default/services/<service name>": dial tcp [::1]:80: connect: connection refused

Providers in terraform file

terraform {
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "3.72.0"
    }
    kubernetes = {
      source  = "hashicorp/kubernetes"
      version = "2.23.0"
    }
    cloudflare = {
      source  = "cloudflare/cloudflare"
      version = "4.15.0"
    }
    random = {
      source = "hashicorp/random"
    }
  }
}

provider "azurerm" {
  features {}
}

provider "cloudflare" {
  api_token = var.CLOUDFLARE_API_TOKEN
}

...

provider "kubernetes" {
  host                   = data.azurerm_kubernetes_cluster.cluster.kube_config.0.host
  client_certificate     = base64decode(data.azurerm_kubernetes_cluster.cluster.kube_config.0.client_certificate)
  client_key             = base64decode(data.azurerm_kubernetes_cluster.cluster.kube_config.0.client_key)
  cluster_ca_certificate = base64decode(data.azurerm_kubernetes_cluster.cluster.kube_config.0.cluster_ca_certificate)
}

...

Answer 1

Error: Get "http://localhost/api/v1/namespaces/default/services/<service name>": dial tcp [::1]:80: connect: connection refused

The error occurred because the Terraform plan indicated a planned replacement of the AKS cluster. consequently, when the planning process reached the Kubernetes provider configuration, there was no known AKS cluster endpoint, causing the provider to default connecting to localhost.

To resolve the issue and connect to your AKS cluster, you can use the following configuration in your provider "kubernetes.

provider "kubernetes" {
  config_path = "~/.kube/config"
}

The config_path specifies the path to your Kubernetes configuration file (~/.kube/config ). This file contains the all necessary configuration details for accessing AKS cluster, including the cluster's API server URL, client certificate, and client key.

If you specify the host, client_certificate, client_key, and cluster_ca_certificate information in the Kubernetes provider, you must execute terraform plan with the target as shown below. This configuration enables you to connect to your AKS cluster instead of the local host.

terraform plan -target *name of the your AKS cluster*

    terraform {
      required_providers {
        azurerm = {
          source  = "hashicorp/azurerm"
          version = "3.0.2"
        }
        kubernetes = {
          source  = "hashicorp/kubernetes"
          version = ">= 2.0.1"
        }
      }
    }
    
    provider "azurerm" {
      features {}
    }
    
    
    data "azurerm_kubernetes_cluster" "example" {
      name                = "Venkat-aks"
      resource_group_name = "existing-RG"
    }
    
    provider "kubernetes" {
      config_path = "~/.kube/config"
    }
    
    resource "kubernetes_deployment" "nginx" {
      metadata {
        name = "venkat-nginx"
        labels = {
          App = "venkatNginx"
        }
      }
    
      spec {
        replicas = 2
        selector {
          match_labels = {
            App = "VenkatNginx"
          }
        }
        template {
          metadata {
            labels = {
              App = "VenkatNginx"
            }
          }
          spec {
            container {
              image = "nginx:1.7.8"
              name  = "venkat"
    
              port {
                container_port = 80
              }
    
              resources {
                limits = {
                  cpu    = "0.5"
                  memory = "512Mi"
                }
                requests = {
                  cpu    = "250m"
                  memory = "50Mi"
                }
              }
            }
          }
        }
      }
    }
    
    resource "kubernetes_service" "nginx" {
      metadata {
        name = "nginx-venkat"
      }
      spec {
        selector = {
          App = kubernetes_deployment.nginx.spec.0.template.0.metadata[0].labels.App
        }
        port {
          port        = 80
          target_port = 80
        }
    
        type = "LoadBalancer"
      }
    }

Terraform apply

Once ran the terraform code, the deployment has been created.

Reference: dial tcp [::1]:80: connect: connection refused by apparentlymart