Cluster

The kubernetes_cluster resource allows you to create immutable Kubernetes clusters running in Docker containers using K3s.

Examples

The following example creates a single node Kubernetes cluster and sets an output to the config path.

Minimal Example

resource "network" "cloud" {
  subnet = "10.5.0.0/16"
}

resource "k8s_cluster" "k3s" {
  network {
    id = resource.network.cloud.meta.id
  }
}

output "KUBECONFIG" {
  value = resource.k8s_cluster.k3s.kube_config.path
}

Full Example

The following example creates a Kubernetes cluster and applies a Helm chart from a remote repository, a local Kubernetes deployment, and configures ingress resources for both the Helm chart and local deployment.

resource "network" "cloud" {
  subnet = "10.5.0.0/16"
}

resource "k8s_cluster" "k3s" {
  network {
    id = resource.network.cloud.meta.id
  }

  copy_image {
    name = "shipyardrun/connector:v0.1.0"
  }
}

resource "k8s_config" "fake_service" {
  cluster = resource.k8s_cluster.k3s

  paths = ["./fake_service.yaml"]

  health_check {
    timeout = "240s"
    pods    = ["app.kubernetes.io/name=fake-service"]
  }
}

resource "helm" "vault" {
  cluster = resource.k8s_cluster.k3s

  repository {
    name = "hashicorp"
    url  = "https://helm.releases.hashicorp.com"
  }

  chart   = "hashicorp/vault"
  version = "v0.18.0"

  values = "./helm/vault-values.yaml"

  health_check {
    timeout = "240s"
    pods    = ["app.kubernetes.io/name=vault"]
  }
}

resource "ingress" "vault_http" {
  port = 18200

  target {
    resource = resource.k8s_cluster.k3s
    port = 8200

    config = {
      service   = "vault"
      namespace = "default"
    }
  }
}

resource "ingress" "fake_service" {
  port = 19090

  target {
    resource = resource.k8s_cluster.k3s
    port = 9090

    config = {
      service   = "fake-service"
      namespace = "default"
    }
  }
}

output "VAULT_ADDR" {
  value = "http://${resource.ingress.vault_http.local_address}"
}

output "KUBECONFIG" {
  value = resource.k8s_cluster.k3s.kube_config.path
}

Image Caching

Kubernetes clusters do not share the local machines Docker image cache. Each node in a cluster has it's own unqiue cache.

To save bandwidth all containers launched in the Kubernetes cluster pulled through an image cache that runs in Docker. After the first pull all images are subsequently pulled from the image cache not the public internet. This cache is global to all Nomad and Kubernetes clusters within the sandbox.

For more information on the image cache see the container_registry resource.