Cluster

The kubernetes_cluster resource allows you to create immutable Kubernetes clusters running in Docker containers using K3s.

Image Caching

Kubernetes clusters do not share the local machines Docker image cache. Each node in a cluster has it's own unqiue cache.

To save bandwidth all containers launched in the Kubernetes cluster pulled through an image cache that runs in Docker. After the first pull all images are subsequently pulled from the image cache not the public internet. This cache is global to all Nomad and Kubernetes clusters created with Jumppad.

For more information on the image cache see the container_registry resource.

Attributes

Attributes are the properties of a resource that can be used to configure it.

Attribute	Description
Networks network required []NetworkAttachment	Attach to the correct network // only when Image is specified Network attaches the container to an existing network defined in a separate stanza. This block can be specified multiple times to attach the container to multiple networks.
Image image Image	Image defines a Docker image to use when creating the container. By default the kubernetes cluster resource will be created using the latest Jumppad container image.
Nodes nodes int	The number of nodes to create in the cluster.
Volumes volume required []Volume	Additional volume to mount to the server and client nodes. volume { source = "./mydirectory" destination = "/path_in_container" }
CopyImages copy_image required []Image	Docker image in the local Docker image cache to copy to the cluster on creation. This image is added to the Kubernetes clients docker cache enabling jobs to use images that may not be in the local registry. Jumppad tracks changes to copied images, should the image change running jumppad up would push any changes to the cluster automatically.
Ports port required []Port	A `port` stanza allows you to expose container ports on the local network or host. This stanza can be specified multiple times.
PortRanges port_range required []PortRange	A `port_range` stanza allows you to expose a range of container ports on the local network or host. This stanza can be specified multiple times. The following example would create 11 ports from 80 to 90 (inclusive) and expose them to the host machine.
Environment environment map[string]string	environment variables to set when starting the container An env stanza allows you to set environment variables in the container. This stanza can be specified multiple times. env { key = "PATH" value = "/usr/local/bin" }
Config config block ClusterConfig	Specifies the configuration for the Kubernetes cluster.
APIPort api_port int	Port to expose the Kubernetes API on the host. By default this uses the standard api port `443`; however, if you are running multiple kubernetes instances you will need to override this value.
ContainerName container_name string	The fully qualified resource name for the Kubernetes cluster, this value can be used to address the server from the Docker network. It is also the name of the Docker container. server.name.k8s-cluster.local.jmpd.in

Computed Attributes

These attributes are computed when the config is parsed and applied, and are therefor only known at parsetime or runtime.

Attribute	Description
Meta ID meta.id string	The full ID of the resource e.g. `resource.type.name`. This is computed from the full resource path: resource "type" "name" { ... }
Meta Type meta.type string	The type of the resource. This taken from the type label of the resource definition. resource "type" "name" { ... }
Meta Name meta.name string	The name of the resource. This taken from the name label of the resource definition. resource "type" "name" { ... }
KubeConfig kube_config KubeConfig	Details for the Kubenetes config file that can be used to interact with the cluster.
ConnectorPort connector_port int	The port where the Jumppad connector is exposed to the host, this property is requied by the ingress resource and is not generally needed when building blueprints.
ExternalIP external_ip string	Local IP address of the Nomad server, this property can be used to set the NOAMD_ADDR on the Jumppad client. output "K8S_ADDR" { value = "https://${resource.k8s_cluster.dev.external_ip}:${resource.k8s_cluster.dev.api_port}" }

Examples

Simple cluster

resource "network" "cloud" {
  subnet = "10.5.0.0/16"
}

resource "k8s_cluster" "k3s" {
  network {
    id = resource.network.cloud.meta.id
  }
}

output "KUBECONFIG" {
  value = resource.k8s_cluster.k3s.kube_config.path
}

Full Example

resource "network" "cloud" {
  subnet = "10.5.0.0/16"
}

resource "k8s_cluster" "k3s" {
  network {
    id = resource.network.cloud.meta.id
  }

  copy_image {
    name = "shipyardrun/connector:v0.1.0"
  }
}

resource "k8s_config" "fake_service" {
  cluster = resource.k8s_cluster.k3s

  paths = ["./fake_service.yaml"]

  health_check {
    timeout = "240s"
    pods    = ["app.kubernetes.io/name=fake-service"]
  }
}

resource "helm" "vault" {
  cluster = resource.k8s_cluster.k3s

  repository {
    name = "hashicorp"
    url  = "https://helm.releases.hashicorp.com"
  }

  chart   = "hashicorp/vault"
  version = "v0.18.0"

  values = "./helm/vault-values.yaml"

  health_check {
    timeout = "240s"
    pods    = ["app.kubernetes.io/name=vault"]
  }
}

resource "ingress" "vault_http" {
  port = 18200

  target {
    resource = resource.k8s_cluster.k3s
    port = 8200

    config = {
      service   = "vault"
      namespace = "default"
    }
  }
}

resource "ingress" "fake_service" {
  port = 19090

  target {
    resource = resource.k8s_cluster.k3s
    port = 9090

    config = {
      service   = "fake-service"
      namespace = "default"
    }
  }
}

output "VAULT_ADDR" {
  value = "http://${resource.ingress.vault_http.local_address}"
}

output "KUBECONFIG" {
  value = resource.k8s_cluster.k3s.kube_config.path
}