Instantiating a Pulumi Kubernetes provider inside a Pod

Instantiating a Pulumi Kubernetes provider inside a Pod

Usually, we call the Kubernetes API from within a Pod using kubectl or by exposing the API via kubectl proxy and making HTTP requests to it.

The Pulumi Kubernetes provider, when instantiated without any arguments, will behave exactly like kubectl. (For the details on how the configuration is discovered, see the Pulumi Kubernetes documentation.) Thus, if the Pod the Pulumi program is executed in is using a service account that has permission to access the Kubernetes API, the Kubernetes provider will be able to act as that service account:

import pulumi
import pulumi_kubernetes as k8s

k8s_provider = k8s.Provider("k8s-provider")

my_deployment = k8s.apps.v1.Deployment(...,
    opts=pulumi.ResourceOptions(provider=k8s_provider)

However, in a typical Pulumi pattern, we manage the Kubernetes cluster in one stack and to export its kubeconfig via pulumi.export() for other stacks to use. In this scenario, it can be desirable to always provide a kubeconfig to the Kubernetes provider. Thus, we need to generate a kubeconfig that is valid from within a Pod.

Based on this gist by Rich Adams, here's a utility function that generates a kubeconfig using the information available in the Pod:

import os
import pathlib


def get_kubeconfig():
    service_host = os.getenv("KUBERNETES_SERVICE HOST")
    service_port = os.getenv("KUBERNETES_SERVICE_PORT")
    service_scheme = (
        "http" if service_port in ("80", "8080", "8081") else "https"
    )
    server_url = f"{service_scheme}://{service_host}:{service_port}"

    service_account_dir = pathlib.Path(
        "/var/run/secrets/kubernetes.io/serviceaccount"
    )
    ca_crt_path = service_account_dir / "ca.crt"
    with open(service_account_dir / "token") as f:
        token = f.read()
    with open(service_account_dir / "namespace") as f:
        namespace = f.read()

    context = "my-context"

    return f"""apiVersion: v1
kind: Config
preferences: {{}}
current-context: {context}

clusters:
- name: myCluster
  cluster:
    server: {server_url}
    certificate-authority: {ca_crt_path}

users:
- name: podServiceAccount
  user:
    token: {token}

contexts:
- name: {context}
  context:
    cluster: myCluster
    namespace: {namespace}
    user: podServiceAccount
"""

Using this utility function, we can generate a kubeconfig on-the-fly when instantiating the Pulumi Kubernetes provider or pass it on as a stack output:

import pulumi
import pulumi_kubernetes as k8s

k8s_provider = k8s.Provider("k8s-provider", 
                            kubeconfig=get_kubeconfig())

pulumi.export("kubeconfig", get_kubeconfig())