Using multiple kubeconfig files and how to merge to a single

Reference: Kubernetes Documentation

When it’s the case that you are managing multiple Kubernetes clusters, you will have to do with multiple kubeconfig files.
There are multiple ways that you can deal with this, my favorite is to keep the files separate, you can achive this by:

Creating a folder and move the kubeconfig files there.

mkdir ~/.kube/clusters
mv /path/cluster1.config /path/cluster2.config ~/.kube/clusters

Add the $KUBECONFIG environment variable to your ~/.bashrc, the value has to be “/path/cluster1.config:/path/cluster2.config”.

export KUBECONFIG=$(find ~/.kube/clusters -type f | sed ':a;N;s/\n/:/;ba')

Start a new bash terminal in order to take effect or “source ~/.bashrc” in the current.

Check, you can see both clusters and of course you can switch to the one that you wish to use.

kubectl config get-clusters
NAME
cluster1
cluster2

Other way would be to merge the multiple kubeconfig files in one and store it in ~/.kube/config the default location that it’s used when the $KUBECONFIG environment variable it’s not set.

KUBECONFIG="/path/cluster1.config:/path/cluster2.config"
kubectl config view --flatten > ~/.kube/config

That being said, I prefer the first one because it makes life easier when adding or removing a cluster is needed by simply adding or removing files.

Understanding Kubernetes ConfigMap with examples

ConfigMaps are objects that help us to store non-confidential data in key-value pairs in order to separate configuration from application.
They should not be used to store sensitive data for such tasks you should use Secrets, also they are not designed to store large amount of data they cannot exceed 1 MiB.
In this article, we will discuss how to create ConfigMaps, but also how to consume them in a Pod.
There are multiple methods that ConfigMaps can be injected in a Pod, we will cover the first three in this post, in all three cases the ConfigMap and Pod needs to run in the same namespace.

Methods:
Command and args
Environment variables
Mounted in the container via a read-only volume
Kubernetes API to read a ConfigMap via own code (not covered)

Resources
Kubernetes documentation

Command and args

We will create a ConfigMap and Pod in the default namespace named command-and-args-configmap and command-and-args-pod.

Create the ConfigMap, we have defined two keys, “message” and “sleep”.

apiVersion: v1
data:
  message: "Hello from ConfigMap"
  sleep: "10"
kind: ConfigMap
metadata:
  creationTimestamp: null
  name: command-and-args-configmap

Create the Pod, the container will use the two keys “message” and “sleep” loaded from the ConfigMap as args.
The “message” key will be used to print “Hello from ConfigMap” in a loop, while the “sleep” value will be used to sleep for 10 seconds between the messages.

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: command-and-args-pod
  name: command-and-args-pod
spec:
  containers:
  - image: busybox
    command: ["/bin/sh"]
    args: ["-c", "while true; do echo $(date) $(MESSAGE); echo 'Sleeping for $(SLEEP)'; sleep $(SLEEP); done"]
    env:
    - name: MESSAGE
      valueFrom:
        configMapKeyRef:
          name: command-and-args-configmap
          key: message
    - name: SLEEP
      valueFrom:
        configMapKeyRef:
          name: command-and-args-configmap
          key: sleep
    name: command-and-args-pod
    resources: {}
  dnsPolicy: ClusterFirst
  restartPolicy: Always
status: {}

Create the ConfigMap and Pod

kubectl create -f https://gitlab.com/oueta.com/oueta/-/raw/main/kubernetes/configmap/command-and-args-configmap.yaml
kubectl create -f https://gitlab.com/oueta.com/oueta/-/raw/main/kubernetes/configmap/command-and-args-pod.yaml

Check the results

kubectl logs command-and-args-pod
Sun Oct 3 15:59:52 UTC 2021 Hello from ConfigMap
Sleeping for 10
Sun Oct 3 16:00:02 UTC 2021 Hello from ConfigMap
Sleeping for 10
Sun Oct 3 16:00:12 UTC 2021 Hello from ConfigMap
Sleeping for 10
Sun Oct 3 16:00:22 UTC 2021 Hello from ConfigMap
Sleeping for 10

Environment variables

Create the ConfigMap with an example of connection information.

apiVersion: v1
data:
  db_host: "192.168.0.100"
  db_port: "6432"
kind: ConfigMap
metadata:
  creationTimestamp: null
  name: environment-variables-variable-configmap

Create the Pod, simpler but similar to the previous and we will do a different test.

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: environment-variables-variable-pod
  name: environment-variables-variable-pod
spec:
  containers:
  - image: busybox
    command: ["sleep", "3600"]
    env:
    - name: DB_HOST
      valueFrom:
        configMapKeyRef:
          name: environment-variables-variable-configmap
          key: db_host
    - name: DB_PORT
      valueFrom:
        configMapKeyRef:
          name: environment-variables-variable-configmap
          key: db_port
    name: environment-variables-variable-pod
    resources: {}
  dnsPolicy: ClusterFirst
  restartPolicy: Always
status: {}

Create the ConfigMap and Pod

kubectl apply -f https://gitlab.com/oueta.com/oueta/-/raw/main/kubernetes/configmap/environment-variables-variable-configmap.yaml
kubectl apply -f https://gitlab.com/oueta.com/oueta/-/raw/main/kubernetes/configmap/environment-variables-variable-pod.yaml

Test, let’s check the environment variables of our container.

kubectl exec environment-variables-variable-pod -- printenv | grep DB_
DB_HOST=192.168.0.100
DB_PORT=6432

Mount a read-only volume in the container

Create the ConfigMap, we have defined two keys, “description” and “certificate”.

apiVersion: v1
data:
  certificate: |
    "This is my
    test certificate"
  description: |
    "This is my 
    test description"
kind: ConfigMap
metadata:
  creationTimestamp: null
  name: volume-mounts-configmap

Create the Pod, the “/config” folder will be mounted and two files will be created, “description” and “certificate” each one containing the values that we have defined.

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: volume-mounts-pod
  name: volume-mounts-pod
spec:
  containers:
  - image: busybox
    command: ["sleep", "3600"]
    name: volume-mounts-pod
    volumeMounts:
      - name: config
        mountPath: "/config"
        readOnly: true
    resources: {}
  volumes:
    - name: config
      configMap:
        name: volume-mounts-configmap
  dnsPolicy: ClusterFirst
  restartPolicy: Always
status: {}

Create the ConfigMap and Pod

kubectl apply -f https://gitlab.com/oueta.com/oueta/-/raw/main/kubernetes/configmap/volume-mounts-configmap.yaml
kubectl apply -f https://gitlab.com/oueta.com/oueta/-/raw/main/kubernetes/configmap/volume-mounts-pod.yaml

Test, let’s check the mounted directory and the two files created.

kubectl exec volume-mounts-pod -- ls -l /config
total 0
lrwxrwxrwx    1 root     root            18 Oct  3 18:19 certificate -> ..data/certificate
lrwxrwxrwx    1 root     root            18 Oct  3 18:19 description -> ..data/description
kubectl exec volume-mounts-pod -- cat /config/certificate
"This is my
test certificate"
kubectl exec volume-mounts-pod -- cat /config/description
"This is my 
test description"

The ConfigMap changes are detected by kubelet and can be customized via configMapAndSecretChangeDetectionStrategy option.