Schlagwort-Archive: Kubernetes

How to use a per Service SSL certificate in a CSE TKGm cluster

Using VMware Cloud Director and the Container Service extension you can use kubectl expose to create a service of type Load Balancer.

The Kubernetes clusters, created by the Container Service extension, can leverage the NSX Advanced Load Balancer (formerly know as AVI Load Balancer). The integration, that is done via the Cloud Controller Manager, support L4 load balancing. With the latest version of CCM, you are now be able to define a certificate per service created.


To follow the following steps, I assume that CSE 3.1.2 is deployed (CSE Installation) , NSX Advanced deployed and configured to be used by CSE (Enable NSX Advanced Load Balancer in VCD).

First of all, you have to check if CCM version 1.1.0 is deployed.

kubectl get deployment vmware-cloud-director-ccm -n kube-system

apiVersion: apps/v1
kind: Deployment
  annotations: "1" |
  creationTimestamp: "2022-01-31T17:00:35Z"
  generation: 1
    app: vmware-cloud-director-ccm
  name: vmware-cloud-director-ccm
  namespace: kube-system
  resourceVersion: "826"
  uid: 9c0ec466-03f1-41c4-81f2-ee14075c7286
  progressDeadlineSeconds: 600
  replicas: 1
  revisionHistoryLimit: 2
      app: vmware-cloud-director-ccm
      maxSurge: 25%
      maxUnavailable: 25%
    type: RollingUpdate
      annotations: ""
      creationTimestamp: null
        app: vmware-cloud-director-ccm
            - matchExpressions:
              - key:
                operator: Exists
      - command:
        - /opt/vcloud/bin/cloud-provider-for-cloud-director
        - --cloud-provider=vmware-cloud-director
        - --cloud-config=/etc/kubernetes/vcloud/vcloud-ccm-config.yaml
        - --allow-untagged-cloud=true
        imagePullPolicy: IfNotPresent
        name: vmware-cloud-director-ccm
        resources: {}
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
        - mountPath: /etc/kubernetes/vcloud
          name: vcloud-ccm-config-volume
        - mountPath: /etc/kubernetes/vcloud/basic-auth
          name: vcloud-ccm-vcloud-basic-auth-volume
      dnsPolicy: Default
      hostNetwork: true
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      serviceAccount: cloud-controller-manager
      serviceAccountName: cloud-controller-manager
      terminationGracePeriodSeconds: 30
      - effect: NoSchedule
        value: "true"
      - key: CriticalAddonsOnly
        operator: Exists
      - effect: NoSchedule
      - configMap:
          defaultMode: 420
          name: vcloud-ccm-configmap
        name: vcloud-ccm-config-volume
      - name: vcloud-ccm-vcloud-basic-auth-volume
          defaultMode: 420
          secretName: vcloud-basic-auth
  availableReplicas: 1
  - lastTransitionTime: "2022-01-31T17:02:50Z"
    lastUpdateTime: "2022-01-31T17:02:50Z"
    message: Deployment has minimum availability.
    reason: MinimumReplicasAvailable
    status: "True"
    type: Available
  - lastTransitionTime: "2022-01-31T17:00:35Z"
    lastUpdateTime: "2022-01-31T17:02:50Z"
    message: ReplicaSet "vmware-cloud-director-ccm-b5d58cd57" has successfully progressed.
    reason: NewReplicaSetAvailable
    status: "True"
    type: Progressing
  observedGeneration: 1
  readyReplicas: 1
  replicas: 1
  updatedReplicas: 1

Search for:

Version 1.1.0.latest is needed for the following steps.

A little bit of background on SSL load Balancers

When creating a Load Balancer of encrypted traffic, you have to decide where your encryption endpoint should be hosted. 

We differentiate between two possible architecture:

  • SSL Termination on the Load Balancer
  • SSL Passthrough

Here you can find more details on the different SSL Load Balancer Architectures. 

In our use-case, exposing SSL workloads, running on a TKGm cluster created by CSE, SSL termination is the supported architecture.  

We need to create a NSX Advanced Load Balancer with an SSL certificate for the endpoint. The traffic will be forwarded from the Load Balancer as http traffic to the containers.

How to configure a service using SSL termination and a custom SSL-certificate

I will show in the following, how to expose a NGINX deployment using https.

First of all, you have to create a deployment:

 kubectl create deployment nginx --image=nginx --replicas=2

To expose a service using SSL termination you need to add the following annotation to your service definition:

annotations: "443" "my-service-cert"

You need to replace my-service-cert by the name of your certificate.

The easiest way to create a service accordingly, is to run kubectlwith the --dry-runoption: 

kubectl expose deployment nginx --type=LoadBalancer --port=443 --targetPort=80 --dry-run -o yaml > nginx-svc.yaml

After adding the annotations, your nginx-svc.yaml should like look the following:

apiVersion: v1
kind: Service
annotations: "443" "my-service-cert"
app: nginx
name: nginx
- port: 443
protocol: TCP
targetPort: 80
app: nginx
type: LoadBalancer
loadBalancer: {}

Excute kubectl apply -f nginx-svc.yaml and you are done:



VMware Cloud director 10.2 is there

VMware Cloud Director 10.2 is there! This is a big release and a big step forward.

I played already with Cloud Director 10.2 for a while and this is a big release with lots of improvements:

  • NSX-T integration: The NSX-T integration was significantly improved so that NSX-V and NSX-T reached feature parity! One of my personal highlights are the support of VRF´s and the AVI load integration
  • Support of vSphere with Tanzu in Cloud Director: VMware Cloud Director supports as of now vSphere with Tanzu integration. It is possible to enable self-service creation of TKG clusters and management out of VMware Cloud Director 10.2

Please stay tuned, I will publish a series of blog posts on the integration of vSphere with Tanzu in VMware Cloud Director very soon!

For your reference: