Kubernetes Deployments in Depth

Kubernetes deployments

Until now we have been creating pods directly but in reality we are not supposed to create pods directly. They should be created via higher level abstractions, such as deployments. This way, Kubernetes can add on useful features and higher level concepts to make your life easier. In this post, I will cover the basics of deployments and in the following posts will cover auto-scaling and rolling updates.

A deployment represents multiple replicas of a pod. Pods in a deployment are identical.

Within a deployment’s manifest you embed a pod template that has the same fields as a pod spec that we have written before. You describe a desired state in the deployment, for example, 5 pod replicas of redis version 10.1, and Kubernetes takes the steps required to bring the actual state of the cluster to the desired state you specify at a controlled rate. If for some reason one of the 5 replica pods is deleted, Kubernetes will automatically create a new one to replace it. You can also modify the desired state and kubernetes will converge the actual state to the desired state.

The Kubernetes master components include a deployment controller that takes care of managing the deployment.

Deploying a 3-tier application

We’ll use our microservices 3-tier application to demonstrate deployments. We’ll replace the individual pods with deployments that manage the pods for us.

We’ll start by creating a new namespace called deployments:

apiVersion: v1
kind: Namespace
metadata:
  name: deployments
  labels:
    app: counter
ubuntu@ip-10-0-128-5:~/src# kubectl create -f 5.1-namespace.yaml
namespace/deployments created
ubuntu@ip-10-0-128-5:~/src#

Now let’s see how a deployment is a template for creating pods. A template is used to create pod replicas. A replica is just a copy of a pod. Applications scale by creating more replicas. This will be more clear when you see the YAML files and as we demonstrate more features throughout this lesson.

Shown below, I am comparing the data tier manifest that uses kind: Pod to our current manifest that uses deployments kind: Deployment. I wanted to highlight how there are significant similarities with just a few changes.

The first change is that the apiVersion is now apps/v1. Higher level abstractions for managing applications are in their own API group and not part of the core API. The kind is set to Deployment. The metadata from the last post is directly applied to the deployment. Next comes the spec. The deployment spec contains deployment-specific settings, and also a pod template which has exactly the same pod spec as the previous post.

The replicas key sets how many pods to create for this particular deployment. Kubernetes will keep this number of pods running. For our example, set the value to 1 because there cannot be multiple redis containers.

Next there is a selector mapping. Just like we saw with services, deployments use label selectors to group pods that are in the deployment.

The match labels mapping should overlap with the labels declared in the pod template below. kubectl will complain if they don’t overlap.

The pod template metadata includes labels on the pods. Note that the metadata doesn’t need a name in the template because Kubernetes generates unique names for each pod in the deployment.

Similar changes are made to the app tier manifest and the support tier manifest. Mainly adding a selector and template for the deployment.

We can complete the same process for the app and support tiers. Also set replicas to 1 for both cases. 1 is actually the default value so it isn’t strictly required but it does emphasize that a deployment manages a group of identical replicas. Shown here are the manifests:

data-tier manifest

apiVersion: v1
kind: Service
metadata:
  name: data-tier
  labels:
    app: microservices
spec:
  ports:
  - port: 6379
    protocol: TCP # default
    name: redis # optional when only 1 port
  selector:
    tier: data
  type: ClusterIP # default
---
apiVersion: apps/v1 # apps API group
kind: Deployment
metadata:
  name: data-tier
  labels:
    app: microservices
    tier: data
spec:
  replicas: 1
  selector:
    matchLabels:
      tier: data
  template:
    metadata:
      labels:
        app: microservices
        tier: data
    spec: # Pod spec
      containers:
      - name: redis
        image: redis:latest
        imagePullPolicy: IfNotPresent
        ports:
          - containerPort: 6379

app-tier manifest

apiVersion: v1
kind: Service
metadata:
  name: app-tier
  labels:
    app: microservices
spec:
  ports:
  - port: 8080
  selector:
    tier: app
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: app-tier
  labels:
    app: microservices
    tier: app
spec:
  replicas: 1
  selector:
    matchLabels:
      tier: app
  template:
    metadata:
      labels:
        app: microservices
        tier: app
    spec:
      containers:
      - name: server
        image: lrakai/microservices:server-v1
        ports:
          - containerPort: 8080
        env:
          - name: REDIS_URL
            # Environment variable service discovery
            # Naming pattern:
            #   IP address: <all_caps_service_name>_SERVICE_HOST
            #   Port: <all_caps_service_name>_SERVICE_PORT
            #   Named Port: <all_caps_service_name>_SERVICE_PORT_<all_caps_port_name>
            value: redis://$(DATA_TIER_SERVICE_HOST):$(DATA_TIER_SERVICE_PORT_REDIS)
            # In multi-container example value was
            # value: redis://localhost:6379

support-tier manifest

apiVersion: apps/v1
kind: Deployment
metadata:
  name: support-tier
  labels:
    app: microservices
    tier: support
spec:
  replicas: 1
  selector:
    matchLabels:
      tier: support
  template:
    metadata:
      labels:
        app: microservices
        tier: support
    spec:
        containers:

        - name: counter
          image: lrakai/microservices:counter-v1
          env:
            - name: API_URL
              # DNS for service discovery
              # Naming pattern:
              #   IP address: <service_name>.<service_namespace>
              #   Port: needs to be extracted from SRV DNS record
              value: http://app-tier.deployments:8080

        - name: poller
          image: lrakai/microservices:poller-v1
          env:
            - name: API_URL
              # omit namespace to only search in the same namespace
              value: http://app-tier:$(APP_TIER_SERVICE_PORT)

Create deployments

ubuntu@ip-10-0-128-5:~/src# kubectl create -n deployments -f 5.2-data_tier.yaml -f 5.3-app_tier.yaml -f 5.4-support_tier.yaml
service/data-tier created
deployment.apps/data-tier created
service/app-tier created
deployment.apps/app-tier created
deployment.apps/support-tier created
ubuntu@ip-10-0-128-5:~/src#

Now lets get the deployment information. kubectl will display the three deployments and their replica information.

ubuntu@ip-10-0-128-5:~/src# kubectl get -n deployments deployments.
NAME           READY   UP-TO-DATE   AVAILABLE   AGE
app-tier       1/1     1            1           62s
data-tier      1/1     1            1           63s
support-tier   1/1     1            1           62s
ubuntu@ip-10-0-128-5:~/src#

So remember that horrible scenario I described at the end of the last post with respect to versioning and all that onto the end of the pods? Well we can see how deployments solve the same problem by asking K8s for the pods.

ubuntu@ip-10-0-128-5:~/src# kubectl -n deployments get pods
NAME                            READY   STATUS    RESTARTS   AGE
app-tier-748cdbdcc5-mzrz5       1/1     Running   1          8m37s
data-tier-599bc4fcf8-kl64m      1/1     Running   0          8m38s
support-tier-58d5d545b6-vrwt5   2/2     Running   0          8m37s
ubuntu@ip-10-0-128-5:~/src#

Note that each pod has hash at the end of it. The deployment adds uniqueness to the names automatically to identify pods of a particular deployment version. We can see how this works by running more than one replica in a deployment.

Scaling

We’ll use kubectl scale command for modifying replica counts. We’ll scale the number of replicas in the support tier to 5 which will cause the counter to increase five times more quickly.

ubuntu@ip-10-0-128-5:~/src# kubectl scale -n deployments deployment support-tier --replicas=5
deployment.extensions/support-tier scaled
ubuntu@ip-10-0-128-5:~/src#

The scale command is equivalent to editing replica value in the manifest file, and then running kubectl apply to apply the change. It’s just optimized for this one off-use case.

Now check the pods again to see what happens:

ubuntu@ip-10-0-128-5:~/src# kubectl -n deployments get pods
NAME                            READY   STATUS    RESTARTS   AGE
app-tier-748cdbdcc5-mzrz5       1/1     Running   1          15m
data-tier-599bc4fcf8-kl64m      1/1     Running   0          15m
support-tier-58d5d545b6-5dls2   2/2     Running   0          79s
support-tier-58d5d545b6-94kgd   2/2     Running   0          79s
support-tier-58d5d545b6-p7rn2   2/2     Running   0          79s
support-tier-58d5d545b6-qc686   2/2     Running   0          79s
support-tier-58d5d545b6-vrwt5   2/2     Running   0          15m
ubuntu@ip-10-0-128-5:~/src#

Note that the support tier pods continue to show two of two ready containers. This is because replicas replicate pods, not individual containers inside of a pod. Deployments ensure that the specified number of replica pods are kept running.

Deleting pods

So we can test this by deleting some pods and running a watch command to watch how Kubernetes brings them back to life.

ubuntu@ip-10-0-128-5:~/src# kubectl delete -n deployments pods support-tier-58d5d545b6-5dls2 support-tier-58d5d545b6-94kgd --wait=false
pod "support-tier-58d5d545b6-5dls2" deleted
pod "support-tier-58d5d545b6-94kgd" deleted
ubuntu@ip-10-0-128-5:~/src# watch -n 1 kubectl -n deployments get pods

Every 1.0s: kubectl -n deployments get pods

NAME                            READY   STATUS        RESTARTS   AGE
app-tier-748cdbdcc5-mzrz5       1/1     Running       1          20m
data-tier-599bc4fcf8-kl64m      1/1     Running       0          20m
support-tier-58d5d545b6-5brjh   2/2     Running       0          27s
support-tier-58d5d545b6-5dls2   2/2     Terminating   0          6m50s
support-tier-58d5d545b6-94kgd   2/2     Terminating   0          6m50s
support-tier-58d5d545b6-p7rn2   2/2     Running       0          6m50s
support-tier-58d5d545b6-qc686   2/2     Running       0          6m50s
support-tier-58d5d545b6-vrwt5   2/2     Running       0          20m
support-tier-58d5d545b6-w44h6   2/2     Running       0          27s

Shown here is the screenshot of how it brought them back up

Alright, so K8s can resurrect pods and make sure the application runs the intended number of pods.

As a side note I used the linux watch command with the -n 1 option to update the output every 1 second. kubectl get also supports watching by using the -w option kubectl get -n deployments pods -w and any changes are appended to the bottom of the output compared to overwriting the entire output with the linux watch command. You might prefer one over the other depending on what you are watching.

Now lets scale out the app-tier as well, and get the list of pods.

ubuntu@ip-10-0-128-5:~/src# kubectl scale -n deployments deployment app-tier --replicas=5
deployment.extensions/app-tier scaled
ubuntu@ip-10-0-128-5:~/src# kubectl get -n deployments pods
NAME                            READY   STATUS    RESTARTS   AGE
app-tier-748cdbdcc5-9jb46       1/1     Running   0          14s
app-tier-748cdbdcc5-mzrz5       1/1     Running   1          47m
app-tier-748cdbdcc5-qq5fc       1/1     Running   0          14s
app-tier-748cdbdcc5-r5hvn       1/1     Running   0          14s
app-tier-748cdbdcc5-v75j7       1/1     Running   0          14s
data-tier-599bc4fcf8-kl64m      1/1     Running   0          47m
support-tier-58d5d545b6-5brjh   2/2     Running   0          27m
support-tier-58d5d545b6-p7rn2   2/2     Running   0          33m
support-tier-58d5d545b6-qc686   2/2     Running   0          33m
support-tier-58d5d545b6-vrwt5   2/2     Running   0          47m
support-tier-58d5d545b6-w44h6   2/2     Running   0          27m
ubuntu@ip-10-0-128-5:~/src#

Load balancing

Kubernetes makes it really quite painless. Kubernetes did all the heavy lifting for us. Now let’s confirm that the app tier service is load balancing requests across the app tier pods. Describe the service

ubuntu@ip-10-0-128-5:~/src# kubectl describe -n deployments service app-tier
Name:              app-tier
Namespace:         deployments
Labels:            app=microservices
Annotations:       <none>
Selector:          tier=app
Type:              ClusterIP
IP:                10.100.247.45
Port:              <unset>  8080/TCP
TargetPort:        8080/TCP
Endpoints:         192.168.114.193:8080,192.168.114.197:8080,192.168.114.198:8080 + 2 more...
Session Affinity:  None
Events:            <none>
ubuntu@ip-10-0-128-5:~/src#

And observe the service now has five endpoints matching the number of pods in the deployment.

Thanks to label selectors the deployment and the service are able to track the pods in the tier.

Summary

  1. We used deployments to have kubernetes manage the pods in each application tier. By using deployments we get the benefits of having kubernetes monitor the actual number of pods and converge to our specified desired state.

  2. We also saw how to use kubectl scale to modify the desired number of replicas and kubernetes does what it takes to realize that desired state. We also saw how it seamlessly integrates with services that load balance across the deployment’s pods.

  3. One word of caution with scaling deployments is that you should make sure that the pods you are working with support horizontal scaling. That usually means that the pods are stateless as opposed to stateful.

  4. The data for the app tier is stored in the data tier and we can add as many app tier pods as we like because the state of the application is stored in the data tier.

  5. With our current setup we can’t scale the data tier out because that would create multiple copies of the application counter. However even if we never scale the data tier we still get the benefit of having kubernetes return the data tier to its desired state by using deployments.

  6. We also get more benefits when it comes to performing updates and rollbacks which we’ll see in a couple of lessons. So it still makes sense to use a deployment for the data tier and we rarely should directly be creating pods.

  7. Kubernetes has even more tricks up its sleeve when it comes to scaling. We arbitrarily scaled the deployment but in practice you would like to scale based on cpu load or some other metric to react to the current state of the system to make the best use of available resources. We will tackle that in the next post about Auto Scaling.