keda, event-driven containers for Kubernetes
Recently during the Microsoft //build 2019 conference, KEDA was announced as a component bringing event-driven containers and functions to Kubernetes.
KEDA allows for fine grained autoscaling (including to/from zero) for event driven Kubernetes workloads. KEDA serves as a Kubernetes Metrics Server and allows users to define autoscaling rules using a dedicated Kubernetes custom resource definition.
You could find 6 samples to get started:
- Go + RabbitMQ
- Python Azure Functions + Kafka
- TypeScript Azure Functions + Kafka
- Azure Functions + Osiris
- Azure Functions + OpenShift 4
# Only install keda and not osiris for the purpose of this Azure Queue (not HTTP) sample: func kubernetes install \ --keda \ --namespace keda # See all the k8s resources deployed by the previous command: kubectl get all,customresourcedefinition \ -n keda # Deploy my hello-keda sample on a specific k8s namespace: func kubernetes deploy \ --name hello-keda \ --registry <your-docker-id> \ --namespace hello-keda # See all the k8s resources deployed by the previous command: kubectl get all,ScaledObject,Secret \ -n hello-keda # Watch the pods, deployments and hpas moving while adding items in the Azure Queue (you could even open 3 panes with [tmux](https://en.wikipedia.org/wiki/Tmux)): kubectl get pod -w kubectl get deploy -w kubectl get hpa -w # Build, tag and deploy a specific version and decoupling the build from the release: docker build . -t <your-docker-id>/hello-keda:1 docker push <your-docker-id>/hello-keda:1 func kubernetes deploy \ --name hello-keda \ --image-name <your-docker-id>/hello-keda:1
I also learned few stuffs around the
- It’s the key and connectionstring to access the Azure Queue, we don’t want it stored in the Docker image nor in the Git repo, right? That’s where the .dockerignore and .gitignore play an important role. The .dockerignore exclude the copy of the local.settings.json file into our Docker image when building the Docker image, that’s what we want, perfect!
- This AzureWebJobsStorage setting is stored as k8s Secret and then used but the k8s Deployment
- KEDA is a single modular component that is trying to do one thing well: provide event driven scale. It has no dependencies and works with any Kubernetes cluster.
- KEDA is directly looking at the event sources (for examples messages in an Azure Queue) and scale up pods based on the outstanding “events” (for examples length of an Azure queue)
- KEDA doesn’t use the Azure Functions runtime. It is an independent component that can scale up any Kubernetes deployment based on events. If the Kubernetes deployment happens to target a Functions container then that will be scaled out.
- There is some tooling in Azure Functions Core Tools (func kubernetes) to be able to easily deploy a Functions container which is scaled through KEDA but again the components are independent.
- Osiris and Virtual Kubelet alongside with HPA and KEDA look really promising by bringing the concept of Serverless containers on Kubernetes.
- KEDA project
- Azure webinar - Build Event-Driven Containers with Azure Functions on Kubernetes
- Microsoft //build 2019 - Serverless Kubernetes, KEDA, and Azure Functions
- Microsoft //build 2019 - Where should I host my code? Choosing between Kubernetes, Containers, and Serverless
Hope you enjoyed this blog article and this new (and experimental for now) open source project bringing more capabilities and more workloads on/with k8s.