Monitor your Azure Kubernetes Cluster (AKS) with Prometheus and Grafana
Prometheus is a popular open source metric monitoring solution and is a part of the Cloud Native Compute Foundation. These tools have recently graduated from the Cloud Native Computing Foundation, which means that they are ready for production use and are first-class citizens among open-source monitoring tools.
With Helm, installing and managing Prometheus and Grafana on your AKS cluster has become much more straightforward. It not only provides you with a hardened, well-tested setup but also provides a lot of preconfigured dashboards to get you started right away.
While Prometheus offers the core monitoring capability of collecting and receiving logs, indexing them, and optimizing storage, Grafana provides a means for visualizing the metrics in graphical dashboards.
In this blog we’ll learn how easy it is to setup Prometheus and Grafana on AKS Cluster.
Prerequisites
- An AKS cluster provisioned and is in Running state- Azure CLI- az aks get-credentials -g {resource-group} -n {aks-cluster-name}- Install Helm for your respective OS - https://github.com/helm/helm/releases
Steps to configure Prometheus in AKS
- Define the Helm repo
We need to define the public Kubernetes chart repository in the Helm configuration
helm repo add stable https://charts.helm.sh/stable2. Install Prometheus Chart
With Helm, you do not need to worry about the internals of writing manifests and wiring things up. Helm provides you with battle-hardened, production-grade setups extensively tested across multiple scenarios and use cases.We will be using the stable/prometheus-operator chart for this. It's also a good idea to install Prometheus operator in a separate namespace.$kubectl create ns monitoring$helm install prometheus stable/prometheus-operator --namespace monitoring
Let’s validate if the prometheus operator was installed successfully.
bash-3.2$ kubectl --namespace monitoring get pods -l "release=prometheus"
NAME READY STATUS RESTARTS AGE
prometheus-prometheus-node-exporter-9fhrl 1/1 Running 0 21m
prometheus-prometheus-oper-operator-6d9c4bdb9f-hfpbb 2/2 Running 0 21mThis output confirms the pods are up and running.
3. Accessing the Prometheus Dashboard from the AKS Cluster.
We will be using the Kubernetes proxy to access it. The Kube proxy allows us to securely tunnel connections to Prometheus using TLS via the Kube API server.
bash-3.2$ kubectl port-forward -n monitoring prometheus-prometheus-oper-operator-6d9c4bdb9f-hfpbb-0 9090
Forwarding from 127.0.0.1:9090 -> 9090
Forwarding from [::1]:9090 -> 90904. Now access the Prometheus Dashboard from your local laptop/desktop; http://localhost:9090
We can use the Prometheus Query Language to query Kubernetes metrics in the old-fashioned geeky way.
For example, let’s look at the time series for the API Server request total:
4. Accessing the Grafana Dashboard from the AKS Cluster.
Let’s locate the grafana pod and then do a kubectl port-forward
bash-3.2$ kubectl get pod -n monitoring|grep grafana
prometheus-grafana-7c78857f5c-qfhld 2/2 Running 0 11m
bash-3.2$ kubectl port-forward -n monitoring prometheus-grafana-7c78857f5c-qfhld 3000
Forwarding from 127.0.0.1:3000 -> 3000
Forwarding from [::1]:3000 -> 3000
Handling connection for 3000
Handling connection for 3000
In order to login to the grafana dashboard we need the username and password. Retrieve these details by executing the following command
kubectl get secret -n monitoring prometheus-grafana -o yaml
Get the username and password.Access the Grafana dashboard on http://localhost:3000
Congratulations!. You are all set with Prometheus and Grafana on Azure Kubernetes Cluster (AKS)
Conclusion
We can discover a lot of other dashboards that gives us valuable insights about the health of AKS cluster, resource usage patterns of particular application pods, network traffic flow across the cluster, and much more.
Prometheus and Grafana are powerful tools for monitoring your Kubernetes cluster, and with Helm it’s so easy to setup and running in few minutes..