Metrics-server

2020.09.19

1. What is Metrics-server ?

    - https://www.replex.io/blog/the-ultimate-guide-to-the-kubernetes-dashboard-how-to-install-and-integrate-metrics-server

      Metrics-server replaces Heapster as the primary cluster-wide metrics aggregator for Kubernetes with an integration into the Kubernetes dashboard.

      Metrics-server aggregates resource consumption data like CPU and memory usage for Kubernetes nodes, pods and containers.

      Since metrics-server shows only a snapshot of metrics in the cluster the metrics-scraper has been added to aggregate and store metrics over time.

          metric-duration of metrics-scraper is 15m(default)

     $ k get pod -n kubernetes-dashboard

     NAME                                        READY   STATUS    RESTARTS   AGE

     dashboard-metrics-scraper-c8b69c96c-s9dd9   1/1     Running   18         22d

     kubernetes-dashboard-6979c57f4c-tdxpb       1/1     Running   40         22d

     $

      metrics-server now uses webhook authentication so for it to work properly, make sure Kubelet has webhook authentication enabled.

        

    - https://github.com/kubernetes-sigs/metrics-server

      Metrics Server is a scalable, efficient source of container resource metrics for Kubernetes built-in autoscaling pipelines.

      You can use Metrics Server for:

          * CPU/Memory based horizontal autoscaling (learn more about Horizontal Pod Autoscaler)

             The Horizontal Pod Autoscaler automatically scales the number of Pods set based on observed CPU utilization

          * Automatically adjusting/suggesting resources needed by containers (learn more about Vertical Pod Autoscaler)

      Compatibility matrix:

 

2. Deployment Metrics-server

    a. Environments

       Kubernetes 1.15.12 

       Metrics-server 0.3.7

 

    b. Deployment

   $ mkdir -p k8s-oss/metrics-server && cd k8s-oss/metrics-server

   $ wget https://github.com/kubernetes-sigs/metrics-server/releases/download/v0.3.7/components.yaml

   $ vi components.yaml 

   …

   apiVersion: apps/v1

   kind: Deployment

   metadata:

     name: metrics-server

   …

           args:

             - --cert-dir=/tmp

             - --secure-port=4443

            - --kubelet-preferred-address-types=InternalIP    # appended this lines

             - --kubelet-insecure-tls                              # 

   …

   $ kubectl apply -f components.yaml 

   $ k get pod -n kube-system| grep metrics-server

   metrics-server-866c9b4dbb-75ndt    1/1     Running   0          2m38s

   $

 

    c. Test

   $ kubectl top nodes

   NAME       CPU(cores)   CPU%   MEMORY(bytes)   MEMORY%

   minikube   524m         26%    3275Mi          85%

   $

 

      Kubernetes dashboard

 

 

3. Trubleshooting

a. Problem:  Minikube에서 Metrics-server deploy후에도 에러 (metrics not available yet) 발생

b. Cause

        Kubelet service 타입이 Headless service로 선언되어 있어 Cluster IP가 없음

    $ kubectl logs metrics-server-866c9b4dbb-75ndt -n kube-system -f

    E0919 01:50:04.671901       1 reststorage.go:135] unable to fetch node metrics for node "minikube": no metrics known for node

    E0919 01:50:16.063602       1 manager.go:111] unable to fully collect metrics: unable to fully scrape metrics from source kubelet_summary:minikube: unable to fetch metrics from Kubelet minikube (minikube): Get https://minikube:10250/stats/summary?only_cpu_and_memory=true: dial tcp: lookup minikube on 10.96.0.10:53: no such host

    … 

    $ kubectl get svc kubelet -n kube-system

    NAME      TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)                        AGE

    kubelet   ClusterIP   None         <none>        10250/TCP,10255/TCP,4194/TCP   22d

    $ minikube ip

    192.168.64.10

    $

c. Solution

    $ kubectl delete -f https://github.com/kubernetes-sigs/metrics-server/releases/download/v0.3.7/components.yaml

    $ minikube addons enable metrics-server