博客
关于我
强烈建议你试试无所不能的chatGPT,快点击我
k8s监控
阅读量:6441 次
发布时间:2019-06-23

本文共 13387 字,大约阅读时间需要 44 分钟。

注意时间同步

#(1)工作原理
node-exporter组件负责收集节点上的metrics监控数据,并将数据推送给prometheus, prometheus负责存储这些数据,grafana将这些数据通过网页以图形的形式展现给用户。
注意: 需要提前部署好metric server

#(2)部署node-exporter组件

1)node-exporter.yaml文件#cat node-exporter.yaml ---apiVersion: extensions/v1beta1kind: DaemonSetmetadata:    name: node-exporter    namespace: kube-system    labels:        k8s-app: node-exporterspec:    template:        metadata:            labels:                k8s-app: node-exporter        spec:            containers:            - image: registry.cn-hangzhou.aliyuncs.com/wangfang-k8s/node-exporter:latest                name: node-exporter                ports:                - containerPort: 9100                    protocol: TCP                    name: http---apiVersion: v1kind: Servicemetadata:    labels:        k8s-app: node-exporter    name: node-exporter    namespace: kube-systemspec:    ports:    - name: http        port: 9100        nodePort: 31672        protocol: TCP    type: NodePort    selector:        k8s-app: node-exporter

2)创建pod和service

kubectl apply -f node-exporter.yaml

3)验证

k8s监控

k8s监控

k8s监控

可以通过node-exporter对外暴露的端口访问到metric server

#(3)部署prometheus组件

1)prometheus配置文件

# cat configmap.yaml apiVersion: v1kind: ConfigMapmetadata:    name: prometheus-config    namespace: kube-systemdata:    prometheus.yml: |        global:            scrape_interval:     15s            evaluation_interval: 15s        scrape_configs:        - job_name: 'kubernetes-apiservers'            kubernetes_sd_configs:            - role: endpoints            scheme: https            tls_config:                ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt            bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token            relabel_configs:            - source_labels: [__meta_kubernetes_namespace, __meta_kubernetes_service_name, __meta_kubernetes_endpoint_port_name]                action: keep                regex: default;kubernetes;https        - job_name: 'kubernetes-nodes'            kubernetes_sd_configs:            - role: node            scheme: https            tls_config:                ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt            bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token            relabel_configs:            - action: labelmap                regex: __meta_kubernetes_node_label_(.+)            - target_label: __address__                replacement: kubernetes.default.svc:443            - source_labels: [__meta_kubernetes_node_name]                regex: (.+)                target_label: __metrics_path__                replacement: /api/v1/nodes/${1}/proxy/metrics        - job_name: 'kubernetes-cadvisor'            kubernetes_sd_configs:            - role: node            scheme: https            tls_config:                ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt            bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token            relabel_configs:            - action: labelmap                regex: __meta_kubernetes_node_label_(.+)            - target_label: __address__                replacement: kubernetes.default.svc:443            - source_labels: [__meta_kubernetes_node_name]                regex: (.+)                target_label: __metrics_path__                replacement: /api/v1/nodes/${1}/proxy/metrics/cadvisor        - job_name: 'kubernetes-service-endpoints'            kubernetes_sd_configs:            - role: endpoints            relabel_configs:            - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_scrape]                action: keep                regex: true            - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_scheme]                action: replace                target_label: __scheme__                regex: (https?)            - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_path]                action: replace                target_label: __metrics_path__                regex: (.+)            - source_labels: [__address__, __meta_kubernetes_service_annotation_prometheus_io_port]                action: replace                target_label: __address__                regex: ([^:]+)(?::\d+)?;(\d+)                replacement: $1:$2            - action: labelmap                regex: __meta_kubernetes_service_label_(.+)            - source_labels: [__meta_kubernetes_namespace]                action: replace                target_label: kubernetes_namespace            - source_labels: [__meta_kubernetes_service_name]                action: replace                target_label: kubernetes_name        - job_name: 'kubernetes-services'            kubernetes_sd_configs:            - role: service            metrics_path: /probe            params:                module: [http_2xx]            relabel_configs:            - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_probe]                action: keep                regex: true            - source_labels: [__address__]                target_label: __param_target            - target_label: __address__                replacement: blackbox-exporter.example.com:9115            - source_labels: [__param_target]                target_label: instance            - action: labelmap                regex: __meta_kubernetes_service_label_(.+)            - source_labels: [__meta_kubernetes_namespace]                target_label: kubernetes_namespace            - source_labels: [__meta_kubernetes_service_name]                target_label: kubernetes_name        - job_name: 'kubernetes-ingresses'            kubernetes_sd_configs:            - role: ingress            relabel_configs:            - source_labels: [__meta_kubernetes_ingress_annotation_prometheus_io_probe]                action: keep                regex: true            - source_labels: [__meta_kubernetes_ingress_scheme,__address__,__meta_kubernetes_ingress_path]                regex: (.+);(.+);(.+)                replacement: ${1}://${2}${3}                target_label: __param_target            - target_label: __address__                replacement: blackbox-exporter.example.com:9115            - source_labels: [__param_target]                target_label: instance            - action: labelmap                regex: __meta_kubernetes_ingress_label_(.+)            - source_labels: [__meta_kubernetes_namespace]                target_label: kubernetes_namespace            - source_labels: [__meta_kubernetes_ingress_name]                target_label: kubernetes_name        - job_name: 'kubernetes-pods'            kubernetes_sd_configs:            - role: pod            relabel_configs:            - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape]                action: keep                regex: true            - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_path]                action: replace                target_label: __metrics_path__                regex: (.+)            - source_labels: [__address__, __meta_kubernetes_pod_annotation_prometheus_io_port]                action: replace                regex: ([^:]+)(?::\d+)?;(\d+)                replacement: $1:$2                target_label: __address__            - action: labelmap                regex: __meta_kubernetes_pod_label_(.+)            - source_labels: [__meta_kubernetes_namespace]                action: replace                target_label: kubernetes_namespace            - source_labels: [__meta_kubernetes_pod_name]                action: replace                target_label: kubernetes_pod_name

2)创建rbac权限

# cat rbac.yaml apiVersion: rbac.authorization.k8s.io/v1kind: ClusterRolemetadata:    name: prometheusrules:- apiGroups: [""]    resources:    - nodes    - nodes/proxy    - services    - endpoints    - pods    verbs: ["get", "list", "watch"]- apiGroups:    - extensions    resources:    - ingresses    verbs: ["get", "list", "watch"]- nonResourceURLs: ["/metrics"]    verbs: ["get"]---apiVersion: v1kind: ServiceAccountmetadata:    name: prometheus    namespace: kube-system---apiVersion: rbac.authorization.k8s.io/v1kind: ClusterRoleBindingmetadata:    name: prometheusroleRef:    apiGroup: rbac.authorization.k8s.io    kind: ClusterRole    name: prometheussubjects:- kind: ServiceAccount    name: prometheus    namespace: kube-system

3)创建deployment文件

# cat prometheus-deployment.yaml ---apiVersion: apps/v1beta2kind: Deploymentmetadata:    labels:        name: prometheus-deployment    name: prometheus    namespace: kube-systemspec:    replicas: 1    selector:        matchLabels:            app: prometheus    template:        metadata:            labels:                app: prometheus        spec:            containers:            - image: registry.cn-hangzhou.aliyuncs.com/wangfang-k8s/prometheus:v2.0.0                name: prometheus                command:                - "/bin/prometheus"                args:                - "--config.file=/etc/prometheus/prometheus.yml"                - "--storage.tsdb.path=/prometheus"                - "--storage.tsdb.retention=24h"                ports:                - containerPort: 9090                    protocol: TCP                volumeMounts:                - mountPath: "/prometheus"                    name: data                - mountPath: "/etc/prometheus"                    name: config-volume                resources:                    requests:                        cpu: 100m                        memory: 100Mi                    limits:                        cpu: 500m                        memory: 2500Mi            serviceAccountName: prometheus                volumes:            - name: data                emptyDir: {}            - name: config-volume                configMap:                    name: prometheus-config

4)创建service文件

# cat prometheus.svc.yml ---kind: ServiceapiVersion: v1metadata:    labels:        app: prometheus    name: prometheus    namespace: kube-systemspec:    type: NodePort    ports:    - port: 9090        targetPort: 9090        nodePort: 30003    selector:        app: prometheus

5)创建pod,service,rbac和configmap;

kubectl apply -f .

6)验证

k8s监控

k8s监控

可以通过对外暴露的接口访问prometheus数据库;

k8s监控

查看状态是否成功连接到apiserver

k8s监控

能够查询到metric的api接口数据

k8s监控

#(4)部署grafana组件

1)准备grafana资源配置清单

# cat grafana.yaml apiVersion: extensions/v1beta1kind: Deploymentmetadata:    name: monitoring-grafana    namespace: kube-systemspec:    replicas: 1    template:        metadata:            labels:                task: monitoring                k8s-app: grafana        spec:            containers:            - name: grafana                #image: gcr.io/google_containers/heapster-grafana-amd64:v4.4.3                image: registry.cn-hangzhou.aliyuncs.com/wangfang-k8s/heapster-grafana-amd64:v4.4.3                imagePullPolicy: IfNotPresent                ports:                - containerPort: 3000                    protocol: TCP                volumeMounts:                - mountPath: /etc/ssl/certs                    name: ca-certificates                    readOnly: true                - mountPath: /var                    name: grafana-storage                env:                - name: INFLUXDB_HOST                    value: monitoring-influxdb                - name: GF_SERVER_HTTP_PORT                    value: "3000"                    # The following env variables are required to make Grafana accessible via                    # the kubernetes api-server proxy. On production clusters, we recommend                    # removing these env variables, setup auth for grafana, and expose the grafana                    # service using a LoadBalancer or a public IP.                - name: GF_AUTH_BASIC_ENABLED                    value: "false"                - name: GF_AUTH_ANONYMOUS_ENABLED                    value: "true"                - name: GF_AUTH_ANONYMOUS_ORG_ROLE                    value: Admin                - name: GF_SERVER_ROOT_URL                    # If you're only using the API Server proxy, set this value instead:                    # value: /api/v1/namespaces/kube-system/services/monitoring-grafana/proxy                    value: /            volumes:            - name: ca-certificates                hostPath:                    path: /etc/ssl/certs            - name: grafana-storage                emptyDir: {}---apiVersion: v1kind: Servicemetadata:    labels:        # For use as a Cluster add-on (https://github.com/kubernetes/kubernetes/tree/master/cluster/addons)        # If you are NOT using this as an addon, you should comment out this line.        kubernetes.io/cluster-service: 'true'        kubernetes.io/name: monitoring-grafana    name: monitoring-grafana    namespace: kube-systemspec:    # In a production setup, we recommend accessing Grafana through an external Loadbalancer    # or through a public IP.    # type: LoadBalancer    # You could also use NodePort to expose the service at a randomly-generated port    type: NodePort    ports:    - port: 80        targetPort: 3000        nodePort: 31111    type: NodePort    selector:        k8s-app: grafana

2)创建pod和service

kubectl apply -f grafana.yaml

3)验证

k8s监控

k8s监控

4)进入grafana和配置数据源

k8s监控

5)下载模板文件

模板文件下载地址: https://gitee.com/love-docker/k8s/raw/master/v1.11/monitor/grafana/kubernetes-cluster-monitoring-via-prometheus_rev3.json

6)导入模板文件

k8s监控

选择数据库

k8s监控

7)查看监控数据

k8s监控

转载于:https://blog.51cto.com/1000682/2359778

你可能感兴趣的文章
git学习网址
查看>>
怎么快速提取PDF文档中的有效图片
查看>>
每天10点定时清空某个文件夹下的子文件夹和文件
查看>>
20.QT-Qpixmap实现图片鼠标缩放,鼠标拖动示例(详解)
查看>>
C# API 获取系统DPI缩放倍数跟分辨率大小
查看>>
Spring Boot 实现 RabbitMQ 延迟消费和延迟重试队列
查看>>
InnoDB 存储引擎的主要知识点介绍
查看>>
jquery 选择器,模糊匹配
查看>>
grid 布局初体验
查看>>
微服务实践分享(5)缓存中心
查看>>
ajax请求数据动态渲染表格
查看>>
Docker修改默认存储位置(转)
查看>>
python 三元表达式 if for 构建List 进阶用法
查看>>
Android如何使用Https
查看>>
2018第32周总结
查看>>
Redis云端架构深入浅出
查看>>
gradle 自定义插件 下载配置文件
查看>>
idea操作整理
查看>>
IP查找所属网段
查看>>
WPF中在摄像头视频上叠加控件的解决方案
查看>>