<kbd id="5sdj3"></kbd>
<th id="5sdj3"></th>

  • <dd id="5sdj3"><form id="5sdj3"></form></dd>
    <td id="5sdj3"><form id="5sdj3"><big id="5sdj3"></big></form></td><del id="5sdj3"></del>

  • <dd id="5sdj3"></dd>
    <dfn id="5sdj3"></dfn>
  • <th id="5sdj3"></th>
    <tfoot id="5sdj3"><menuitem id="5sdj3"></menuitem></tfoot>

  • <td id="5sdj3"><form id="5sdj3"><menu id="5sdj3"></menu></form></td>
  • <kbd id="5sdj3"><form id="5sdj3"></form></kbd>

    K8S Service實(shí)戰(zhàn)與原理初探

    共 7950字,需瀏覽 16分鐘

     ·

    2022-03-02 06:10

    本文主要講述了K8S Service的基本概念,使用方式及實(shí)現(xiàn)原理。

    【作者】陳成,中國(guó)聯(lián)通軟件研究院容器云研發(fā)工程師,公共平臺(tái)與架構(gòu)研發(fā)事業(yè)部云計(jì)算研發(fā)組長(zhǎng),長(zhǎng)期從事大規(guī)模基礎(chǔ)平臺(tái)建設(shè)相關(guān)工作,先后從事Mesos、KVM、K8S等研究,專注于容器云計(jì)算框架、集群調(diào)度、虛擬化等。


    故事的開始,讓我們先從一件生產(chǎn)故障說(shuō)起。5月29日,內(nèi)部某系統(tǒng)出現(xiàn)大規(guī)模訪問(wèn)Service故障,發(fā)現(xiàn)Pod容器內(nèi)無(wú)法正常訪問(wèn)ServiceIP:Port,整個(gè)故障持續(xù)時(shí)間超過(guò)12h,相關(guān)運(yùn)維支撐人員沒(méi)有找到根本原因和解決辦法。

    經(jīng)過(guò)復(fù)盤,我們發(fā)現(xiàn),大家對(duì)于K8S Service的原理不夠清晰,導(dǎo)致對(duì)問(wèn)題的定位不能做得到快速準(zhǔn)確,如果當(dāng)時(shí)能夠按照如下的思路去思考問(wèn)題,排查過(guò)程不至于花費(fèi)如此久的時(shí)間。

    下面,我們就來(lái)細(xì)說(shuō)一下Service在Kubernetes中的作用、使用方法及原理

    Service是一種暴露一組Pod網(wǎng)絡(luò)的抽象方式,K8S Service提供了針對(duì)于一組Pod的負(fù)載均衡的暴露。通過(guò)這樣的方式,可以避免不同的pod之間訪問(wèn)時(shí)需要知曉對(duì)應(yīng)pod網(wǎng)絡(luò)信息的痛苦。例如:前端->后端,由于前端POD IP隨時(shí)變動(dòng),后端亦如此,如何處理前端POD和后端POD的通信,就需要Service這一抽象,來(lái)保證簡(jiǎn)單可靠。


    Service的使用

    1、典型服務(wù)配置方法

    當(dāng)配置了selector之后,Service Controller會(huì)自動(dòng)查找匹配這個(gè)selector的pod,并且創(chuàng)建出一個(gè)同名的endpoint對(duì)象,負(fù)責(zé)具體service之后連接。

    apiVersion: v1kind: Servicemetadata:  name: my-servicespec:  selector:    app: MyApp  ports:    - protocol: TCP      port: 80      targetPort: 9376

    2、配置沒(méi)有selector的服務(wù)

    沒(méi)有selector的service不會(huì)出現(xiàn)Endpoint的信息,需要手工創(chuàng)建Endpoint綁定,Endpoint可以是內(nèi)部的pod,也可以是外部的服務(wù)。

    apiVersion: v1kind: Servicemetadata:  name: my-servicespec:  ports:    - protocol: TCP      port: 80      targetPort: 9376---apiVersion: v1kind: Endpointsmetadata:  name: my-servicesubsets:  - addresses:      - ip: 192.0.2.42    ports:      - port: 9376

    Service的類型

    1.CluserIP

    kubectl expose pod nginx --type=CluserIP --port=80 --name=ng-svc apiVersion: v1kind: Servicemetadata:  name: ng-svc  namespace: defaultspec:  selector:      name: nginx  clusterIP: 11.254.0.2  ports:  - name: http    port: 80    protocol: TCP    targetPort: 1234  sessionAffinity: None  type: ClusterIP

    2.LoadBalance

    apiVersion: v1kind: Servicemetadata:  name: my-servicespec:  selector:    app: MyApp  ports:  - protocol: TCP    port: 80    targetPort: 9376  clusterIP: 10.0.171.239  type: LoadBalancerstatus:  loadBalancer:    ingress:    - ip: 192.0.2.127
    3.NodePort
    apiVersion: v1kind: Servicemetadata:name: my-servicespec:type: NodePortselector:  app: MyAppports:  - port: 80    targetPort: 80    nodePort: 30007
    4.ExternalName
    5.Headless
    apiVersion: v1kind: Servicemetadata:  labels:    run: curl  name: my-headless-service  namespace: defaultspec:  clusterIP: None  ports:  - port: 80    protocol: TCP    targetPort: 80  selector:    run: curl  type: ClusterIP
    對(duì)定義了選擇算符的無(wú)頭服務(wù),Endpoint 控制器在 API 中創(chuàng)建了 Endpoints 記錄, 并且修改 DNS 配置返回 A 記錄(IP 地址),通過(guò)這個(gè)地址直接到達(dá) Service 的后端 Pod 上。
    # ping my-headless-servicePING my-headless-service (172.200.6.207): 56 data bytes64 bytes from 172.200.6.207: seq=0 ttl=64 time=0.040 ms64?bytes?from?172.200.6.207:?seq=1?ttl=64?time=0.063?ms

    對(duì)沒(méi)有定義選擇算符的無(wú)頭服務(wù),Endpoint 控制器不會(huì)創(chuàng)建 Endpoints 記錄。然而 DNS 系統(tǒng)會(huì)查找和配置,無(wú)論是:

    • 對(duì)于 ExternalName 類型的服務(wù),查找其 CNAME 記錄

    • 對(duì)所有其他類型的服務(wù),查找與 Service 名稱相同的任何 Endpoints 的記錄


    Service的實(shí)現(xiàn)方式

    1.用戶態(tài)代理訪問(wèn)

    即:當(dāng)對(duì)于每個(gè)Service,Kube-Proxy會(huì)在本地Node上打開一個(gè)隨機(jī)選擇的端口,連接到代理端口的請(qǐng)求,都會(huì)被代理轉(zhuǎn)發(fā)給Pod。那么通過(guò)Iptables規(guī)則,捕獲到達(dá)Service:Port的請(qǐng)求都會(huì)被轉(zhuǎn)發(fā)到代理端口,代理端口重新轉(zhuǎn)為對(duì)Pod的訪問(wèn)

    這種方式的缺點(diǎn)是存在內(nèi)核態(tài)轉(zhuǎn)為用戶態(tài),再有用戶態(tài)轉(zhuǎn)發(fā)的兩次轉(zhuǎn)換,性能較差,一般不再使用

    2.Iptables模式

    3.Ipvs模式

    Service Iptables實(shí)現(xiàn)原理

    Iptables表和鏈及處理過(guò)程

    Service的Traffic流量將會(huì)通過(guò)prerouting和output重定向到kube-service鏈

    -A?PREROUTING?-m?comment?--comment?"kubernetes?service?portals"?-j?KUBE-SERVICES-A?POSTROUTING?-m?comment?--comment?"kubernetes?postrouting?rules"?-j?KUBE-POSTROUTING-A OUTPUT -m comment --comment "kubernetes service portals" -j KUBE-SERVICES
    • KUBE-SERVICES->KUBE-SVC-XXXXXXXXXXXXXXXX->KUBE-SEP-XXXXXXXXXXXXXXXX represents a ClusterIP service

    • KUBE-NODEPORTS->KUBE-SVC-XXXXXXXXXXXXXXXX->KUBE-SEP-XXXXXXXXXXXXXXXX represents a NodePort service

    幾種不同類型的Service在Kube-Proxy啟用Iptables模式下上的表現(xiàn)

    • ClusterIP

    -A KUBE-SERVICES ! -s 172.200.0.0/16 -d 10.100.160.92/32 -p tcp -m comment --comment "default/ccs-gateway-clusterip:http cluster IP" -m tcp --dport 30080 -j KUBE-MARK-MASQ-A KUBE-SERVICES -d 10.100.160.92/32 -p tcp -m comment --comment "default/ccs-gateway-clusterip:http cluster IP" -m tcp --dport 30080 -j KUBE-SVC-76GERFBRR2RGHNBJ  

    -A KUBE-SVC-76GERFBRR2RGHNBJ -m comment --comment "default/ccs-gateway-clusterip:http" -m statistic --mode random --probability 0.33333333349 -j KUBE-SEP-GBVECAZBIC3ZKMXB-A KUBE-SVC-76GERFBRR2RGHNBJ -m comment --comment "default/ccs-gateway-clusterip:http" -m statistic --mode random --probability 0.50000000000 -j KUBE-SEP-PVCYYXEU44D3IMGK-A KUBE-SVC-76GERFBRR2RGHNBJ -m comment --comment "default/ccs-gateway-clusterip:http" -j KUBE-SEP-JECGZLHE32MEARRX-A?KUBE-SVC-CEZPIJSAUFW5MYPQ?-m?comment?--comment?"kubernetes-dashboard/kubernetes-dashboard"?-j?KUBE-SEP-QO6MV4HR5U56RP7M??

    -A KUBE-SEP-GBVECAZBIC3ZKMXB -s 172.200.6.224/32 -m comment --comment "default/ccs-gateway-clusterip:http" -j KUBE-MARK-MASQ-A?KUBE-SEP-GBVECAZBIC3ZKMXB?-p?tcp?-m?comment?--comment?"default/ccs-gateway-clusterip:http"?-m?tcp?-j?DNAT?--to-destination?172.200.6.224:80...
    • NodePort

    apiVersion: v1kind: Servicemetadata:labels:  app: ccs-gatewayspec:clusterIP: 10.101.156.39externalTrafficPolicy: Clusterports:- name: http  nodePort: 30081  port: 30080  protocol: TCP  targetPort: 80selector:  app: ccs-gatewaysessionAffinity: Nonetype: NodePort

    -A?KUBE-NODEPORTS?-p?tcp?-m?comment?--comment?"default/ccs-gateway-service:http"?-m?tcp?--dport?30081?-j?KUBE-MARK-MASQ-A?KUBE-NODEPORTS?-p?tcp?-m?comment?--comment?"default/ccs-gateway-service:http"?-m?tcp?--dport?30081?-j?KUBE-SVC-QYHRFFHL5VINYT2K############################-A?KUBE-SVC-QYHRFFHL5VINYT2K?-m?comment?--comment?"default/ccs-gateway-service:http"?-m?statistic?--mode?random?--probability?0.50000000000?-j?KUBE-SEP-2NPKETIWKKVUXGCL-A?KUBE-SVC-QYHRFFHL5VINYT2K?-m?comment?--comment?"default/ccs-gateway-service:http"?-j?KUBE-SEP-6O5FHQRN5IVNPW4Q##########################-A?KUBE-SEP-2NPKETIWKKVUXGCL?-s?172.200.6.224/32?-m?comment?--comment?"default/ccs-gateway-service:http"?-j?KUBE-MARK-MASQ-A?KUBE-SEP-2NPKETIWKKVUXGCL?-p?tcp?-m?comment?--comment?"default/ccs-gateway-service:http"?-m?tcp?-j?DNAT?--to-destination?172.200.6.224:80#########################-A?KUBE-SEP-6O5FHQRN5IVNPW4Q?-s?172.200.6.225/32?-m?comment?--comment?"default/ccs-gateway-service:http"?-j?KUBE-MARK-MASQ-A KUBE-SEP-6O5FHQRN5IVNPW4Q -p tcp -m comment --comment "default/ccs-gateway-service:http" -m tcp -j DNAT --to-destination 172.200.6.225:80

    同時(shí),可以看到Service所申請(qǐng)的端口38081被Kube-proxy所代理和監(jiān)聽(tīng)

    # netstat -ntlp | grep 30081tcp       0      00.0.0.0:30081           0.0.0.0:*               LISTEN     3665705/kube-proxy

    • LoadBalancer

    不帶有Endpoint的Service

    kubectl create svc clusterip fake-endpoint --tcp=80 -A KUBE-SERVICES -d 10.101.117.0/32 -p tcp -m comment --comment "default/fake-endpoint:80 has no endpoints" -m tcp --dport 80 -j REJECT --reject-with icmp-port-unreachable

    帶有外部endpoint的Service

    直接通過(guò)iptable規(guī)則轉(zhuǎn)發(fā)到對(duì)應(yīng)的外部ep地址

    apiVersion: v1kind: Servicemetadata:  labels:    app: external  name: external  namespace: defaultspec:  ports:  - name: http    protocol: TCP    port: 80  sessionAffinity: None  type: ClusterIP---apiVersion: v1kind: Endpointsmetadata:  labels:    app: external  name: external  namespace: defaultsubsets:- addresses:  - ip: 10.124.142.43  ports:  - name: http    port: 80protocol: TCP


    -A KUBE-SERVICES ! -s 172.200.0.0/16 -d 10.111.246.87/32 -p tcp -m comment --comment "default/external:http cluster IP" -m tcp --dport 80 -j KUBE-MARK-MASQ-A KUBE-SERVICES -d 10.111.246.87/32 -p tcp -m comment --comment "default/external:http cluster IP" -m tcp --dport 80 -j KUBE-SVC-LI2K5327B6J24KJ3 

    -A KUBE-SEP-QTGIPNOYXN2CZGD5 -s 10.124.142.43/32 -m comment --comment "default/external:http" -j KUBE-MARK-MASQ-A KUBE-SEP-QTGIPNOYXN2CZGD5 -p tcp -m comment --comment "default/external:http" -m tcp -j DNAT --to-destination 10.124.142.43:80


    總結(jié)

    • ClusterIP類型,KubeProxy監(jiān)聽(tīng)Service和Endpoint創(chuàng)建規(guī)則,采用DNAT將目標(biāo)地址轉(zhuǎn)換為Pod的ip和端口,當(dāng)有多個(gè)ep時(shí),按照策略進(jìn)行轉(zhuǎn)發(fā),默認(rèn)RR模式時(shí),iptables采用:比如有4個(gè)實(shí)例,四條規(guī)則的概率分別為0.25, 0.33, 0.5和 1,按照順序,一次匹配完成整個(gè)流量的分配。

    • NodePort類型,將會(huì)在上述ClusterIP模式之后,再加上Kube-Proxy的監(jiān)聽(tīng)(為了確保其他服務(wù)不會(huì)占用該端口)和KUBE-NODEPORT的iptable規(guī)則

    參考文獻(xiàn)

    1、iptables https://en.wikipedia.org/wiki/Iptables

    2、ipvs https://en.wikipedia.org/wiki/IP_Virtual_Server

    3、K8S Service https://kubernetes.io/zh/docs/concepts/services-networking/service/

    文章轉(zhuǎn)載:twt企業(yè)IT社區(qū)???

    (版權(quán)歸原作者所有,侵刪)


    點(diǎn)擊下方“閱讀原文”查看更多

    瀏覽 65
    點(diǎn)贊
    評(píng)論
    收藏
    分享

    手機(jī)掃一掃分享

    分享
    舉報(bào)
    評(píng)論
    圖片
    表情
    推薦
    點(diǎn)贊
    評(píng)論
    收藏
    分享

    手機(jī)掃一掃分享

    分享
    舉報(bào)

    <kbd id="5sdj3"></kbd>
    <th id="5sdj3"></th>

  • <dd id="5sdj3"><form id="5sdj3"></form></dd>
    <td id="5sdj3"><form id="5sdj3"><big id="5sdj3"></big></form></td><del id="5sdj3"></del>

  • <dd id="5sdj3"></dd>
    <dfn id="5sdj3"></dfn>
  • <th id="5sdj3"></th>
    <tfoot id="5sdj3"><menuitem id="5sdj3"></menuitem></tfoot>

  • <td id="5sdj3"><form id="5sdj3"><menu id="5sdj3"></menu></form></td>
  • <kbd id="5sdj3"><form id="5sdj3"></form></kbd>
    免费伦片A片在线观看警官 | 手机无码在线观看 | 大鸡巴日骚逼视频 | 可以免费看黄片的网站 | avtt在线看 |