大天使之剑
作者大天使之剑·2019-05-31 15:04
售前技术支持·杭州才云科技

K8s系列之service是个啥东西?

字数 17548阅读 1479评论 1赞 2

service到底是什么?

k8s的Service定义了一个服务的访问入口地址,前端的应用通过这个入口地址访问其背后的一组由Pod副本组成的集群实例,来自外部的访问请求被负载均衡到后端的各个容器应用上。Service与其后端Pod副本集群之间则是通过Label Selector来实现对接的。而RC的作用相当于是保证Service的服务能力和服务质量始终处于预期的标准。Service 定义可以基于 POST 方式,请求 apiserver 创建新的实例。

废话不多说,上实验。

新建一个nginx pod

[root@master ~]# kubectl run nginx-deploy --image=nginx:1.14-alpine --port=80 --replicas=1
deployment.apps/nginx-deploy created
[root@master ~]# kubectl get pods
NAME READY STATUS RESTARTS AGE
nginx-deploy-5b595999-8zskg 0/1 ContainerCreating 0 6s
[root@master ~]# kubectl get deploy
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
nginx-deploy 1 1 1 1 18s
[root@master ~]# kubectl get pods
NAME READY STATUS RESTARTS AGE
nginx-deploy-5b595999-8zskg 1/1 Running 0 41s

检查是否可以访问

[root@master ~]# curl 10.244.2.2
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
body {
width: 35em;
margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif;
}
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>

这时候访问ip为pod_ip,那么问题来了,如果我把pod删除,则deploy控制器会重新启动一个pod以实现第一次创建pod时指定的副本数为1的条件,这个时候pod ip会变化,这不是我们期望的,所以需要创建一个service 来保证pod的ip的变化对pod客户端(其他pod,集群内部客户端)来说是透明的,总而言之,service为pods提供固定访问端点。

创建service,service有很多类型,如 ClusterIP, NodePort, LoadBalancer, or ExternalName,默认为ClusterIP,意思是这个services IP 只能被集群内的pod客户端访问。

[root@master ~]# kubectl expose deployment nginx-deploy --name=nginx --port=80 --target-port=80 --protocol=TCP
service/nginx exposed
[root@master ~]# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 1h
nginx ClusterIP 10.101.46.8 <none> 80/TCP 29s
[root@master ~]# curl 10.101.46.8
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
body {
width: 35em;
margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif;
}
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>

如果将集群内的节点dns改为 kube-dns的地址,则可以通过service name 直接解析到对应的IP。

[root@master ~]# kubectl get svc -n kube-system
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kube-dns ClusterIP 10.96.0.10 <none> 53/UDP,53/TCP 1h

我创建一个client(使用busybox)来验证是否可以实现dns功能。

[root@master ~]# kubectl run client --image=busybox --replicas=1 -it --restart=Never
If you don't see a command prompt, try pressing enter.
/ # cat /etc/resolv.conf
nameserver 10.96.0.10
search default.svc.cluster.local svc.cluster.local cluster.local
options ndots:5
/ # wget -O - -q http://nginx:80
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
body {
width: 35em;
margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif;
}
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>

可以看到client获取的dns为 default.svc.cluster.local ,则nginx的完整域名为:nginx.default.svc.cluster.local。

[root@master ~]# dig -t A nginx.default.svc.cluster.local @10.96.0.10
; <<>> DiG 9.9.4-RedHat-9.9.4-61.el7_5.1 <<>> -t A nginx.default.svc.cluster.local @10.96.0.10
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 13045
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;nginx.default.svc.cluster.local. IN A
;; ANSWER SECTION:
nginx.default.svc.cluster.local. 5 IN A 10.101.46.8
;; Query time: 0 msec
;; SERVER: 10.96.0.10#53(10.96.0.10)
;; WHEN: Wed Sep 26 16:03:51 CST 2018
;; MSG SIZE rcvd: 107

这时我手动删除一个nginx pod ,deploy控制器会自动创建一个新的nginx pod 调度至对应的node节点,这是我使用busybox 再次解析nginx,查看service是否生效。

[root@master ~]# kubectl delete pods nginx-deploy-5b595999-q7kcr
pod "nginx-deploy-5b595999-q7kcr" deleted
[root@master ~]# kubectl get pods
NAME READY STATUS RESTARTS AGE
client 1/1 Running 0 11m
nginx-deploy-5b595999-xgh9r 1/1 Running 0 13s
/ # wget -O - -q http://nginx:80
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
body {
width: 35em;
margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif;
}
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>

services是什么?

它是一个iptables或ipvs规则,使用标签选择器,将访问service_ip:port所有的都调度至相应的pod后端,怎么知道service关联到哪些pod呢?

[root@master ~]# kubectl describe svc nginx
Name: nginx
Namespace: default
Labels: run=nginx-deploy
Annotations: <none>
Selector: run=nginx-deploy
Type: ClusterIP
IP: 10.101.46.8
Port: <unset> 80/TCP
TargetPort: 80/TCP
Endpoints: 10.244.1.3:80
Session Affinity: None
Events: <none>

上面的 Endpoints:10.244.1.3:80 就是关联的pods,而 Selector: run=nginx-deploy 就是选择对应的pod来进行关联,现在查看nginx_pod的标签是否为“run=nginx-deploy”。

[root@master ~]# kubectl describe pods nginx-deploy-5b595999-xgh9r
Name: nginx-deploy-5b595999-xgh9r
Namespace: default
Priority: 0
PriorityClassName: <none>
Node: node1/192.168.100.238
Start Time: Wed, 26 Sep 2018 16:11:39 +0800
Labels: pod-template-hash=16151555
run=nginx-deploy
Annotations: <none>
Status: Running
IP: 10.244.1.3

如果对应的pod资源被删除了,则services后端关联的pod_ip 会动态变化,当然我们也可以去改service的ip,则kube-dns内对应的记录也会动态改变。

[root@master ~]# kubectl delete svc nginx
service "nginx" deleted
[root@master ~]# kubectl expose deployment nginx-deploy --name=nginx
service/nginx exposed
[root@master ~]# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 1h
nginx ClusterIP 10.103.33.148 <none> 80/TCP 5s

这时使用busybox再次访问nginx,看dns是否动态改变。

/ # wget -O -q http://nginx:80
Connecting to nginx:80 (10.101.46.8:80)
-q 100% |*| 612 0:00:00 ETA
/ # wget -O -q http://nginx:80
Connecting to nginx:80 (10.103.33.148:80)
-q 100% |*| 612 0:00:00 ETA
/ # wget -O - -q http://nginx:80
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
body {
width: 35em;
margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif;
}
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>

一个创建好的deployment是可以动态改变副本数的,也就意味着可以实现弹性扩容。

我新建一个两个副本的deployment。

[root@master ~]# kubectl run myapp --image=ikubernetes/myapp:v1 --replicas=2
deployment.apps/myapp created
[root@master ~]# kubectl get deployment
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
myapp 2 2 2 0 14s
nginx-deploy 1 1 1 1 1h
[root@master ~]# kubectl get deployment -w
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
myapp 2 2 2 0 21s
nginx-deploy 1 1 1 1 1h
myapp 2 2 2 1 24s
myapp 2 2 2 2 25s
q^C[root@master ~]# kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE
client 1/1 Running 0 34m 10.244.2.3 node2 <none>
myapp-848b5b879b-6kqp2 1/1 Running 0 46s 10.244.1.4 node1 <none>
myapp-848b5b879b-vg8sm 1/1 Running 0 46s 10.244.2.4 node2 <none>
nginx-deploy-5b595999-xgh9r 1/1 Running 0 23m 10.244.1.3 node1 <none>

kubectl get deployment -w 为动态监控delpoyment信息 w意为watch。

这时我使用busybox访问查看是否生效。

/ # wget -O - -q 10.244.1.4
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
/ # wget -O - -q 10.244.2.4
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
/ # wget -O - -q 10.244.1.4/hostname.html
myapp-848b5b879b-6kqp2
/ # wget -O - -q 10.244.2.4/hostname.html
myapp-848b5b879b-vg8sm

当然我们也需要创建一个service来提供固定访问端点。

[root@master ~]# kubectl expose deployment myapp --name=myapp --port=80
service/myapp exposed
[root@master ~]# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 2h
myapp ClusterIP 10.110.24.147 <none> 80/TCP 8s
nginx ClusterIP 10.103.33.148 <none> 80/TCP 9m

客户端再访问一次,并测试负载均衡的效果。

/ # wget -O - -q myapp
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
/ # while true;do wget -O - -q myapp/hostname.html; sleep 1;done
myapp-848b5b879b-vg8sm
myapp-848b5b879b-vg8sm
myapp-848b5b879b-6kqp2
myapp-848b5b879b-6kqp2
myapp-848b5b879b-6kqp2
myapp-848b5b879b-6kqp2

最后验证一下service到底是什么。

[root@master ~]# iptables -vnL -t nat
Chain PREROUTING (policy ACCEPT 1 packets, 78 bytes)
pkts bytes target prot opt in out source destination
3418 568K KUBE-SERVICES all -- 0.0.0.0/0 0.0.0.0/0 / kubernetes service portals /
767 91755 DOCKER all -- 0.0.0.0/0 0.0.0.0/0 ADDRTYPE match dst-type LOCAL
Chain INPUT (policy ACCEPT 1 packets, 78 bytes)
pkts bytes target prot opt in out source destination
Chain OUTPUT (policy ACCEPT 1 packets, 60 bytes)
pkts bytes target prot opt in out source destination
7355 532K KUBE-SERVICES all -- 0.0.0.0/0 0.0.0.0/0 / kubernetes service portals /
1173 70380 DOCKER all -- 0.0.0.0/0 !127.0.0.0/8 ADDRTYPE match dst-type LOCAL
Chain POSTROUTING (policy ACCEPT 1 packets, 60 bytes)
pkts bytes target prot opt in out source destination
8271 598K KUBE-POSTROUTING all -- 0.0.0.0/0 0.0.0.0/0 / kubernetes postrouting rules /
0 0 MASQUERADE all -- * !docker0 172.17.0.0/16 0.0.0.0/0
2487 160K RETURN all -- 10.244.0.0/16 10.244.0.0/16
1 62 MASQUERADE all -- 10.244.0.0/16 !224.0.0.0/4
0 0 RETURN all -- !10.244.0.0/16 10.244.0.0/24
0 0 MASQUERADE all -- !10.244.0.0/16 10.244.0.0/16
Chain DOCKER (2 references)
pkts bytes target prot opt in out source destination
0 0 RETURN all -- docker0 * 0.0.0.0/0 0.0.0.0/0
Chain KUBE-MARK-DROP (0 references)
pkts bytes target prot opt in out source destination
0 0 MARK all -- 0.0.0.0/0 0.0.0.0/0 MARK or 0x8000
Chain KUBE-MARK-MASQ (14 references)
pkts bytes target prot opt in out source destination
0 0 MARK all -- 0.0.0.0/0 0.0.0.0/0 MARK or 0x4000
Chain KUBE-NODEPORTS (1 references)
pkts bytes target prot opt in out source destination
Chain KUBE-POSTROUTING (1 references)
pkts bytes target prot opt in out source destination
0 0 MASQUERADE all -- 0.0.0.0/0 0.0.0.0/0 / kubernetes service traffic requiring SNAT / mark match 0x4000/0x4000
Chain KUBE-SEP-34GZOZPWVIII2XSR (1 references)
pkts bytes target prot opt in out source destination
0 0 KUBE-MARK-MASQ all -- 192.168.100.113 0.0.0.0/0 / default/kubernetes:https /
0 0 DNAT tcp -- 0.0.0.0/0 0.0.0.0/0 / default/kubernetes:https / tcp to:192.168.100.113:6443
Chain KUBE-SEP-6E7XQMQ4RAYOWTTM (1 references)
pkts bytes target prot opt in out source destination
0 0 KUBE-MARK-MASQ all -- 10.244.0.3 0.0.0.0/0 / kube-system/kube-dns:dns /
0 0 DNAT udp -- 0.0.0.0/0 0.0.0.0/0 / kube-system/kube-dns:dns / udp to:10.244.0.3:53
Chain KUBE-SEP-ASUKHZ3BY2L3C3LX (1 references)
pkts bytes target prot opt in out source destination
0 0 KUBE-MARK-MASQ all -- 10.244.1.3 0.0.0.0/0 / default/nginx: /
0 0 DNAT tcp -- 0.0.0.0/0 0.0.0.0/0 / default/nginx: / tcp to:10.244.1.3:80
Chain KUBE-SEP-DKFTHMPG5FFKBJK4 (1 references)
pkts bytes target prot opt in out source destination
0 0 KUBE-MARK-MASQ all -- 10.244.2.9 0.0.0.0/0 / default/myapp: /
0 0 DNAT tcp -- 0.0.0.0/0 0.0.0.0/0 / default/myapp: / tcp to:10.244.2.9:80
Chain KUBE-SEP-IT2ZTR26TO4XFPTO (1 references)
pkts bytes target prot opt in out source destination
0 0 KUBE-MARK-MASQ all -- 10.244.0.2 0.0.0.0/0 / kube-system/kube-dns:dns-tcp /
0 0 DNAT tcp -- 0.0.0.0/0 0.0.0.0/0 / kube-system/kube-dns:dns-tcp / tcp to:10.244.0.2:53
Chain KUBE-SEP-PBOEWUAIN3XEJ4KY (1 references)
pkts bytes target prot opt in out source destination
0 0 KUBE-MARK-MASQ all -- 10.244.2.8 0.0.0.0/0 / default/myapp: /
0 0 DNAT tcp -- 0.0.0.0/0 0.0.0.0/0 / default/myapp: / tcp to:10.244.2.8:80
Chain KUBE-SEP-VXUAG2QWVQYXGMBX (1 references)
pkts bytes target prot opt in out source destination
0 0 KUBE-MARK-MASQ all -- 10.244.1.8 0.0.0.0/0 / default/myapp: /
0 0 DNAT tcp -- 0.0.0.0/0 0.0.0.0/0 / default/myapp: / tcp to:10.244.1.8:80
Chain KUBE-SEP-YIL6JZP7A3QYXJU2 (1 references)
pkts bytes target prot opt in out source destination
0 0 KUBE-MARK-MASQ all -- 10.244.0.2 0.0.0.0/0 / kube-system/kube-dns:dns /
0 0 DNAT udp -- 0.0.0.0/0 0.0.0.0/0 / kube-system/kube-dns:dns / udp to:10.244.0.2:53
Chain KUBE-SEP-ZXMNUKOKXUTL2MK2 (1 references)
pkts bytes target prot opt in out source destination
0 0 KUBE-MARK-MASQ all -- 10.244.0.3 0.0.0.0/0 / kube-system/kube-dns:dns-tcp /
0 0 DNAT tcp -- 0.0.0.0/0 0.0.0.0/0 / kube-system/kube-dns:dns-tcp / tcp to:10.244.0.3:53
Chain KUBE-SERVICES (2 references)
pkts bytes target prot opt in out source destination
0 0 KUBE-MARK-MASQ tcp -- !10.244.0.0/16 10.110.24.147 / default/myapp: cluster IP / tcp dpt:80
0 0 KUBE-SVC-NPJI2GAOYBRMPXVD tcp -- 0.0.0.0/0 10.110.24.147 / default/myapp: cluster IP / tcp dpt:80
0 0 KUBE-MARK-MASQ tcp -- !10.244.0.0/16 10.96.0.1 / default/kubernetes:https cluster IP / tcp dpt:443
0 0 KUBE-SVC-NPX46M4PTMTKRN6Y tcp -- 0.0.0.0/0 10.96.0.1 / default/kubernetes:https cluster IP / tcp dpt:443
0 0 KUBE-MARK-MASQ udp -- !10.244.0.0/16 10.96.0.10 / kube-system/kube-dns:dns cluster IP / udp dpt:53
0 0 KUBE-SVC-TCOU7JCQXEZGVUNU udp -- 0.0.0.0/0 10.96.0.10 / kube-system/kube-dns:dns cluster IP / udp dpt:53
0 0 KUBE-MARK-MASQ tcp -- !10.244.0.0/16 10.96.0.10 / kube-system/kube-dns:dns-tcp cluster IP / tcp dpt:53
0 0 KUBE-SVC-ERIFXISQEP7F7OF4 tcp -- 0.0.0.0/0 10.96.0.10 / kube-system/kube-dns:dns-tcp cluster IP / tcp dpt:53
0 0 KUBE-MARK-MASQ tcp -- !10.244.0.0/16 10.103.33.148 / default/nginx: cluster IP / tcp dpt:80
0 0 KUBE-SVC-4N57TFCL4MD7ZTDA tcp -- 0.0.0.0/0 10.103.33.148 / default/nginx: cluster IP / tcp dpt:80
4 240 KUBE-NODEPORTS all -- 0.0.0.0/0 0.0.0.0/0 / kubernetes service nodeports; NOTE: this must be the last rule in this chain / ADDRTYPE match dst-type LOCAL
Chain KUBE-SVC-4N57TFCL4MD7ZTDA (1 references)
pkts bytes target prot opt in out source destination
0 0 KUBE-SEP-ASUKHZ3BY2L3C3LX all -- 0.0.0.0/0 0.0.0.0/0 / default/nginx: /
Chain KUBE-SVC-ERIFXISQEP7F7OF4 (1 references)
pkts bytes target prot opt in out source destination
0 0 KUBE-SEP-IT2ZTR26TO4XFPTO all -- 0.0.0.0/0 0.0.0.0/0 / kube-system/kube-dns:dns-tcp / statistic mode random probability 0.50000000000
0 0 KUBE-SEP-ZXMNUKOKXUTL2MK2 all -- 0.0.0.0/0 0.0.0.0/0 / kube-system/kube-dns:dns-tcp /
Chain KUBE-SVC-NPJI2GAOYBRMPXVD (1 references)
pkts bytes target prot opt in out source destination
0 0 KUBE-SEP-VXUAG2QWVQYXGMBX all -- 0.0.0.0/0 0.0.0.0/0 / default/myapp: / statistic mode random probability 0.33332999982
0 0 KUBE-SEP-PBOEWUAIN3XEJ4KY all -- 0.0.0.0/0 0.0.0.0/0 / default/myapp: / statistic mode random probability 0.50000000000
0 0 KUBE-SEP-DKFTHMPG5FFKBJK4 all -- 0.0.0.0/0 0.0.0.0/0 / default/myapp: /
Chain KUBE-SVC-NPX46M4PTMTKRN6Y (1 references)
pkts bytes target prot opt in out source destination
0 0 KUBE-SEP-34GZOZPWVIII2XSR all -- 0.0.0.0/0 0.0.0.0/0 / default/kubernetes:https /
Chain KUBE-SVC-TCOU7JCQXEZGVUNU (1 references)
pkts bytes target prot opt in out source destination
0 0 KUBE-SEP-YIL6JZP7A3QYXJU2 all -- 0.0.0.0/0 0.0.0.0/0 / kube-system/kube-dns:dns / statistic mode random probability 0.50000000000
0 0 KUBE-SEP-6E7XQMQ4RAYOWTTM all -- 0.0.0.0/0 0.0.0.0/0 / kube-system/kube-dns:dns /

service_ip存在于每个节点上的iptables,ipvs 所以导致ping不通,但是可正常使用 若想在集群外部访问到myapp或者nginx怎么办? 修改service类型为NodePort即可(注意大小写)。

[root@master ~]# kubectl edit svc myapp
service/myapp edited
[root@master ~]# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 2h
myapp NodePort 10.110.24.147 <none> 80:30854/TCP 35m
nginx ClusterIP 10.103.33.148 <none> 80/TCP 45m

这时可以看见动态映射为30854,访问每个节点的30854端口都可以访问到myapp 那么问题来了,若一个node节点挂掉了,那客户端访问的ip也改为了其他node ip,对用户很不友好,所以我们需要在外部设置一个负载均衡器(需要高可用)消除这个影响。

如果觉得我的文章对您有用,请点赞。您的支持将鼓励我继续创作!

2

添加新评论1 条评论

michael1983michael1983联盟成员技术总监某证券
2019-06-03 13:39
谢谢分享
Ctrl+Enter 发表

作者其他文章

相关文章

相关问题

相关资料

X社区推广