環境
[zaki@cloud-dev ~]$ kc version --short Client Version: v1.18.8 Server Version: v1.18.5
みんな大好きkubeadmで作ったオンプレK8s(MetalLBによるLoadbalancer Service対応)です。
[zaki@cloud-dev ~]$ istioctl version client version: 1.6.5 control plane version: 1.6.5 data plane version: 1.6.5 (22 proxies)
Istioバージョンは最新追えていなくて1.6.5。
quickstart
とりあえずデプロイしてみるぞ!
https://microservices-demo.github.io/docs/quickstart.html
と言っても、まずGitのソースをclone
します。
$ git clone https://github.com/microservices-demo/microservices-demo cd microservices-demo
ただし、quickstartの記載はdocker-compose
の場合なので、Any Kubernetes Clusterを参照。
まずnamespaceを作成
$ kubectl create namespace sock-shop
そして、デプロイ用マニフェストをapply
する。
マニフェスト見る限り、namespace
のデプロイ先はsock-shop
固定になっている。
$ kubectl apply -f deploy/kubernetes/complete-demo.yaml
[zaki@cloud-dev microservices-demo]$ kubectl apply -f deploy/kubernetes/complete-demo.yaml service/carts-db created service/carts created service/catalogue-db created service/catalogue created service/front-end created service/orders-db created service/orders created service/payment created service/queue-master created service/rabbitmq created service/shipping created service/user-db created service/user created unable to recognize "deploy/kubernetes/complete-demo.yaml": no matches for kind "Deployment" in version "extensions/v1beta1" unable to recognize "deploy/kubernetes/complete-demo.yaml": no matches for kind "Deployment" in version "extensions/v1beta1" unable to recognize "deploy/kubernetes/complete-demo.yaml": no matches for kind "Deployment" in version "extensions/v1beta1" unable to recognize "deploy/kubernetes/complete-demo.yaml": no matches for kind "Deployment" in version "extensions/v1beta1" unable to recognize "deploy/kubernetes/complete-demo.yaml": no matches for kind "Deployment" in version "extensions/v1beta1" unable to recognize "deploy/kubernetes/complete-demo.yaml": no matches for kind "Deployment" in version "extensions/v1beta1" unable to recognize "deploy/kubernetes/complete-demo.yaml": no matches for kind "Deployment" in version "extensions/v1beta1" unable to recognize "deploy/kubernetes/complete-demo.yaml": no matches for kind "Deployment" in version "extensions/v1beta1" unable to recognize "deploy/kubernetes/complete-demo.yaml": no matches for kind "Deployment" in version "extensions/v1beta1" unable to recognize "deploy/kubernetes/complete-demo.yaml": no matches for kind "Deployment" in version "extensions/v1beta1" unable to recognize "deploy/kubernetes/complete-demo.yaml": no matches for kind "Deployment" in version "extensions/v1beta1" unable to recognize "deploy/kubernetes/complete-demo.yaml": no matches for kind "Deployment" in version "extensions/v1beta1" unable to recognize "deploy/kubernetes/complete-demo.yaml": no matches for kind "Deployment" in version "extensions/v1beta1"
あらら、DeploymentリソースのapiVersion
がextensions/v1beta1
になっている。
これ、Kubernetes 1.16からはapps/v1
にする必要があるので要修正。(後述)
diff --git a/deploy/kubernetes/complete-demo.yaml b/deploy/kubernetes/complete-demo.yaml index a11f99b..64ac9c0 100644 --- a/deploy/kubernetes/complete-demo.yaml +++ b/deploy/kubernetes/complete-demo.yaml @@ -1 +1 @@ -apiVersion: extensions/v1beta1 +apiVersion: apps/v1 @@ -55 +55 @@ spec: -apiVersion: extensions/v1beta1 +apiVersion: apps/v1 [...]
※ 同じ修正の繰り返しなので省略
これで再デプロイ
…をすると、今度はselector
のエラー
$ kubectl apply -f deploy/kubernetes/complete-demo.yaml error: error validating "deploy/kubernetes/complete-demo.yaml": error validating data: ValidationError(Deployment.spec): missing required field "selector" in io.k8s.api.apps.v1.DeploymentSpec; if you choose to ignore these errors, turn validation off with --validate=false
spec
にはselector
が必要ということ。
前述のextensions/v1beta1
の変更と合わせて、修正内容は以下の通り。
これでデプロイ
[zaki@cloud-dev microservices-demo]$ kubectl apply -f deploy/kubernetes/complete-demo.yaml deployment.apps/carts-db created service/carts-db unchanged deployment.apps/carts created service/carts unchanged deployment.apps/catalogue-db created service/catalogue-db unchanged deployment.apps/catalogue created service/catalogue unchanged deployment.apps/front-end created service/front-end unchanged deployment.apps/orders-db created service/orders-db unchanged deployment.apps/orders created service/orders unchanged deployment.apps/payment created service/payment unchanged deployment.apps/queue-master created service/queue-master unchanged deployment.apps/rabbitmq created service/rabbitmq unchanged deployment.apps/shipping created service/shipping unchanged deployment.apps/user-db created service/user-db unchanged deployment.apps/user created service/user unchanged
とりあえずデプロイは成功。
しばらく待てば、各podともにRunningになるはず。
[zaki@cloud-dev microservices-demo]$ kc get pod,svc -n sock-shop NAME READY STATUS RESTARTS AGE pod/carts-7bbbd7779d-bwwm2 1/1 Running 0 8m9s pod/carts-db-84b777d9c-nzt8m 1/1 Running 0 8m9s pod/catalogue-8684f655d9-m9dxk 1/1 Running 0 8m9s pod/catalogue-db-5579f7f4cb-55gtk 1/1 Running 0 8m9s pod/front-end-6f5fc69d6-brlb8 1/1 Running 0 8m9s pod/orders-7ccf68495-p599t 1/1 Running 0 8m9s pod/orders-db-77c46b9c85-m5q55 1/1 Running 0 8m9s pod/payment-74976d749f-ktmp7 1/1 Running 0 8m9s pod/queue-master-799b6b57d5-bc9pt 1/1 Running 0 8m9s pod/rabbitmq-8458d7d4c-2zwlf 1/1 Running 0 8m9s pod/shipping-5c89f4886c-8pppb 1/1 Running 0 8m9s pod/user-d6c48f668-wqs2d 1/1 Running 0 8m8s pod/user-db-5cdfd6d44b-26w5j 1/1 Running 0 8m8s NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/carts ClusterIP 10.96.85.13 <none> 80/TCP 25m service/carts-db ClusterIP 10.108.236.130 <none> 27017/TCP 25m service/catalogue ClusterIP 10.110.18.120 <none> 80/TCP 25m service/catalogue-db ClusterIP 10.97.198.131 <none> 3306/TCP 25m service/front-end NodePort 10.101.204.208 <none> 80:30001/TCP 25m service/orders ClusterIP 10.103.82.66 <none> 80/TCP 25m service/orders-db ClusterIP 10.104.77.171 <none> 27017/TCP 25m service/payment ClusterIP 10.107.33.54 <none> 80/TCP 25m service/queue-master ClusterIP 10.100.115.94 <none> 80/TCP 25m service/rabbitmq ClusterIP 10.110.36.96 <none> 5672/TCP 25m service/shipping ClusterIP 10.107.24.182 <none> 80/TCP 25m service/user ClusterIP 10.101.65.142 <none> 80/TCP 25m service/user-db ClusterIP 10.101.67.24 <none> 27017/TCP 25m
webアクセス
NodePortのServiceもデプロイされているので、ノードにアクセスする。
[zaki@cloud-dev microservices-demo]$ kc get svc -n sock-shop front-end NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE front-end NodePort 10.101.204.208 <none> 80:30001/TCP 36m
[zaki@cloud-dev microservices-demo]$ kc get node -o wide NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME k8s-master01.esxi.jp-z.jp Ready master 53d v1.18.5 192.168.0.121 <none> CentOS Linux 7 (Core) 3.10.0-1127.13.1.el7.x86_64 docker://19.3.12 k8s-worker01.esxi.jp-z.jp Ready <none> 53d v1.18.5 192.168.0.125 <none> CentOS Linux 7 (Core) 3.10.0-1127.13.1.el7.x86_64 docker://19.3.12 k8s-worker02.esxi.jp-z.jp Ready <none> 53d v1.18.5 192.168.0.126 <none> CentOS Linux 7 (Core) 3.10.0-1127.13.1.el7.x86_64 docker://19.3.12
この場合であれば3台のノードのうちどれでも任意のノードの30001/TCP
にアクセスすればOK
※ NodePort Serviceを使っているので、URLがノードのアドレス:NodePortのポートであるhttp://192.168.0.125:30001/
になっている。
apiVersion
のextensions/v1beta1
については、こちらのKubernetes 1.15 -> 1.16の変更点を参照。
Istio対応
サイドカーインジェクション用namespace
Istio上で動かすにはistio-injection=enabled
というラベルが設定されたnamespaceにデプロイする。
これだけ!
[zaki@cloud-dev microservices-demo]$ kc create namespace sock-shop-mesh namespace/sock-shop-mesh created [zaki@cloud-dev microservices-demo]$ kc label namespace sock-shop-mesh istio-injection=enabled namespace/sock-shop-mesh labeled
で、Sock Shopデプロイ用マニフェストから(任意のネームスペースにデプロイできるように)全てのnamespace: sock-shop
を削除して、sock-shop-mesh
ネームスペースを指定してデプロイ
↓はgrep -v
で雑に除外してるけど、↑に修正版マニフェスト置いています。
[zaki@cloud-dev microservices-demo]$ grep -v namespace deploy/kubernetes/complete-demo.yaml | kubectl apply -f - -n sock-shop-mesh deployment.apps/carts-db created service/carts-db created deployment.apps/carts created service/carts created deployment.apps/catalogue-db created service/catalogue-db created deployment.apps/catalogue created service/catalogue created deployment.apps/front-end created deployment.apps/orders-db created service/orders-db created deployment.apps/orders created service/orders created deployment.apps/payment created service/payment created deployment.apps/queue-master created service/queue-master created deployment.apps/rabbitmq created service/rabbitmq created deployment.apps/shipping created service/shipping created deployment.apps/user-db created service/user-db created deployment.apps/user created service/user created The Service "front-end" is invalid: spec.ports[0].nodePort: Invalid value: 30001: provided port is already allocated
おっと、NodePortのポート番号指定してたのか。とりあえず無視…はNodePortとしては不要だけど、Service自体は(外部からのアクセス時のルーティング先として)必要なので定義を変更してClusterIPで再デプロイ。
[zaki@cloud-dev microservices-demo]$ grep -v namespace deploy/kubernetes/complete-demo.yaml | kubectl apply -f - -n sock-shop-mesh deployment.apps/carts-db unchanged service/carts-db unchanged deployment.apps/carts unchanged service/carts unchanged deployment.apps/catalogue-db unchanged service/catalogue-db unchanged deployment.apps/catalogue unchanged service/catalogue unchanged deployment.apps/front-end unchanged service/front-end created ### <- これ deployment.apps/orders-db unchanged service/orders-db unchanged deployment.apps/orders unchanged service/orders unchanged deployment.apps/payment unchanged service/payment unchanged deployment.apps/queue-master unchanged service/queue-master unchanged deployment.apps/rabbitmq unchanged service/rabbitmq unchanged deployment.apps/shipping unchanged service/shipping unchanged deployment.apps/user-db unchanged service/user-db unchanged deployment.apps/user unchanged service/user unchanged
[zaki@cloud-dev microservices-demo]$ kc get pod -n sock-shop-mesh NAME READY STATUS RESTARTS AGE carts-7bbbd7779d-hvvzf 2/2 Running 0 70s carts-db-84b777d9c-mptnz 2/2 Running 0 70s catalogue-8684f655d9-gf6xn 0/2 PodInitializing 0 69s catalogue-db-5579f7f4cb-xqf48 2/2 Running 0 70s front-end-6f5fc69d6-dbqxn 0/2 PodInitializing 0 69s orders-7ccf68495-tgwqv 0/2 PodInitializing 0 69s orders-db-77c46b9c85-5k4zg 2/2 Running 0 69s payment-74976d749f-wfwjn 0/2 PodInitializing 0 68s queue-master-799b6b57d5-l4gq8 2/2 Running 0 68s rabbitmq-8458d7d4c-cbhcx 2/2 Running 0 68s shipping-5c89f4886c-m4s4n 0/2 PodInitializing 0 68s user-d6c48f668-krwgr 0/2 PodInitializing 0 68s user-db-5cdfd6d44b-gjdg4 0/2 PodInitializing 0 68s
この通り、コンテナ数が2/2
になっており、サイドカーのistio-proxyのインジェクションが行われている。
[zaki@cloud-dev microservices-demo]$ kc get pod -n sock-shop-mesh NAME READY STATUS RESTARTS AGE carts-7bbbd7779d-hvvzf 2/2 Running 0 2m15s carts-db-84b777d9c-mptnz 2/2 Running 0 2m15s catalogue-8684f655d9-gf6xn 2/2 Running 0 2m14s catalogue-db-5579f7f4cb-xqf48 2/2 Running 0 2m15s front-end-6f5fc69d6-dbqxn 2/2 Running 0 2m14s orders-7ccf68495-tgwqv 2/2 Running 0 2m14s orders-db-77c46b9c85-5k4zg 2/2 Running 0 2m14s payment-74976d749f-wfwjn 2/2 Running 0 2m13s queue-master-799b6b57d5-l4gq8 2/2 Running 0 2m13s rabbitmq-8458d7d4c-cbhcx 2/2 Running 0 2m13s shipping-5c89f4886c-m4s4n 2/2 Running 0 2m13s user-d6c48f668-krwgr 2/2 Running 0 2m13s user-db-5cdfd6d44b-gjdg4 2/2 Running 0 2m13s
デプロイ完了。
Gateway/VirtualService
Istio用のサイドカー込みのアプリケーションpodはデプロイできたので、次は外部アクセスに必要なリソースを作成。
最低限必要なのは、GatewayリソースとVirtualServiceリソース、それと、Istio IngressGateway経由でアクセスする際に使用するドメイン。(今回はIngress的にほかのアプリケーション用の定義との重複条件が無いので無くても良い:hosts: "*"
としている)
GatewayとVirtualServiceはBookInfoの設定を参考。
Gateway
apiVersion: networking.istio.io/v1beta1 kind: Gateway metadata: name: sock-shop spec: selector: istio: ingressgateway servers: - hosts: - '*' port: # 外部からのIstio IngressGatewayへのアクセスについての定義を記述 name: http number: 80 protocol: HTTP
VirtualService
apiVersion: networking.istio.io/v1beta1 kind: VirtualService metadata: name: sock-shop spec: gateways: - sock-shop # 上記の対象Gatewayリソース名 hosts: - '*' http: - route: - destination: # このリソースが受けたトラフィックをどのServiceへ転送するかを指定(これがNodePortは不要だけどServiceが必要と書いた部分) host: front-end port: number: 80
ソースはこちら
デプロイして実行
[zaki@cloud-dev sock-shop]$ kc apply -f sockshop-gateway.yaml -n sock-shop-mesh gateway.networking.istio.io/sock-shop created virtualservice.networking.istio.io/bsock-shop created
[zaki@cloud-dev sock-shop]$ kc get svc -n istio-system istio-ingressgateway NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE istio-ingressgateway LoadBalancer 10.104.82.61 192.168.0.182 15020:31343/TCP,80:30446/TCP,443:31013/TCP,31400:32549/TCP,15443:31115/TCP 6d1h
オリジナルのpodはNodePortを使ったけれど、Istio上のpodへのアクセスはIstio IngressGatewayを使用する。
※ NodePort Serviceでなく、Istio IngressGatewayを使っているので、Istio未使用版のURLと異なっている。
Kialiでトラフィックを確認
Istioで動いてるので、Kiali使って状態を確認もできる。
Kialiはこちらの記事もどぞ。
春から初夏まで仕事でIstioを ちょっとだけ 触ってて(ほぼトラブルシューティング)、もっと基礎力を上げたくて何か良い題材がないかツイートしたところ、OpenShift環境でSock Shopをアレンジしている方からリプ頂いていたのに、なかなか時間取れずに間が空いてしまいましたが、2020年8月のKubernetes Meetup Tokyo #33でもSock ShopとKialiの話が出てきて、今こそやらねば!と、ひとまずコミュニティ版IstioでSock Shop対応してみました。
mtls化やサーキットブレーカーなどやりましたか?最近、bookinfoではないのですが、sockshopでこんなの作りました。すごく基礎的で網羅的ではないですがhttps://t.co/1CREXwjetf
— もーすけ (@mosuke5) 2020年7月1日