前回はうっかりオプション何もなしでクラスタ作ってしまったので、eksctlで作成できるEKSクラスタ、オプションを指定してノードの設定などいろいろ初期設定してみる。
新規作成時のオプション
オプションをざっと見た感じでは、検証レベルで使用するには以下を指定すれば良さそう。(個人の感想)
あと--versionも必要に応じて。
$ eksctl create cluster \
--name ${cluster_name} \
--region ap-northeast-1 \
--nodegroup-name ${node_group_name} \
--node-type t3.micro \
--nodes 3 \
--node-volume-size 40 \
--ssh-access \
--ssh-public-key ~/.ssh/id_rsa.pub \
--managed
次点で--node-amiでAMI(デフォルトでAmazonLinux2)の指定あたりかな。
オプション名そのままなので、説明は多分要らないと思う。
最初ミスったのは--ssh-public-keyで、ノードに送り込みたい公開鍵のファイルパスを指定すればOK。(鍵ファイルの内容を最初指定してた笑)
実行例
[zaki@cloud-dev ~]$ time eksctl create cluster \
> --name zaki-eks \
> --region ap-northeast-1 \
> --nodegroup-name zaki-ng \
> --node-type t3.micro \
> --nodes 3 \
> --node-volume-size 20 \
> --ssh-access \
> --ssh-public-key ~/.ssh/id_rsa.pub \
> --managed
[ℹ] eksctl version 0.23.0
[ℹ] using region ap-northeast-1
[ℹ] setting availability zones to [ap-northeast-1c ap-northeast-1d ap-northeast-1a]
[ℹ] subnets for ap-northeast-1c - public:192.168.0.0/19 private:192.168.96.0/19
[ℹ] subnets for ap-northeast-1d - public:192.168.32.0/19 private:192.168.128.0/19
[ℹ] subnets for ap-northeast-1a - public:192.168.64.0/19 private:192.168.160.0/19
[ℹ] using SSH public key "/home/zaki/.ssh/id_rsa.pub" as "eksctl-zaki-eks-nodegroup-zaki-ng-9c:3a:59:75:86:d8:8c:6a:a4:a0:ef:55:4f:62:c5:0d"
[ℹ] using Kubernetes version 1.16
[ℹ] creating EKS cluster "zaki-eks" in "ap-northeast-1" region with managed nodes
[ℹ] will create 2 separate CloudFormation stacks for cluster itself and the initial managed nodegroup
[ℹ] if you encounter any issues, check CloudFormation console or try 'eksctl utils describe-stacks --region=ap-northeast-1 --cluster=zaki-eks'
[ℹ] CloudWatch logging will not be enabled for cluster "zaki-eks" in "ap-northeast-1"
[ℹ] you can enable it with 'eksctl utils update-cluster-logging --region=ap-northeast-1 --cluster=zaki-eks'
[ℹ] Kubernetes API endpoint access will use default of {publicAccess=true, privateAccess=false} for cluster "zaki-eks" in "ap-northeast-1"
[ℹ] 2 sequential tasks: { create cluster control plane "zaki-eks", 2 sequential sub-tasks: { no tasks, create managed nodegroup "zaki-ng" } }
[ℹ] building cluster stack "eksctl-zaki-eks-cluster"
[ℹ] deploying stack "eksctl-zaki-eks-cluster"
[ℹ] building managed nodegroup stack "eksctl-zaki-eks-nodegroup-zaki-ng"
[ℹ] deploying stack "eksctl-zaki-eks-nodegroup-zaki-ng"
[ℹ] waiting for the control plane availability...
[✔] saved kubeconfig as "/home/zaki/.kube/config"
[ℹ] no tasks
[✔] all EKS cluster resources for "zaki-eks" have been created
[ℹ] nodegroup "zaki-ng" has 3 node(s)
[ℹ] node "ip-192-168-27-102.ap-northeast-1.compute.internal" is ready
[ℹ] node "ip-192-168-36-215.ap-northeast-1.compute.internal" is ready
[ℹ] node "ip-192-168-77-134.ap-northeast-1.compute.internal" is ready
[ℹ] waiting for at least 3 node(s) to become ready in "zaki-ng"
[ℹ] nodegroup "zaki-ng" has 3 node(s)
[ℹ] node "ip-192-168-27-102.ap-northeast-1.compute.internal" is ready
[ℹ] node "ip-192-168-36-215.ap-northeast-1.compute.internal" is ready
[ℹ] node "ip-192-168-77-134.ap-northeast-1.compute.internal" is ready
[ℹ] kubectl command should work with "/home/zaki/.kube/config", try 'kubectl get nodes'
[✔] EKS cluster "zaki-eks" in "ap-northeast-1" region is ready
real 15m52.252s
user 0m1.080s
sys 0m0.220s
[zaki@cloud-dev ~]$ kubectl get node NAME STATUS ROLES AGE VERSION ip-192-168-27-102.ap-northeast-1.compute.internal Ready <none> 6m40s v1.16.8-eks-fd1ea7 ip-192-168-36-215.ap-northeast-1.compute.internal Ready <none> 6m38s v1.16.8-eks-fd1ea7 ip-192-168-77-134.ap-northeast-1.compute.internal Ready <none> 6m26s v1.16.8-eks-fd1ea7
デプロイ可能なpod数について
describeするとこの通り。
Capacity: attachable-volumes-aws-ebs: 25 cpu: 2 ephemeral-storage: 20959212Ki hugepages-1Gi: 0 hugepages-2Mi: 0 memory: 980068Ki pods: 4 Allocatable: attachable-volumes-aws-ebs: 25 cpu: 1930m ephemeral-storage: 18242267924 hugepages-1Gi: 0 hugepages-2Mi: 0 memory: 571492Ki pods: 4
t3.micro使用時のデプロイできるpod数4というのはなかなか厳しいが、システム系DaemonSetも乗ったりするので、t3.microではユーザーがデプロイできるpodは実質1か2だと思って良い。
ちなみにEKSではノードサイズによってデプロイ可能なpod数(正確にはpodに割り当てることができるIPアドレス数)が決まっているため、利用時は注意。
[zaki@cloud-dev ~]$ kc get pod -A NAMESPACE NAME READY STATUS RESTARTS AGE kube-system aws-node-f89fq 1/1 Running 0 10m kube-system aws-node-x7bdl 1/1 Running 0 11m kube-system aws-node-x7h9w 1/1 Running 0 11m kube-system coredns-cdd78ff87-82mdr 1/1 Running 0 15m kube-system coredns-cdd78ff87-vndtb 1/1 Running 0 15m kube-system kube-proxy-897l6 1/1 Running 0 11m kube-system kube-proxy-cmqmc 1/1 Running 0 11m kube-system kube-proxy-rddzb 1/1 Running 0 10m
ノードにsshしてみる
アドレスはwebコンソールやkubectl get node -o wideで確認できる。
[zaki@cloud-dev ~]$ ssh ec2-user@---.---.---.---
The authenticity of host '---.---.---.--- (---.---.---.---)' can't be established.
ECDSA key fingerprint is SHA256:hr4KTgg0V+oBtIAgkyT85ayae8b8iY+eq/Vs00ZcmbE.
ECDSA key fingerprint is MD5:9e:ee:1a:d9:5a:52:8b:09:1c:df:de:c8:b2:95:56:3b.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '---.---.---.---' (ECDSA) to the list of known hosts.
Last login: Thu Jun 18 01:20:30 2020 from ---.---.---.---
__| __|_ )
_| ( / Amazon Linux 2 AMI
___|\___|___|
https://aws.amazon.com/amazon-linux-2/
4 package(s) needed for security, out of 10 available
Run "sudo yum update" to apply all updates.
[ec2-user@ip-192-168-27-102 ~]$ ls
[ec2-user@ip-192-168-27-102 ~]$ df -h
ファイルシス サイズ 使用 残り 使用% マウント位置
devtmpfs 466M 0 466M 0% /dev
tmpfs 479M 0 479M 0% /dev/shm
tmpfs 479M 576K 478M 1% /run
tmpfs 479M 0 479M 0% /sys/fs/cgroup
/dev/nvme0n1p1 20G 2.2G 18G 11% /
tmpfs 96M 0 96M 0% /run/user/1000
ストレージも20GBになってる。
ノードグループの作成を個別に実施
ノードグループを作成しクラスタへ追加
$ eksctl create nodegroup \
--name ${node_group_name} \
--region ap-northeast-1 \
--cluster ${cluster_name} \
--node-type t3.micro \
--nodes 2 \
--node-volume-size 40 \
--ssh-access \
--ssh-public-key ~/.ssh/id_rsa.pub \
--managed
実行例
[zaki@cloud-dev ~]$ time eksctl create nodegroup \
> --name zaki-small-ng \
> --region ap-northeast-1 \
> --cluster zaki-eks \
> --node-type t3.small \
> --nodes 1 \
> --node-volume-size 30 \
> --ssh-access \
> --ssh-public-key ~/.ssh/id_rsa.pub \
> --managed
[ℹ] eksctl version 0.23.0
[ℹ] using region ap-northeast-1
[ℹ] will use version 1.16 for new nodegroup(s) based on control plane version
[ℹ] 1 existing nodegroup(s) (zaki-ng) will be excluded
[ℹ] using SSH public key "/home/zaki/.ssh/id_rsa.pub" as "eksctl-zaki-eks-nodegroup-zaki-small-ng-9c:3a:59:75:86:d8:8c:6a:a4:a0:ef:55:4f:62:c5:0d"
[ℹ] 1 nodegroup (zaki-small-ng) was included (based on the include/exclude rules)
[ℹ] combined exclude rules: zaki-ng
[ℹ] no nodegroups present in the current set were excluded by the filter
[ℹ] will create a CloudFormation stack for each of 1 managed nodegroups in cluster "zaki-eks"
[ℹ] 2 sequential tasks: { fix cluster compatibility, 1 task: { 1 task: { create managed nodegroup "zaki-small-ng" } } }
[ℹ] checking cluster stack for missing resources
[ℹ] cluster stack has all required resources
[ℹ] building managed nodegroup stack "eksctl-zaki-eks-nodegroup-zaki-small-ng"
[ℹ] deploying stack "eksctl-zaki-eks-nodegroup-zaki-small-ng"
[ℹ] no tasks
[✔] created 0 nodegroup(s) in cluster "zaki-eks"
[ℹ] nodegroup "zaki-small-ng" has 1 node(s)
[ℹ] node "ip-192-168-63-151.ap-northeast-1.compute.internal" is ready
[ℹ] waiting for at least 1 node(s) to become ready in "zaki-small-ng"
[ℹ] nodegroup "zaki-small-ng" has 1 node(s)
[ℹ] node "ip-192-168-63-151.ap-northeast-1.compute.internal" is ready
[✔] created 1 managed nodegroup(s) in cluster "zaki-eks"
[ℹ] checking security group configuration for all nodegroups
[ℹ] all nodegroups have up-to-date configuration
real 3m0.456s
user 0m0.133s
sys 0m0.059s
1ノードのノードグループを作成したので、最初の3ノードと合わせて計4ノードになった。
[zaki@cloud-dev ~]$ kubectl get node NAME STATUS ROLES AGE VERSION ip-192-168-27-102.ap-northeast-1.compute.internal Ready <none> 19m v1.16.8-eks-fd1ea7 ip-192-168-36-215.ap-northeast-1.compute.internal Ready <none> 19m v1.16.8-eks-fd1ea7 ip-192-168-63-151.ap-northeast-1.compute.internal Ready <none> 4m50s v1.16.8-eks-fd1ea7 ip-192-168-77-134.ap-northeast-1.compute.internal Ready <none> 18m v1.16.8-eks-fd1ea7
追加されたノードをdescribeすると、
Capacity: attachable-volumes-aws-ebs: 25 cpu: 2 ephemeral-storage: 31444972Ki hugepages-1Gi: 0 hugepages-2Mi: 0 memory: 2002020Ki pods: 11 Allocatable: attachable-volumes-aws-ebs: 25 cpu: 1930m ephemeral-storage: 27905944324 hugepages-1Gi: 0 hugepages-2Mi: 0 memory: 1514596Ki pods: 11
t3.smallのノードが追加された。
クラスタのみ作成(ノード無し)
$ eksctl create cluster \
--name ${cluster_name} \
--region ap-northeast-1 \
--without-nodegroup \
--managed
[zaki@cloud-dev ~]$ time eksctl create cluster \
> --name new-eks \
> --region ap-northeast-1 \
> --without-nodegroup \
> --managed
[ℹ] eksctl version 0.23.0
[ℹ] using region ap-northeast-1
[ℹ] setting availability zones to [ap-northeast-1a ap-northeast-1c ap-northeast-1d]
[ℹ] subnets for ap-northeast-1a - public:192.168.0.0/19 private:192.168.96.0/19
[ℹ] subnets for ap-northeast-1c - public:192.168.32.0/19 private:192.168.128.0/19
[ℹ] subnets for ap-northeast-1d - public:192.168.64.0/19 private:192.168.160.0/19
[ℹ] using Kubernetes version 1.16
[ℹ] creating EKS cluster "new-eks" in "ap-northeast-1" region with
[ℹ] if you encounter any issues, check CloudFormation console or try 'eksctl utils describe-stacks --region=ap-northeast-1 --cluster=new-eks'
[ℹ] CloudWatch logging will not be enabled for cluster "new-eks" in "ap-northeast-1"
[ℹ] you can enable it with 'eksctl utils update-cluster-logging --region=ap-northeast-1 --cluster=new-eks'
[ℹ] Kubernetes API endpoint access will use default of {publicAccess=true, privateAccess=false} for cluster "new-eks" in "ap-northeast-1"
[ℹ] 2 sequential tasks: { create cluster control plane "new-eks", no tasks }
[ℹ] building cluster stack "eksctl-new-eks-cluster"
[ℹ] deploying stack "eksctl-new-eks-cluster"
[ℹ] waiting for the control plane availability...
[✔] saved kubeconfig as "/home/zaki/.kube/config"
[ℹ] no tasks
[✔] all EKS cluster resources for "new-eks" have been created
[ℹ] kubectl command should work with "/home/zaki/.kube/config", try 'kubectl get nodes'
[✔] EKS cluster "new-eks" in "ap-northeast-1" region is ready
real 13m44.178s
user 0m1.057s
sys 0m0.176s
この場合、クラスタはあるけどノードは無い状態となる。
[zaki@cloud-dev ~]$ kubectl get node No resources found in default namespace.
このあとノードグループをこのクラスタへ追加すればOK
ノードのスケーリング
$ eksctl scale nodegroup \
--name ${node_group_name} \
--cluster ${cluster_name} \
--region ap-northeast-1 \
--node 1
3ノードで作成したノードグループを1ノードに変更してみる。
[zaki@cloud-dev ~]$ time eksctl scale nodegroup \ > --name zaki-ng \ > --cluster zaki-eks \ > --region ap-northeast-1 \ > --nodes 1 [ℹ] scaling nodegroup stack "eksctl-zaki-eks-nodegroup-zaki-ng" in cluster eksctl-zaki-eks-cluster [!] the desired nodes 1 is less than current nodes-min/minSize 3 Error: failed to scale nodegroup for cluster "zaki-eks", error the desired nodes 1 is less than current nodes-min/minSize 3 real 0m0.613s user 0m0.068s sys 0m0.015s
あ、minを未指定でノードグループ作ったから減らせない。。
--nodes-minも同時指定して再度実行。
[zaki@cloud-dev ~]$ time eksctl scale nodegroup \ > --name zaki-ng \ > --cluster zaki-eks \ > --region ap-northeast-1 \ > --nodes-min 1 \ > --nodes 1 [ℹ] scaling nodegroup stack "eksctl-zaki-eks-nodegroup-zaki-ng" in cluster eksctl-zaki-eks-cluster [ℹ] scaling nodegroup, desired capacity from 3 to 1, min size from 3 to 1 real 1m10.660s user 0m0.070s sys 0m0.030s [zaki@cloud-dev ~]$ kubectl get node NAME STATUS ROLES AGE VERSION ip-192-168-27-102.ap-northeast-1.compute.internal Ready <none> 26m v1.16.8-eks-fd1ea7 ip-192-168-36-215.ap-northeast-1.compute.internal Ready,SchedulingDisabled <none> 26m v1.16.8-eks-fd1ea7 ip-192-168-63-151.ap-northeast-1.compute.internal Ready <none> 11m v1.16.8-eks-fd1ea7 ip-192-168-77-134.ap-northeast-1.compute.internal NotReady,SchedulingDisabled <none> 25m v1.16.8-eks-fd1ea7
しばらく待てば
[zaki@cloud-dev ~]$ kubectl get node NAME STATUS ROLES AGE VERSION ip-192-168-27-102.ap-northeast-1.compute.internal Ready <none> 27m v1.16.8-eks-fd1ea7 ip-192-168-63-151.ap-northeast-1.compute.internal Ready <none> 12m v1.16.8-eks-fd1ea7
これで現状zaki-ngのノードが1台、zaki-small-ngが1台の、ノードサイズの異なる計2台でワーカーノードが起動している。
ノードグループの一覧
$ eksctl get nodegroup \
--cluster ${node_group_name}
[zaki@cloud-dev ~]$ eksctl get nodegroup --cluster zaki-eks CLUSTER NODEGROUP CREATED MIN SIZE MAX SIZE DESIRED CAPACITY INSTANCE TYPE IMAGE ID zaki-eks zaki-ng 2020-07-09T11:01:52Z 1 3 1 t3.micro zaki-eks zaki-small-ng 2020-07-09T11:16:20Z 1 1 1 t3.small
ノードグループの削除
$ eksctl delete nodegroup \
--name ${node_group_name} \
--cluster ${cluster_name} \
--region ap-northeast-1
削除対象となるノードに乗っていたpodは退避される。
削除前は以下の通り。
pod退避のbefore/afterを見るように、replicas=2の適当なwebサーバーpodをデプロイしてる。
[zaki@cloud-dev ~]$ kubectl get pod -A -o wide NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES default sample-http-748f7cdcf4-59bbb 1/1 Running 0 22s 192.168.21.127 ip-192-168-27-102.ap-northeast-1.compute.internal <none> <none> default sample-http-748f7cdcf4-dv27s 1/1 Running 0 22s 192.168.46.194 ip-192-168-63-151.ap-northeast-1.compute.internal <none> <none> kube-system aws-node-cttg4 1/1 Running 0 23m 192.168.63.151 ip-192-168-63-151.ap-northeast-1.compute.internal <none> <none> kube-system aws-node-x7h9w 1/1 Running 0 37m 192.168.27.102 ip-192-168-27-102.ap-northeast-1.compute.internal <none> <none> kube-system coredns-cdd78ff87-82mdr 1/1 Running 0 42m 192.168.7.154 ip-192-168-27-102.ap-northeast-1.compute.internal <none> <none> kube-system coredns-cdd78ff87-8q6x8 1/1 Running 0 12m 192.168.42.160 ip-192-168-63-151.ap-northeast-1.compute.internal <none> <none> kube-system kube-proxy-4fhjj 1/1 Running 0 23m 192.168.63.151 ip-192-168-63-151.ap-northeast-1.compute.internal <none> <none> kube-system kube-proxy-cmqmc 1/1 Running 0 37m 192.168.27.102 ip-192-168-27-102.ap-northeast-1.compute.internal <none> <none>
ノードグループを削除。
[zaki@cloud-dev ~]$ time eksctl delete nodegroup \
> --name zaki-ng \
> --cluster zaki-eks \
> --region ap-northeast-1
[ℹ] eksctl version 0.23.0
[ℹ] using region ap-northeast-1
[ℹ] combined include rules: zaki-ng
[ℹ] 1 nodegroup (zaki-ng) was included (based on the include/exclude rules)
[ℹ] will drain 1 nodegroup(s) in cluster "zaki-eks"
[ℹ] cordon node "ip-192-168-27-102.ap-northeast-1.compute.internal"
[!] ignoring DaemonSet-managed Pods: kube-system/aws-node-x7h9w, kube-system/kube-proxy-cmqmc
[!] ignoring DaemonSet-managed Pods: kube-system/aws-node-x7h9w, kube-system/kube-proxy-cmqmc
[✔] drained nodes: [ip-192-168-27-102.ap-northeast-1.compute.internal]
[ℹ] will delete 1 nodegroups from cluster "zaki-eks"
[ℹ] 1 task: { delete nodegroup "zaki-ng" [async] }
[ℹ] will delete stack "eksctl-zaki-eks-nodegroup-zaki-ng"
[ℹ] will delete 0 nodegroups from auth ConfigMap in cluster "zaki-eks"
[✔] deleted 1 nodegroup(s) from cluster "zaki-eks"
real 0m1.699s
user 0m0.102s
sys 0m0.016s
[zaki@cloud-dev ~]$ kubectl get pod -A -o wide NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES default sample-http-748f7cdcf4-c2sg5 1/1 Running 0 117s 192.168.43.194 ip-192-168-63-151.ap-northeast-1.compute.internal <none> <none> default sample-http-748f7cdcf4-dv27s 1/1 Running 0 3m53s 192.168.46.194 ip-192-168-63-151.ap-northeast-1.compute.internal <none> <none> kube-system aws-node-cttg4 1/1 Running 0 26m 192.168.63.151 ip-192-168-63-151.ap-northeast-1.compute.internal <none> <none> kube-system coredns-cdd78ff87-6xtlz 1/1 Running 0 117s 192.168.42.82 ip-192-168-63-151.ap-northeast-1.compute.internal <none> <none> kube-system coredns-cdd78ff87-8q6x8 1/1 Running 0 16m 192.168.42.160 ip-192-168-63-151.ap-northeast-1.compute.internal <none> <none> kube-system kube-proxy-4fhjj 1/1 Running 0 26m 192.168.63.151 ip-192-168-63-151.ap-northeast-1.compute.internal <none> <none>
DaemonSetでないreplicas=2のsample-httpが、残ったノードに移動している。
