EKSのCLIツールであるeksctlを使ってEKSクラスタを作った時のメモ。
必要な操作はすべて公式ドキュメントに載っているので、野良記事を参考にせずに公式ドキュメントを見ること。
CLIツールをインストール
- aws
- kubectl
- eksctl
の3つを用意。
awsはこちら
kubectlはkubeadm使って構築した時のkubectlがあるのでそれをそのまま使用。
(新たにインストールはしていない)
そしてeksctl。
と言ってもバイナリ落として/usr/local/binに配置するだけ。
[zaki@cloud-dev eksctl]$ curl --silent --location "https://github.com/weaveworks/eksctl/releases/latest/download/eksctl_$(uname -s)_amd64.tar.gz" | tar xz -C /tmp [zaki@cloud-dev eksctl]$ sudo mv /tmp/eksctl /usr/local/bin/ [zaki@cloud-dev eksctl]$ eksctl version 0.23.0
ちなみにこれらは全てAWSとは関係ない自宅PCのLinux(CentOS 7)に入れている。
EKSクラスタの作成
WebコンソールでEC2でVM作っていろいろ始めようとしたとき、ネットワークやら何やらいろいろ作成した面倒な記憶があったけど、eksctl使ったクラスタ作成は、コマンド1発。
ヘルプメッセージを見たかったのに、やらかしてhelpという名前のクラスタを作成する際の実行例です。
チョマテヨ！！って素で口走ってしまった…
— z a k i (@zaki_hmkc) 2020年7月6日
(--helpにしなかった自分が悪い) pic.twitter.com/RUgMEON4i3
[zaki@cloud-dev eksctl]$ eksctl create cluster help
[ℹ] eksctl version 0.23.0
[ℹ] using region ap-northeast-1
[ℹ] setting availability zones to [ap-northeast-1d ap-northeast-1a ap-northeast-1c]
[ℹ] subnets for ap-northeast-1d - public:192.168.0.0/19 private:192.168.96.0/19
[ℹ] subnets for ap-northeast-1a - public:192.168.32.0/19 private:192.168.128.0/19
[ℹ] subnets for ap-northeast-1c - public:192.168.64.0/19 private:192.168.160.0/19
[ℹ] nodegroup "ng-fe9db98d" will use "ami-0f3b1f2c30ccbc54d" [AmazonLinux2/1.16]
[ℹ] using Kubernetes version 1.16
[ℹ] creating EKS cluster "help" in "ap-northeast-1" region with un-managed nodes
[ℹ] will create 2 separate CloudFormation stacks for cluster itself and the initial nodegroup
[ℹ] if you encounter any issues, check CloudFormation console or try 'eksctl utils describe-stacks --region=ap-northeast-1 --cluster=help'
[ℹ] CloudWatch logging will not be enabled for cluster "help" in "ap-northeast-1"
[ℹ] you can enable it with 'eksctl utils update-cluster-logging --region=ap-northeast-1 --cluster=help'
[ℹ] Kubernetes API endpoint access will use default of {publicAccess=true, privateAccess=false} for cluster "help" in "ap-northeast-1"
[ℹ] 2 sequential tasks: { create cluster control plane "help", 2 sequential sub-tasks: { no tasks, create nodegroup "ng-fe9db98d" } }
[ℹ] building cluster stack "eksctl-help-cluster"
[ℹ] deploying stack "eksctl-help-cluster"
[ℹ] building nodegroup stack "eksctl-help-nodegroup-ng-fe9db98d"
[ℹ] --nodes-min=2 was set automatically for nodegroup ng-fe9db98d
[ℹ] --nodes-max=2 was set automatically for nodegroup ng-fe9db98d
[ℹ] deploying stack "eksctl-help-nodegroup-ng-fe9db98d"
[ℹ] waiting for the control plane availability...
[✔] saved kubeconfig as "/home/zaki/.kube/config"
[ℹ] no tasks
[✔] all EKS cluster resources for "help" have been created
[ℹ] adding identity "arn:aws:iam::********:role/eksctl-help-nodegroup-ng-fe9db98d-NodeInstanceRole-********" to auth ConfigMap
[ℹ] nodegroup "ng-fe9db98d" has 0 node(s)
[ℹ] waiting for at least 2 node(s) to become ready in "ng-fe9db98d"
[ℹ] nodegroup "ng-fe9db98d" has 2 node(s)
[ℹ] node "ip-192-168-29-220.ap-northeast-1.compute.internal" is ready
[ℹ] node "ip-192-168-83-201.ap-northeast-1.compute.internal" is ready
[ℹ] kubectl command should work with "/home/zaki/.kube/config", try 'kubectl get nodes'
[✔] EKS cluster "help" in "ap-northeast-1" region is ready
[zaki@cloud-dev eksctl]$
作成中はwebコンソールで見ると「作成中」になってるのが分かる。
timeコマンドもつけ忘れてたから正確にはわからないけど、たぶん20分くらいでクラスタ完成。
認証情報
この時点で~/.kube/configも更新されて、作成したEKSクラスタの認証情報が設定される。
[zaki@cloud-dev eksctl]$ kubectl config get-contexts
CURRENT NAME CLUSTER AUTHINFO NAMESPACE
kubernetes-admin@kubernetes kubernetes kubernetes-admin
* zaki@help.ap-northeast-1.eksctl.io help.ap-northeast-1.eksctl.io zaki@help.ap-northeast-1.eksctl.io
version
[zaki@cloud-dev eksctl]$ kubectl version
Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.5", GitCommit:"e6503f8d8f769ace2f338794c914a96fc335df0f", GitTreeState:"clean", BuildDate:"2020-06-26T03:47:41Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"16+", GitVersion:"v1.16.8-eks-e16311", GitCommit:"e163110a04dcb2f39c3325af96d019b4925419eb", GitTreeState:"clean", BuildDate:"2020-03-27T22:37:12Z", GoVersion:"go1.13.8", Compiler:"gc", Platform:"linux/amd64"}
ノード情報
オプション無しでクラスタ作成したときのノード情報。
[zaki@cloud-dev eksctl]$ kubectl get node NAME STATUS ROLES AGE VERSION ip-192-168-29-220.ap-northeast-1.compute.internal Ready <none> 3m58s v1.16.8-eks-fd1ea7 ip-192-168-83-201.ap-northeast-1.compute.internal Ready <none> 3m58s v1.16.8-eks-fd1ea7
ノードのスペック情報
[zaki@cloud-dev eksctl]$ kubectl describe node ip-192-168-29-220.ap-northeast-1.compute.internal Name: ip-192-168-29-220.ap-northeast-1.compute.internal Roles: <none> : : Capacity: attachable-volumes-aws-ebs: 25 cpu: 2 ephemeral-storage: 83873772Ki hugepages-1Gi: 0 hugepages-2Mi: 0 memory: 7865020Ki pods: 29 Allocatable: attachable-volumes-aws-ebs: 25 cpu: 1930m ephemeral-storage: 76224326324 hugepages-1Gi: 0 hugepages-2Mi: 0 memory: 5875388Ki pods: 29
Webで確認するとこの通りで、m5.largeのインスタンスがノードとして作成されたのが分かる。
namespaceとpod
[zaki@cloud-dev eksctl]$ kubectl get ns NAME STATUS AGE default Active 13m kube-node-lease Active 13m kube-public Active 13m kube-system Active 13m [zaki@cloud-dev eksctl]$ kubectl get pod -A NAMESPACE NAME READY STATUS RESTARTS AGE kube-system aws-node-gflrw 1/1 Running 0 5m38s kube-system aws-node-w4rz9 1/1 Running 0 5m38s kube-system coredns-cdd78ff87-kbwsf 1/1 Running 0 13m kube-system coredns-cdd78ff87-krws4 1/1 Running 0 13m kube-system kube-proxy-9w4xg 1/1 Running 0 5m38s kube-system kube-proxy-zh99j 1/1 Running 0 5m38s
サンプルアプリ(自前)
こんなマニフェストでデプロイしてみる。
apiVersion: apps/v1 kind: Deployment metadata: labels: app: sample-http name: sample-http spec: replicas: 2 selector: matchLabels: app: sample-http template: metadata: labels: app: sample-http spec: containers: - image: httpd name: httpd --- apiVersion: v1 kind: Service metadata: labels: app: sample-http name: sample-http spec: ports: - port: 80 protocol: TCP targetPort: 80 selector: app: sample-http type: LoadBalancer
[zaki@cloud-dev sample]$ kubectl apply -f sample-http.yaml deployment.apps/sample-http created service/sample-http created
type:LoadBalancer Serviceが作成される。
[zaki@cloud-dev sample]$ kubectl get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP 10.100.0.1 <none> 443/TCP 15m sample-http LoadBalancer 10.100.58.190 ac728d278df6846269a1ebf87b983b98-497542522.ap-northeast-1.elb.amazonaws.com 80:32106/TCP 6s
確認し忘れたけど、webコンソールで見てもロードバランサーが作成されてたはず。
[zaki@cloud-dev sample]$ kubectl get pod NAME READY STATUS RESTARTS AGE sample-http-748f7cdcf4-4d4qt 1/1 Running 0 68s sample-http-748f7cdcf4-jslz8 1/1 Running 0 68s
DNSの名前解決ができるようになるまでちょっとタイムラグあるけど
[zaki@cloud-dev sample]$ curl ac728d278df6846269a1ebf87b983b98-497542522.ap-northeast-1.elb.amazonaws.com <html><body><h1>It works!</h1></body></html>
この通り。
クラスタ一覧
[zaki@cloud-dev sample]$ eksctl get cluster NAME REGION help ap-northeast-1
今作ったhelpという名前のクラスタがある。
クラスタ削除
[zaki@cloud-dev sample]$ eksctl delete cluster help
[ℹ] eksctl version 0.23.0
[ℹ] using region ap-northeast-1
[ℹ] deleting EKS cluster "help"
[ℹ] deleted 0 Fargate profile(s)
[✔] kubeconfig has been updated
[ℹ] cleaning up LoadBalancer services
[ℹ] 2 sequential tasks: { delete nodegroup "ng-fe9db98d", delete cluster control plane "help" [async] }
[ℹ] will delete stack "eksctl-help-nodegroup-ng-fe9db98d"
[ℹ] waiting for stack "eksctl-help-nodegroup-ng-fe9db98d" to get deleted
[ℹ] will delete stack "eksctl-help-cluster"
[✔] all cluster resources were deleted
削除は5分ほど。
削除されると~/.kube/configからも消える。
[zaki@cloud-dev sample]$ kubectl config get-contexts
CURRENT NAME CLUSTER AUTHINFO NAMESPACE
kubernetes-admin@kubernetes kubernetes kubernetes-admin
--help
[zaki@cloud-dev sample]$ eksctl create cluster --help
Create a cluster
Usage: eksctl create cluster [flags]
General flags:
-n, --name string EKS cluster name (generated if unspecified, e.g. "attractive-sheepdog-1594048272")
--tags stringToString Used to tag the AWS resources. List of comma separated KV pairs "k1=v1,k2=v2" (default [])
-r, --region string AWS region
--zones strings (auto-select if unspecified)
--version string Kubernetes version (valid options: 1.14, 1.15, 1.16) (default "1.16")
-f, --config-file string load configuration from a file (or stdin if set to '-')
--timeout duration maximum waiting time for any long-running operation (default 25m0s)
--install-vpc-controllers Install VPC controller that's required for Windows workloads
--managed Create EKS-managed nodegroup
--fargate Create a Fargate profile scheduling pods in the default and kube-system namespaces onto Fargate
Initial nodegroup flags:
--nodegroup-name string name of the nodegroup (generated if unspecified, e.g. "ng-e7cda11c")
--without-nodegroup if set, initial nodegroup will not be created
-t, --node-type string node instance type (default "m5.large")
-N, --nodes int total number of nodes (for a static ASG) (default 2)
-m, --nodes-min int minimum nodes in ASG (default 2)
-M, --nodes-max int maximum nodes in ASG (default 2)
--node-volume-size int node volume size in GB (default 80)
--node-volume-type string node volume type (valid options: gp2, io1, sc1, st1) (default "gp2")
--max-pods-per-node int maximum number of pods per node (set automatically if unspecified)
--ssh-access control SSH access for nodes. Uses ~/.ssh/id_rsa.pub as default key path if enabled
--ssh-public-key string SSH public key to use for nodes (import from local path, or use existing EC2 key pair)
--node-ami string Advanced use cases only. If 'ssm' is supplied (default) then eksctl will use SSM Parameter; if 'auto' is supplied then eksctl will
automatically set the AMI based on version/region/instance type; if static is supplied (deprecated), then static AMIs will be used; if any other value is supplied it will override the AMI to use for the nodes. Use with extreme care.
--node-ami-family string Advanced use cases only. If 'AmazonLinux2' is supplied (default), then eksctl will use the official AWS EKS AMIs (Amazon Linux 2);
if 'Ubuntu1804' is supplied, then eksctl will use the official Canonical EKS AMIs (Ubuntu 18.04). (default "AmazonLinux2")
-P, --node-private-networking whether to make nodegroup networking private
--node-security-groups strings Attach additional security groups to nodes, so that it can be used to allow extra ingress/egress access from/to pods
--node-labels stringToString Extra labels to add when registering the nodes in the nodegroup. List of comma separated KV pairs "k1=v1,k2=v2" (default [])
--node-zones strings (inherited from the cluster if unspecified)
--instance-prefix string Add a prefix value in front of the instance's name.
--instance-name string Overrides the default instance's name. It can be used in combination with the instance-prefix flag.
Cluster and nodegroup add-ons flags:
--install-neuron-plugin Install Neuron plugin for Inferentia nodes (default true)
--asg-access enable IAM policy for cluster-autoscaler
--external-dns-access enable IAM policy for external-dns
--full-ecr-access enable full access to ECR
--appmesh-access enable full access to AppMesh
--appmesh-preview-access enable full access to AppMesh Preview
--alb-ingress-access enable full access for alb-ingress-controller
VPC networking flags:
--vpc-cidr ipNet global CIDR to use for VPC (default 192.168.0.0/16)
--vpc-private-subnets strings re-use private subnets of an existing VPC
--vpc-public-subnets strings re-use public subnets of an existing VPC
--vpc-from-kops-cluster string re-use VPC from a given kops cluster
--vpc-nat-mode string VPC NAT mode, valid options: HighlyAvailable, Single, Disable (default "Single")
AWS client flags:
-p, --profile string AWS credentials profile to use (overrides the AWS_PROFILE environment variable)
--cfn-role-arn string IAM role used by CloudFormation to call AWS API on your behalf
Output kubeconfig flags:
--kubeconfig string path to write kubeconfig (incompatible with --auto-kubeconfig) (default "/home/zaki/.kube/config")
--authenticator-role-arn string AWS IAM role to assume for authenticator
--set-kubeconfig-context if true then current-context will be set in kubeconfig; if a context is already set then it will be overwritten (default true)
--auto-kubeconfig save kubeconfig file by cluster name, e.g. "/home/zaki/.kube/eksctl/clusters/attractive-sheepdog-1594048272"
--write-kubeconfig toggle writing of kubeconfig (default true)
Common flags:
-C, --color string toggle colorized logs (valid options: true, false, fabulous) (default "true")
-h, --help help for this command
-v, --verbose int set log level, use 0 to silence, 4 for debugging and 5 for debugging with AWS debug logging (default 3)
Use 'eksctl create cluster [command] --help' for more information about a command.
helpじゃなくて--helpにすれば、ちゃんと見れる。
-t, --node-type string node instance type (default "m5.large")
デフォルトでm5.largeのノードが作成されるのはこのオプションによるもの。
また、デフォルトではeksctl create clusterでノードグループが作成されるが、別途ノードグループを作成しておいて、それを後からアタッチするなど柔軟な構築もできる。
(その辺また別記事にしよう。。ちなみにinductorさんに少しお話聞いてみたら、普通は別々に作る方が良いらしい)
kubeadmで手作りもいいぞ
