2020.06.10
1. K8s 설치 개요
내용: K8s 1.16.15 (Stacked control plane nodes) install
환경 : Centos 7.8, Baremetal server
참조 : Creating Highly Available clusters with kubeadm
https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/high-availability/
2. Stacked control plane nodes
3. 사전 작업
- K8s master nodes & worker nodes 사전 작업
$ sudo su -
# swapoff -a && echo 0 > /proc/sys/vm/swappiness # /etc/fstab에서 swap절 주석 처리 할 것
# echo "vm.swappiness=0" >> /etc/sysctl.conf
# systemctl disable firewalld && systemctl stop firewalld
# setenforce 0 && sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config
# modprobe br_netfilter && lsmod | grep br_netfilter
# cat <<EOF > /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
# sysctl --system
# yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
# yum list docker-ce --showduplicates | sort -r | grep "19.03"
docker-ce.x86_64 3:19.03.15-3.el7 docker-ce-stable
...
# yum install docker-ce-19.03.15-3.el7 docker-ce-cli-19.03.15-3.el7 containerd.io -y
# systemctl enable docker.service --now
- K8s 1.15.11 설치 시 실행
# echo 1 > /proc/sys/net/ipv4/ip_forward
# echo 'net.ipv4.ip_forward = 1' >> /etc/sysctl.conf
4. Kubernetes 설치하기
4.1 K8s 주요 명령어 설치 (전체 노드)
# cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-$basearch
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
exclude=kubelet kubeadm kubectl
#
## K8s 특정 버전 설치
# yum install -y kubeadm-1.16.15 kubelet-1.16.15 kubectl-1.16.15 --disableexcludes=kubernetes
# yum install -y kubeadm-1.15.11 kubelet-1.15.11 kubectl-1.15.11 --disableexcludes=kubernetes
## K8s 최신 버전 설치
# yum install y kubelet kubeadm kubectl --disableexcludes=kubernetes
# systemctl enable kubelet --now
4.2 K8s Master node install - case #1
- kubeadm 1.16 이상 (1st)
# kubeadm init --control-plane-endpoint="14.52.244.136:7443" --upload-certs --kubernetes-version=1.16.15
or
# kubeadm init --control-plane-endpoint="14.52.244.136:7443" --upload-certs --kubernetes-version=1.16.15 \
--pod-network-cidr=10.244.0.0/16
## The --pod-network-cidr=10.244.0.0/16 option is a requirement for Flannel - don't change that network address!
- kubeadm 1.15 이하 (1st)
# vi kubeadm-config.yaml
apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
kubernetesVersion: 1.15.12
controlPlaneEndpoint: "14.52.244.136:7443"
# kubeadm init --config=kubeadm-config.yaml --upload-certs
- K8s Master nodes (2nd, 3rd)
# kubeadm join 14.52.244.136:7443 --token tqte4l.m15k0fwtvqp3aa1p \
--discovery-token-ca-cert-hash sha256:580dee2eaa1bd376ae8344a4e69d06553e3901be0bbaec76f14bdc8cded4aa3d \
--control-plane \
--certificate-key e31030aa8f9f8f943057fd1d57732ce5b2e88834bff3dfa435e78aa1cf2c9ab0
4.3 K8s Master node install - case #2
- K8s를 설치할 노드가 2개 이상의 인터페이스를 가지고 있고, private network(디폴트 인터페이스가 아님)를 사용하고자 할 경우
https://stackoverflow.com/questions/54722289/kubernetes-internet-access-with-two-network-interfaces
- K8s Master nodes (1st)
# kubeadm init --control-plane-endpoint="192.168.100.10:7443" \
--apiserver-advertise-address=192.168.100.11 \
--upload-certs --kubernetes-version=1.16.15 --pod-network-cidr=10.244.0.0/16 --v=5
- K8s Master nodes (2nd, 3rd)
# kubeadm join 192.168.100.10:7443 --token osvhga.a26utx39g1y3dqew \
--discovery-token-ca-cert-hash sha256:d106ad18f72f1c091448c06b69cc2972fa6e8d01a5eb7430e1d85bcfeffdeaf4 \
--control-plane --certificate-key 1ca48e06dcd7ee03b6fb0fa6593c8bfcfeecd1103072d0a41e1eb9539dbf1a5a \
--apiserver-advertise-address=192.168.100.12 --v=5
# kubeadm join 192.168.100.10:7443 --token osvhga.a26utx39g1y3dqew \
--discovery-token-ca-cert-hash sha256:d106ad18f72f1c091448c06b69cc2972fa6e8d01a5eb7430e1d85bcfeffdeaf4 \
--control-plane --certificate-key 1ca48e06dcd7ee03b6fb0fa6593c8bfcfeecd1103072d0a41e1eb9539dbf1a5a \
--apiserver-advertise-address=192.168.100.13 --v=5
4.4 Kubernetes network plugins(CNI) install
- flannel (https://github.com/coreos/flannel) install (v0.12.0)
# kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/v0.12.0/Documentation/kube-flannel.yml
- Weave-net Install (https://www.weave.works/docs/net/latest/overview/)
# kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d '\n')"
- CNI Comparision (https://kubedex.com/kubernetes-network-plugins)
If you’re using a cloud service like GKE, EKS or AKS you are covered already and should simply use their Calico integration for network policy.
To summarise my recommendation is Calico 9 times out of 10. It’s simple, fast, has lots of features and a good company behind it.어 있는 경우
4.5 K8s Worker nodes
- Worker node가 분리되어 있는 경우
# kubeadm join 14.52.244.136:7443 --token tqte4l.m15k0fwtvqp3aa1p \
--discovery-token-ca-cert-hash sha256:580dee2eaa1bd376ae8344a4e69d06553e3901be0bbaec76f14bdc8cded4aa3d
- Master node를 Worker node로 사용할 경우
# kubectl taint nodes --all node-role.kubernetes.io/master-
or
# kubectl taint nodes --all node-role.kubernetes.io=master:NoSchedule-
node/gmd01 untainted
...
# kubectl describe node -A | grep Taints
Taints: <none>
...
#
4.6 Setup for K8s users
# yum install -y bash-completion
$ mkdir -p $HOME/.kube
$ sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
$ sudo chown $(id -u):$(id -g) $HOME/.kube/config
$ echo -e "\n# Setup autocomplete in bash into the current shell." >> .bash_profile
$ echo "source <(kubectl completion bash)" >> .bash_profile
$ echo "alias k=kubectl" >> ~/.bash_profile
$ echo "complete -F __start_kubectl k" >> .bash_profile
$ source ~/.bash_profile
$ k version --short=true
Client Version: v1.16.15
Server Version: v1.16.15
$
5. K8s Install log
'Kubernetes > Install' 카테고리의 다른 글
| MetalLB (0) | 2021.09.15 |
|---|---|
| keepalived, haproxy for K8s (0) | 2021.09.15 |
| K8s 구성 - MiniKube on MacOS (0) | 2021.09.14 |
| K8s 구성 - KinD on MacOS (0) | 2021.09.14 |
| K8s 구성 - Single on GCE (0) | 2021.09.14 |
댓글