2021.05.03
1. 환경
- Kubeflow 1.2.0
- Kubernetes 1.16.15
- CentOS Linux release 7.8
- Dex (https://github.com/dexidp/dex)
Dex is an identity service that uses OpenID Connect to drive authentication for other apps.
- Kubernetes compatibility
2. Kubeflow Install
- https://v1-2-branch.kubeflow.org/docs/started/k8s/kfctl-istio-dex/
a. Notes on the configuration file
- Istio configuration for trustworthy JWTs
This configuration uses Istio version 1.3.1 with SDS enabled, which requires Kubernetes 1.13 or later.
SDS stands for Secret Delivery Service and allows Istio components to receive SSL Certificates by the API.
Master node 마다 kube-apiserver.yaml 파일을 수정해야 하며. 수정 후 kube-apiserver-iap0[1-3] POD가 자동으로 재기동 됨
$ sudo vi /etc/kubernetes/manifests/kube-apiserver.yaml
…
- --service-account-key-file=/etc/kubernetes/pki/sa.pub
- --service-account-issuer=kubernetes.default.svc # appended
- --service-account-signing-key-file=/etc/kubernetes/pki/sa.key # appended
…
$- Disabling istio installation If your Kubernetes cluster has an existing Istio installation
✓ If your Kubernetes cluster has an existing Istio installation you may choose to not install Istio by removing the applications istio-crds and istio-install in the configuration file kfctl_istio_dex.v1.2.0.yaml.
✓ istio compatible matrix
istio 1.8 : a compatible version of Kubernetes (1.16, 1.17, 1.18, 1.19)
istio 1.9 : a compatible version of Kubernetes (1.17, 1.18, 1.19, 1.20)
Kubeflow 1.2 uses Istio 1.3.1 and is compatible up to 1.5.x.
- Default password in static file configuration for Dex
The configuration file kfctl_istio_dex.v1.2.0.yaml contains a default staticPasswords user
user with email set to admin@kubeflow.org and password 12341234.
b. Default StorageClass for on-premises deployments
- This Kubeflow deployment requires a default StorageClass with a dynamic volume provisioner.
$ k get storageclasses.storage.k8s.io --all-namespaces
NAME PROVISIONER AGE
rook-ceph-block-sc-iap rook-ceph.rbd.csi.ceph.com 270d
rook-ceph-fs-sc-iap rook-ceph.cephfs.csi.ceph.com 270d
rook-ceph-object-bucket-iap ceph.rook.io/bucket 270d
nfs-sc-iap nfs-provisioner 178d
$ kubectl patch storageclass nfs-sc-iap -p '{"metadata": {"annotations": {"storageclass.kubernetes.io/is-default-class":"true"}}}'
storageclass.storage.k8s.io/nfs-sc-iap patched
$ k get storageclasses.storage.k8s.io --all-namespaces
NAME PROVISIONER AGE
rook-ceph-block-sc-iap rook-ceph.rbd.csi.ceph.com 270d
rook-ceph-fs-sc-iap rook-ceph.cephfs.csi.ceph.com 270d
rook-ceph-object-bucket-iap ceph.rook.io/bucket 270d
nfs-sc-iap (default) nfs-provisioner 178d
$
c. Prepare your environment
$ mkdir ~/bin && cd ~/bin
$ wget https://github.com/kubeflow/kfctl/releases/download/v1.2.0/kfctl_v1.2.0-0-gbc038f9_linux.tar.gz
$ tar -xf kfctl_v1.2.0-0-ga476281_linux.tar.gz
$ cd ..- The value of KF_NAME must consist of lower case alphanumeric characters or ‘-', and must start and end with an alphanumeric character.
$ export CONFIG_URI="https://raw.githubusercontent.com/kubeflow/manifests/v1.2-branch/kfdef/kfctl_istio_dex.v1.2.0.yaml"
$ export KF_NAME=acp-kubeflow
$ export BASE_DIR=/home/iap/k8s-oss/kf_deployments
$ export KF_DIR=${BASE_DIR}/${KF_NAME}
d. set up and deploy Kubeflow
$ mkdir -p ${KF_DIR} && cd ${KF_DIR}
$ kfctl build -V -f ${CONFIG_URI}
INFO[0000] Downloading https://raw.githubusercontent.com/kubeflow/manifests/v1.2-branch/kfdef/kfctl_istio_dex.v1.2.0.yaml to /tmp/374689392/tmp.yaml filename="utils/k8utils.go:178"
...
INFO[0018] Creating folder /home/iap/k8s-oss/kf_deployments/acp-kubeflow/kustomize/kfserving filename="kustomize/kustomize.go:667"
$
$ export CONFIG_FILE=${KF_DIR}/kfctl_istio_dex.v1.2.0.yaml
$ kfctl apply -V -f ${CONFIG_FILE}
...
$ k get ns | egrep 'NAME|auth|cert-manager|istio-system|kubeflow'
NAME STATUS AGE
auth Active 15h
cert-manager Active 15h
istio-system Active 15h
kubeflow Active 15h
$
e. Add static users for basic auth
- Dex supports several authentication methods:
✓ Log in as a static user
The credentials are the ones you specified in the KfDef file, or the default (Email: admin@kubeflow.org, Password:12341234).
It is highly recommended to change the default credentials.
✓ Add static users for basic auth
https://v1-0-branch.kubeflow.org/docs/started/k8s/kfctl-istio-dex/#add-static-users-for-basic-auth
▷ hash: The password must be hashed with bcrypt with an at least 10 difficulty level.
You can use an online tool like: https://passwordhashing.com/BCrypt
▷ userID: UUID 같은 고유의 값을 설정, Online UUID Generator(https://www.uuidgenerator.net/)
$ cd k8s-oss/kf_deployments/acp-kubeflow
$ kubectl get configmap dex -n auth -o jsonpath='{.data.config\.yaml}' > dex-config.yaml
## Hash tool (for passowrd): https://passwordhashing.com/BCrypt
## UUID Version 4 tool (for userID): https://www.uuidgenerator.net/
## username is comment
$ vi dex-config.yaml
...
staticPasswords:
- email: yoosung.jeon@kt.com
hash: $2b$10$wi7rg0ZVWynmnVMqoOuZde0DlWJc6.VNVyzf/k.RF7J3Z/AiEhLnK
userID: 08a8684b-db88-4b73-90a9-3cd1661f5466
username: Yoosung Jeon
...
$ kubectl create configmap dex --from-file=config.yaml=dex-config.yaml -n auth --dry-run -o yaml | kubectl apply -f -
$ kubectl rollout restart deployment dex -n auth✓ LDAP / Active Directory
✓ External Identity Provider (IdP) (for example Google, LinkedIn, GitHub, …)
f. Accessing Kubeflow
- Access with a Nodeport
# URL - http://[K8s 노드의 EXTERNAL_IP]:nodePort/
# Port: k get svc/istio-ingressgateway -n istio-system -o=jsonpath={.spec.ports[1].nodePort}
http://api.acp.kt.co.kr:31380/
- Expose with a LoadBalancer
https://v1-2-branch.kubeflow.org/docs/started/k8s/kfctl-istio-dex/#expose-with-a-loadbalancer
$ kubectl patch service istio-ingressgateway -n istio-system -p '{ "spec": { "type": "LoadBalancer" } }'
service/istio-ingressgateway patched
$ k get svc -n istio-system | egrep 'NAME|istio-ingressgateway'
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
istio-ingressgateway LoadBalancer 10.100.201.168 14.52.244.137 15020:32434/TCP,80:31380/TCP,443:31390/TCP,... 67m
$
- istio-injection 설정 변경
처음 로그인 할 때 로그인 유저를 위한 kubernetes namespace을 생성, 생성된 namespace 는 istio-injection이 enabled 되어 있음
istio를 사용하지 않을 경우는 다음과 같이 disabled 시킬 것 (권고)
$ kubectl label namespace admin istio-injection=disabled --overwrite
3. 참고 사항
- 설치 된 Kubeflow 1.0.2 삭제
✓(주의) kfctl 명령어로 Kubeflow 삭제 할 경우 사용자 관련 Namingspace도 같이 삭제됨
Kubeflow 관련 namespaces: auth, cert-manager, knative-serving
사용자 관련 namespaces: admin, ccp, yoosung-jeon, ...
$ cd /home/iap/kf_deployments/iap-kubeflow
$ kfctl delete -V -f kfctl_istio_dex.v1.2.0.yaml
INFO[0000] .cache/manifests exists; not resyncing filename="kfconfig/types.go:473"
INFO[0000] Processing application: namespaces filename="kustomize/kustomize.go:569"
INFO[0000] folder kustomize/namespaces exists filename="kustomize/kustomize.go:665"
...
$
- 수동 삭제
$ kubectl delete ns admin auth cert-manager istio-system knative-serving kubeflow
$ kubectl get CustomResourceDefinition.apiextensions.k8s.io | grep cert-manager | cut -d' ' -f1 | \
xargs -n 1 kubectl delete CustomResourceDefinition.apiextensions.k8s.io
$ kubectl get CustomResourceDefinition.apiextensions.k8s.io | grep knative.dev | cut -d' ' -f1 | \
xargs -n 1 kubectl delete CustomResourceDefinition.apiextensions.k8s.io
$ kubectl get ValidatingWebhookConfiguration | egrep 'katib|knative'
config.webhook.istio.networking.internal.knative.dev 2021-05-03T10:11:26Z # kubeflow 1.2.0
config.webhook.serving.knative.dev 2020-08-05T08:24:02Z # Kubeflow 1.0.2
validation.webhook.serving.knative.dev 2020-08-05T08:24:02Z # Kubeflow 1.0.2
katib-validating-webhook-config 2020-08-05T08:24:39Z
se]$ kubectl get MutatingWebhookConfiguration | egrep 'katib|knative'
katib-validating-webhook-config 2020-08-05T08:24:39Z
webhook.istio.networking.internal.knative.dev 2021-05-03T10:11:26Z # kubeflow 1.2.0
webhook.serving.knative.dev 2020-08-05T08:24:01Z # Kubeflow 1.0.2
$ kubectl delete ValidatingWebhookConfiguration config.webhook.serving.knative.dev validation.webhook.serving.knative.dev
$ kubectl delete MutatingWebhookConfiguration webhook.serving.knative.dev
$ kubectl api-resources --verbs=list --namespaced -o name | xargs -n 1 kubectl get -n kubeflow 2>&1 | grep -v "No resources"
$- Kubeflow와 별개로 설치된 Istio 1.5.8 삭제
$ k delete ns istio-system
$ istioctl x uninstall --purge # istioctl manifest generate --set profile=demo | kubectl delete -f -
failed to get proxy infos: unable to find any Istiod instances
All Istio resources will be pruned from the cluster
Proceed? (y/N) y
✔ Uninstall complete
$ kubectl get CustomResourceDefinition.apiextensions.k8s.io | grep isio.io | cut -d' ' -f1 | \
xargs -n 1 kubectl delete CustomResourceDefinition.apiextensions.k8s.io
$'Kubeflow > Install' 카테고리의 다른 글
| Kubeflow 1.4.1 in Minikube 구성 (0) | 2021.12.30 |
|---|---|
| Kubeflow 1.2 in Minikube 구성 (0) | 2021.09.24 |
| Kubeflow 1.0 in On-prem 구성 (0) | 2021.09.24 |
| Kubeflow 1.0 using MiniKF 구성 (Windows 10) (0) | 2021.09.24 |
| Kubeflow 1.0 in GCE 구성 (0) | 2021.09.24 |
댓글