전체 글109 Kubeflow 1.0 기능 #5 (KFServing, TFServing) 2020.03.13 1. Model serving overview - https://v1-0-branch.kubeflow.org/docs/reference/pytorchjob/v1/pytorch/ - Kubeflow supports two model serving systems that allow multi-framework model serving: KFServing and Seldon Core. Alternatively, you can use a standalone model serving system. a. Multi-framework model serving - A check mark (✓) indicates that the system (KFServing or Seldon Core) suppor.. 2021. 9. 25. Kubeflow 1.0 기능 #4 (PyTorch Training) 2020.03.09 1. PyTorchJob ? - Kubeflow에서 PyTorch training할 때 사용되는 Kubernetes custom resource - https://v1-0-branch.kubeflow.org/docs/reference/pytorchjob/v1/pytorch/ 2. PyTorch training 하기 - https://v1-0-branch.kubeflow.org/docs/components/training/pytorch/components/training/pytorch/ a. Cloud shell 기동 b. Verify that PyTorch support is included in your Kubeflow deployment $ kubectl get crd | head.. 2021. 9. 25. Kubeflow 1.0 기능 #3 (Katib) 2020.03.09 1. Kubeflow Katib ? - Katib uses for automated tuning of ML model’s hyperparameters. Hyperparameters are the variables that control the model training process. For example: ✓ Learning rate. ✓ Number of layers in a neural network. ✓ Number of nodes in each layer. Hyperparameter values are not learned. Hyperparameter tuning is the process of optimizing the hyperparameter values to maxim.. 2021. 9. 25. Kubeflow 1.0 기능 #2 (TF-Job, TF-Serving, Kubeflow pipeline) 2020.02.26 1. 개요 - GKE에 설치한 Kubeflow의 Pipeline 기능을 이해하기 위해 아래 사이트를 참조하여 사용 해 봄 - Using Kubeflow for Financial Time Series (https://github.com/kubeflow/examples/tree/master/financial_time_series) - This example covers the following concepts: a. Deploying Kubeflow to a GKE cluster b. Exploration via JupyterHub (prospect data, preprocess data, develop ML model) c. Training several tensorflow models.. 2021. 9. 24. Kubeflow 1.0 기능 #1 (Jupyter notebook) 2020.03.12 1. 참고 문서 - https://v1-0-branch.kubeflow.org/docs/notebooks/setup/ 2. Notebook server 생성 a. Cloud shell 기동 b. URL 확인 $ kubectl -n istio-system get ingress NAME HOSTS ADDRESS PORTS AGE envoy-ingress my-kubeflow.endpoints.my-kubeflow-269301.cloud.goog 34.107.211.135 80 6m42s$ $ c. Kubeflow 접속 (URL: my-kubeflow.endpoints.my-kubeflow-269301.cloud.goog) d. Create a Jupyter notebook server a.. 2021. 9. 24. Running the MNIST using distributed training 2021.5.28 1. Running the MNIST on-prem Jupyter notebook - The MNIST on-prem notebook builds a Docker image, launches a TFJob to train a model, and creates an InferenceService (KFServing) to deploy the trained model. - https://v1-2-branch.kubeflow.org/docs/started/workstation/minikube-linux/#running-the-mnist-on-prem-jupyter-notebook a. Prerequisites - Step 1: Set up Python environment in MacOS y.. 2021. 9. 24. Kubeflow Jupyter Custom Image 추가 2021.06.29 1. 개요 - kubeflow Notebook Server 생성을 위하여 Custom Image를 추가 - References https://www.kangwoo.kr/tag/jupyter/ https://github.com/kubeflow/kubeflow/tree/v1.2.0/components/tensorflow-notebook-image https://towardsdatascience.com/make-kubeflow-into-your-own-data-science-workspace-cc8162969e29 - Custom Image spec ✓ Base image: tensorflow/tensorflow:2.5.0-gpu-jupyter (cuda 11.2, cuDNN 8.1.0),.. 2021. 9. 24. Kubeflow 1.2 in On-prem 구성 2021.05.03 1. 환경 - Kubeflow 1.2.0 - Kubernetes 1.16.15 - CentOS Linux release 7.8 - Dex (https://github.com/dexidp/dex) Dex is an identity service that uses OpenID Connect to drive authentication for other apps. - Kubernetes compatibility 2. Kubeflow Install - https://v1-2-branch.kubeflow.org/docs/started/k8s/kfctl-istio-dex/ a. Notes on the configuration file - Istio configuration for trustwort.. 2021. 9. 24. Kubeflow 1.2 in Minikube 구성 2021.04.27 1. 개요 - Minikube 기반 하에 kubeflow 구성 - https://v1-2-branch.kubeflow.org/docs/started/workstation/minikube-linux/ 문서에서 설명한 옵션을 사용할 경우 Kubeflow dashboard 접근시 오류 발생 (service-account-signing-key-file 값, Troubleshooting 참조) 2. Envrionments - minikube 1.17.1 - kubernetes 1.16.15 - kubeflow 1.2 - macOS 11.2 3. Install Kubeflow a. Prerequisites - Recommended resources 8 cores, 16GB RAM, 250GB s.. 2021. 9. 24. Kubeflow 1.0 in On-prem 구성 2020.06.08 1. 환경 - Kubeflow 1.0.2 - K8s Server Version: v1.15.12 - CentOS Linux release 7.8 - Dex (https://github.com/dexidp/dex) Dex is an identity service that uses OpenID Connect to drive authentication for other apps. 2. Kubeflow Install - https://www.kubeflow.org/docs/started/k8s/kfctl-istio-dex/ a. Notes on the configuration file a-1. Disabling istio installation If your Kubernetes cluster.. 2021. 9. 24. Kubeflow 1.0 using MiniKF 구성 (Windows 10) 2020.02.13 1. What is MiniKF ? - A fast and easy way to deploy Kubeflow on your laptop. - MiniKF = MiniKube + Kubeflow + Arrikto’s Rok Data Management Platform https://www.kubeflow.org/docs/other-guides/virtual-dev/getting-started-minikf/ https://www.arrikto.com/minikf/ https://www.cncf.io/wp-content/uploads/2019/10/Arrikto-Webinar-Scalable-ML-Workflows-with-Advanced-Data-Management-on-Kubeflow... 2021. 9. 24. Kubeflow 1.0 in GCE 구성 2020.02.20 1. 개요 - 본 문서에서는 GCP(Google Cloud Platform)에서 Kubernetes 기반의 End 2 End ML Platform인 Kubeflow를 구성하는 절차를 설명하고자 함 - Ref. Page: https://www.kubeflow.org/docs/gke/deploy/deploy-cli/ 2. What is Kubeflow ? - Kubeflow is the ML toolkit for Kubernetes. The following diagram shows Kubeflow as a platform for arranging the components of your ML system on top of Kubernetes. - https://www.kubeflow.o.. 2021. 9. 24. Istio - Virtual service config 2020.04.27 1. 참고 사이트 - Istio Traffic management (https://bcho.tistory.com/1367) - https://istio.io/docs/ops/best-practices/traffic-management/ - https://istio.io/docs/ops/best-practices/traffic-management/#split-virtual-services 2. 등록 고려 대상 - 대상 ✓ kubernetes-dashboard ✓ weave scope ✓ istio tool : kiali, jaeger, prometheus, grafana - 관련 서비스 $ k get svc grafana jaeger-query kiali prometheus -n ist.. 2021. 9. 23. Istio 1.5 구성 2020.04.24 1. 개요 - Environments: Google Compute Engine, CentOS 7.7, Kubernetes 1.15 - Istio 1.5 has been tested with these Kubernetes releases: 1.14, 1.15, 1.16. - The Helm installation approach will be deprecated in the future. - 참고 사이트 https://istio.io/docs/setup/getting-started/ https://istio.io/docs/setup/install/istioctl/ #1 - 마이크로 서비스와 서비스 매쉬 (https://bcho.tistory.com/1293?category=731548).. 2021. 9. 23. Cert-manager with LetsEncrypt (HTTP challenge) 2021.03.23, 2020.07.13 Cert-manager with LetsEncrypt (DNS challenge): https://1week.tistory.com/2 Cert-manager with LetsEncrypt (HTTP challenge) 1. 개요 - Cert-manager is a native Kubernetes certificate management controller. It can help with issuing certificates from a variety of sources, such as Let’s Encrypt, HashiCorp Vault, Venafi, a simple signing key pair, or self signed. Cert-manager can b.. 2021. 9. 23. 이전 1 2 3 4 5 6 7 8 다음
