jupyter5 Distributed training 사례 #3 (In Jupyter) 2021.06.25 1. Kubeflow Jupyter(GPU 할당) 환경에서 Distributed training (Tensorflow) - Environments ✓ Remote - 개발 환경 Kubeflow Jupyter (GPU) ✓ Remote - 학습 환경 Kubeflow 1.2 (The machine learning toolkit for Kubernetes) / Kubernetes 1.16.15 Harbor 2.2.1 (Private docker registry) Nvidia V100 / Driver 450.80, cuda 11.2, cuDNN 8.1.0 CentOS 7.8 - 관련 기술 - Tensorflow MirroredStrategy (Data parallelism) 2. 사전작업 a.. 2021. 9. 26. Distributed training 사례 #2 (From KF Jupyter, Tensorflow) 2021.06.24 1. Kubeflow Jupyter 환경에서 Distributed training (Tensorflow) - Environments ✓ Remote - 개발 환경 Kubeflow Jupyter ✓ Remote - 학습 환경 Kubeflow 1.2 (The machine learning toolkit for Kubernetes) / Kubernetes 1.16.15 Nexus (Private docker registry) Nvidia V100 / Driver 450.80, cuda 11.2, cuDNN 8.1.0 CentOS 7.8 - Flow (Tensorflow) ✓ Remote (Kubeflow Jupyter) a. Docker Image build b. Docker Image P.. 2021. 9. 26. Kubeflow 1.0 기능 #1 (Jupyter notebook) 2020.03.12 1. 참고 문서 - https://v1-0-branch.kubeflow.org/docs/notebooks/setup/ 2. Notebook server 생성 a. Cloud shell 기동 b. URL 확인 $ kubectl -n istio-system get ingress NAME HOSTS ADDRESS PORTS AGE envoy-ingress my-kubeflow.endpoints.my-kubeflow-269301.cloud.goog 34.107.211.135 80 6m42s$ $ c. Kubeflow 접속 (URL: my-kubeflow.endpoints.my-kubeflow-269301.cloud.goog) d. Create a Jupyter notebook server a.. 2021. 9. 24. Running the MNIST using distributed training 2021.5.28 1. Running the MNIST on-prem Jupyter notebook - The MNIST on-prem notebook builds a Docker image, launches a TFJob to train a model, and creates an InferenceService (KFServing) to deploy the trained model. - https://v1-2-branch.kubeflow.org/docs/started/workstation/minikube-linux/#running-the-mnist-on-prem-jupyter-notebook a. Prerequisites - Step 1: Set up Python environment in MacOS y.. 2021. 9. 24. Kubeflow Jupyter Custom Image 추가 2021.06.29 1. 개요 - kubeflow Notebook Server 생성을 위하여 Custom Image를 추가 - References https://www.kangwoo.kr/tag/jupyter/ https://github.com/kubeflow/kubeflow/tree/v1.2.0/components/tensorflow-notebook-image https://towardsdatascience.com/make-kubeflow-into-your-own-data-science-workspace-cc8162969e29 - Custom Image spec ✓ Base image: tensorflow/tensorflow:2.5.0-gpu-jupyter (cuda 11.2, cuDNN 8.1.0),.. 2021. 9. 24. 이전 1 다음
