본문 바로가기
Kubeflow/Management

Dex 인증 / 우회

by 여행을 떠나자! 2021. 9. 29.

1. 개요

- Dex ? (https://github.com/dexidp/dex)

   Dex is an identity service that uses OpenID Connect to drive authentication for other apps.

- Dex 구성

   'kfctl_istio_dex.v1.2.0.yaml'를 이용하여 Kubeflow(v1.2)를 구성할 경우 dex(v2.22)가 포함되어 설치 된다.

 

- Dex 인증 범위

   다음과 같은 유형의 자원에 호출할 경우 Dex 인증을 요구 한다.

   ✓ Kubeflow dashboard login

   ✓ KFServing (Knative 기반에서 동작)
   ✓ Knative serving (Istio 기반에서 동작)
   ✓ Istio Virtual service 

 

- 적용된 auth filter 

$ k get envoyfilters.networking.istio.io authn-filter -n istio-system
NAME           AGE
authn-filter   135d
$

 

 

2. Dex 인증 요구

- KFServing, Knative serving, Istio Virtual service 등를 호출할 경우 '/dex/auth'로 리다이텍트 되면서 인증을 요구 한다.

   ✓ 요청할 URL에 대하여 Dex 인증을 우회하거나,

   ✓ 사전에 인증 과정을 거쳐서 authservice_session 값을 얻은 후, 요청시 Token을 전달하면 된다.

- KFServing 호출

$ curl -v http://flowers-sample.yoosung-jeon.kf-serv.acp.kt.co.kr:80/v1/models/flowers-sample:predict -d @./input.json
* About to connect() to flowers-sample.yoosung-jeon.kf-serv.acp.kt.co.kr port 80 (#0)
*   Trying 14.52.244.137...
* Connected to flowers-sample.yoosung-jeon.kf-serv.acp.kt.co.kr (14.52.244.137) port 80 (#0)
> POST /v1/models/flowers-sample:predict HTTP/1.1
> User-Agent: curl/7.29.0
> Host: flowers-sample.yoosung-jeon.kf-serv.acp.kt.co.kr
> Accept: */*
> Content-Length: 16201
> Content-Type: application/x-www-form-urlencoded
> Expect: 100-continue
>
< HTTP/1.1 100 Continue
< HTTP/1.1 302 Found
< location: /dex/auth?client_id=kubeflow-oidc-authservice&redirect_uri=%2Flogin%2Foidc&response_type=code&scope=profile+email+groups+openid&state=MTYzMjg5NjI1NHxFd3dBRUV0eFFYUjRhbEF5V20xRU1XcDBkRE09fEz3NQUn3zDR9uI_xrfHiAyt6JuqvrCR8bp4hIzFzdX9
< date: Wed, 29 Sep 2021 06:17:33 GMT
< content-length: 0
< x-envoy-upstream-service-time: 3
< server: istio-envoy
* HTTP error before end of send, stop sending
<
* Closing connection 0
$

 

 

3. Dex 인증 우회

- Dex 인증을 제외할 URL(flowers-sample.yoosung-jeon.kf-serv.acp.kt.co.kr:80)를 아래와 같이 EnvoyFilter에 적용한다.

   spec.configPatches[].match.routeConfiguration.vhost.name: flowers-sample.yoosung-jeon.kf-serv.acp.kt.co.kr:80

   spec.configPatches[].patch.value.per_filter_config.envoy.ext_authz.disabled: true

- 추가할 EnvoyFilter는 istio-system namespace로 반듯이 지정해야 한다.

- https://stackoverflow.com/questions/62270590/apply-envoyfilter-ext-authz-per-virtual-hosts

$ cat bypass-auth-flowers-sample.yaml
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: bypass-auth-flowers-sample
  namespace: istio-system
spec:
  workloadSelector:
    labels:
      istio: ingressgateway
  configPatches:
  - applyTo: VIRTUAL_HOST
    match:
      routeConfiguration:
        vhost:
          name: flowers-sample.yoosung-jeon.kf-serv.acp.kt.co.kr:80
    patch:
        operation: MERGE
        value:
          per_filter_config:
            envoy.ext_authz:
              disabled: true
$ k apply -f bypass-auth-flowers-sample.yaml
envoyfilter.networking.istio.io/bypass-auth-flowers-sample created
[iap@iap01 tensorflow]$ k get envoyfilters.networking.istio.io -n istio-system
NAME                         AGE
authn-filter                 135d
bypass-auth-filter           49d
bypass-auth-flowers-sample   9s
$

 

- KFServing 호출

$ curl -v http://flowers-sample.yoosung-jeon.kf-serv.acp.kt.co.kr:80/v1/models/flowers-sample:predict -d @./input.json
* About to connect() to flowers-sample.yoosung-jeon.kf-serv.acp.kt.co.kr port 80 (#0)
*   Trying 14.52.244.137...
* Connected to flowers-sample.yoosung-jeon.kf-serv.acp.kt.co.kr (14.52.244.137) port 80 (#0)
> POST /v1/models/flowers-sample:predict HTTP/1.1
> User-Agent: curl/7.29.0
> Host: flowers-sample.yoosung-jeon.kf-serv.acp.kt.co.kr
> Accept: */*
> Content-Length: 16201
> Content-Type: application/x-www-form-urlencoded
> Expect: 100-continue
>
< HTTP/1.1 100 Continue
< HTTP/1.1 200 OK
< content-length: 221
< content-type: application/json
< date: Wed, 29 Sep 2021 06:31:13 GMT
< x-envoy-upstream-service-time: 2018
< server: istio-envoy
<
{
    "predictions": [
        {
            "prediction": 0,
            "key": "   1",
            "scores": [0.999114931, 9.2098875e-05, 0.000136786344, 0.000337258534, 0.000300534302, 1.84814126e-05]
        }
    ]
* Connection #0 to host flowers-sample.yoosung-jeon.kf-serv.acp.kt.co.kr left intact
}
$

 

- Trouble-shooting 

   ▷ Problem: curl 명령어로 요청시 응답지연 또는 타임아웃 에러가 발생

                       외부 네트워크에서 요청 시에만 발생되며, 외부 네트워크에서 브라우저로 요청시 정상 동작 됨 

yoosungjeon@ysjeon-Dev ~ % curl -v http://flowers-sample.yoosung-jeon.kf-serv.acp.kt.co.kr:80/v1/models/flowers-sample:predict -d @./input.json
*   Trying 14.52.244.137:80...
* Connected to flowers-sample.yoosung-jeon.kf-serv.acp.kt.co.kr (14.52.244.137) port 80 (#0)
> POST /v1/models/flowers-sample:predict HTTP/1.1
> Host: flowers-sample.yoosung-jeon.kf-serv.acp.kt.co.kr
> User-Agent: curl/7.71.1
> Accept: */*
> Content-Length: 16192
> Content-Type: application/x-www-form-urlencoded
>
* upload completely sent off: 16192 out of 16192 bytes
* Recv failure: Operation timed out
* Closing connection 0
curl: (56) Recv failure: Operation timed out
yoosungjeon@ysjeon-Dev ~ %

   ▷ Solution: curl  명령어로 요청시 User-Agent를 명시적으로 "curl/7.71.1" 대신 null 또는 Chrome으로 지정

       curl -v http://{URL} -d @./input.json -A ""

       curl -v http://{URL} -d @./input.json -A "Mozilla/5.0 (Macintosh; Intel Mac OS X 11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.61 Safari/537.36"

       본 현상은 외부 네트워크에서 요청할 경우만 발생되며 Kubernetes, Kubeflow, Dex 설정과는 무관함

       테스트 서버 앞단에 있는 IPS (침입방지시스템)와 관련이 있을 것으로 예상 됨

yoosungjeon@ysjeon-Dev ~ % curl -v http://flowers-sample.yoosung-jeon.kf-serv.acp.kt.co.kr:80/v1/models/flowers-sample:predict -d @./input.json -A ""
*   Trying 14.52.244.137:80...
* Connected to flowers-sample.yoosung-jeon.kf-serv.acp.kt.co.kr (14.52.244.137) port 80 (#0)
> POST /v1/models/flowers-sample:predict HTTP/1.1
> Host: flowers-sample.yoosung-jeon.kf-serv.acp.kt.co.kr
> Accept: */*
> Content-Length: 16192
> Content-Type: application/x-www-form-urlencoded
>
* upload completely sent off: 16192 out of 16192 bytes
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< content-length: 221
< content-type: application/json
< date: Wed, 29 Sep 2021 06:41:42 GMT
< x-envoy-upstream-service-time: 1957
< server: istio-envoy
<
{
    "predictions": [
        {
            "prediction": 0,
            "key": "   1",
            "scores": [0.999114931, 9.2098875e-05, 0.000136786344, 0.000337258534, 0.000300534302, 1.84814126e-05]
        }
    ]
* Connection #0 to host flowers-sample.yoosung-jeon.kf-serv.acp.kt.co.kr left intact
}%
yoosungjeon@ysjeon-Dev ~ %

 

 

4. ID token 발급 using kfp

- kfp is KubeFlow Pipelines SDK

- kfp package를 이용하여 아래와 같이 token을 얻는 다.

   ✓ 발급 받은 token은 24시간 유효하다.

yoosungjeon@ysjeon-Dev ~ % pip install kfp
Looking in indexes: http://mirror.kakao.com/pypi/simple
Collecting kfp
...
Successfully installed Deprecated-1.2.13 absl-py-0.11.0 docstring-parser-0.10 fire-0.4.0 google-api-python-client-1.12.8 kfp-1.8.3 kfp-pipeline-spec-0.1.11 kfp-server-api-1.7.0 pydantic-1.8.2 requests-toolbelt-0.9.1 strip-hints-0.1.10 tabulate-0.8.9 typing-extensions-3.10.0.2
yoosungjeon@ysjeon-Dev ~ % vi get_token.py
import requests
import kfp

HOST = "http://kf.acp.kt.co.kr/"
USERNAME = "admin@kubeflow.org"
PASSWORD = "12341234"

session = requests.Session()
response = session.get(HOST)

headers = {
    "Content-Type": "application/x-www-form-urlencoded",
}

data = {"login": USERNAME, "password": PASSWORD}
session.post(response.url, headers=headers, data=data)
session_cookie = session.cookies.get_dict()["authservice_session"]

print(session_cookie)
yoosungjeon@ysjeon-Dev ~ % python get_token.py
MTYzMjk2NTA5NHxOd3dBTkVoTU1rcFhVVTlRUmt4Q1dqYzFWMWxQTWtkWlZGQk1XRkZQTlVsSFVWUkdSak5WVEU1SFdsQkJRMWRPV2tJMU5saElVVUU9fOFpbeudhGgoPGfXouHc-s3rcwVjAEKS7hmouSmjsh4o
yoosungjeon@ysjeon-Dev ~ %

 

- Dex log

$ k logs dex-7b5b7f4fb6-qq8qp -n auth
...
time="2021-09-30T01:19:48Z" level=info msg="login successful: connector \"local\", username=\"admin\", preferred_username=\"\", email=\"admin@kubeflow.org\", groups=[]"

  

- KFserving 호출하기

   해더에 token을 추가 후 호출 한다.

      -H "Cookie: authservice_session=${TOKEN}"


$ TOKEN=MTYzMjk2NTA5NHxOd3dBTkVoTU1rcFhVVTlRUmt4Q1dqYzFWMWxQTWtkWlZGQk1XRkZQTlVsSFVWUkdSak5WVEU1SFdsQkJRMWRPV2tJMU5saElVVUU9fOFpbeudhGgoPGfXouHc
$ curl -v http://flowers-sample.yoosung-jeon.kf-serv.acp.kt.co.kr:80/v1/models/flowers-sample:predict -d @./input.json -H "Cookie: authservice_session=${TOKEN}"
* About to connect() to flowers-sample.yoosung-jeon.kf-serv.acp.kt.co.kr port 80 (#0)
*   Trying 14.52.244.137...
* Connected to flowers-sample.yoosung-jeon.kf-serv.acp.kt.co.kr (14.52.244.137) port 80 (#0)
> POST /v1/models/flowers-sample:predict HTTP/1.1
> User-Agent: curl/7.29.0
> Host: flowers-sample.yoosung-jeon.kf-serv.acp.kt.co.kr
> Accept: */*
> Cookie: authservice_session=MTYzMjk2NTA5NHxOd3dBTkVoTU1rcFhVVTlRUmt4Q1dqYzFWMWxQTWtkWlZGQk1XRkZQTlVsSFVWUkdSak5WVEU1SFdsQkJRMWRPV2tJMU5saElVVUU9fOFpbeudhGgoPGfXouHc
> Content-Length: 16201
> Content-Type: application/x-www-form-urlencoded
> Expect: 100-continue
>
< HTTP/1.1 100 Continue
< HTTP/1.1 200 OK
< content-length: 221
< content-type: application/json
< date: Wed, 29 Sep 2021 09:16:55 GMT
< x-envoy-upstream-service-time: 2034
< server: istio-envoy
<
{
    "predictions": [
        {
            "scores": [0.999114931, 9.2098875e-05, 0.000136786344, 0.000337258534, 0.000300534302, 1.84814126e-05],
            "prediction": 0,
            "key": "   1"
        }
    ]
* Connection #0 to host flowers-sample.yoosung-jeon.kf-serv.acp.kt.co.kr left intact
$

 

 

5. ID token 발급 using rest api

- ID token 발급 빋기

## istio 주소 설정
$ k get svc -n istio-system istio-ingressgateway
NAME                   TYPE           CLUSTER-IP      EXTERNAL-IP     PORT(S)                                                                                                                                      AGE
istio-ingressgateway   LoadBalancer   10.110.60.217   14.52.244.137   15020:30943/TCP,80:31380/TCP,443:31390/TCP,31400:31400/TCP,15029:31449/TCP,15030:31395/TCP,15031:31117/TCP,15032:30569/TCP,15443:31974/TCP   135d
$ ISTIO_URL=14.52.244.137
$

## 임의의 URI 호출
$ curl http://${ISTIO_URL}/v1/models/flowers-sample:predict
<a href="/dex/auth?client_id=kubeflow-oidc-authservice&amp;redirect_uri=%2Flogin%2Foidc&amp;response_type=code&amp;scope=profile+email+groups+openid&amp;state=MTYzMjkwNTYyOHxFd3dBRUhKUmJqUkhjV0pCVEVGNFUwSllkMjA9fJKaeQAdBimGkiqF1Bw6UV6Q0dQL2WlXBLdwZoGbMHCC">Found</a>.
$

## Req 얻기
$ curl "http://${ISTIO_URL}/dex/auth?client_id=kubeflow-oidc-authservice&amp;redirect_uri=%2Flogin%2Foidc&amp;response_type=code&amp;scope=profile+email+groups+openid&amp;state=MTYzMjkwNTYyOHxFd3dBRUhKUmJqUkhjV0pCVEVGNFUwSllkMjA9fJKaeQAdBimGkiqF1Bw6UV6Q0dQL2WlXBLdwZoGbMHCC"
<a href="/dex/auth/local?req=fmpegyloe6jeh6gjfxlbarwnd">Found</a>.

$ REQ=fmpegyloe6jeh6gjfxlbarwnd

## 인증하기
$ LOGIN=admin%40kubeflow.org
$ PASSWORD=12341234
$ curl "http://${ISTIO_URL}/dex/auth/local?req=${REQ}" -H 'Content-Type: application/x-www-form-urlencoded' --data "login=${LOGIN}&password=${PASSWORD}"
$

## approval 얻기
$ curl "http://${ISTIO_URL}/dex/approval?req=${REQ}"
<a href="/login/oidc?code=rsytn2zptxhmq56gcj65wz2co&amp;state=MTYzMjkwNTYyOHxFd3dBRUhKUmJqUkhjV0pCVEVGNFUwSllkMjA9fJKaeQAdBimGkiqF1Bw6UV6Q0dQL2WlXBLdwZoGbMHCC">See Other</a>.

## ID token 얻기
$ curl -v "http://${ISTIO_URL}/login/oidc?code=j4j2lheobduu4i2xe7r446pzf&amp;state=MTYzMjkwNjE3NHxFd3dBRURsbVNHcDRSMVY2VW0xRE9UaG5RWG89fDRNILnTBS2qLrQdhh_qT3v6Ol-Yyjr57ouquKg-sCX8" | grep set-cookie
* About to connect() to 14.52.244.137 port 80 (#0)
*   Trying 14.52.244.137...
* Connected to 14.52.244.137 (14.52.244.137) port 80 (#0)
> GET /login/oidc?code=j4j2lheobduu4i2xe7r446pzf&amp;state=MTYzMjkwNjE3NHxFd3dBRURsbVNHcDRSMVY2VW0xRE9UaG5RWG89fDRNILnTBS2qLrQdhh_qT3v6Ol-Yyjr57ouquKg-sCX8 HTTP/1.1
> User-Agent: curl/7.29.0
> Host: 14.52.244.137
> Accept: */*
>
< HTTP/1.1 302 Found
< content-type: text/html; charset=utf-8
< location: /hello
< set-cookie: authservice_session=MTYzMjkwNjIzN3xOd3dBTkZoQlUwMVRRa3hTVGt0WVRscFFNa1ZHUTB4RlFrOVRWMDlPV1ZaS1IxcEhTRU5UUjFaWFVrTlJRazVPVVVOQlNqSkZRMEU9fM9NitxyMMnG6ye3CSjyQr4J4ntKc_RnJW4DqWQ181QV; Path=/; Expires=Thu, 30 Sep 2021 09:03:57 GMT; Max-Age=86400
< date: Wed, 29 Sep 2021 09:03:57 GMT
< content-length: 29
< x-envoy-upstream-service-time: 27
< server: istio-envoy
<
<a href="/hello">Found</a>.

* Connection #0 to host 14.52.244.137 left intact
$

   ✓ 위 과정을 거쳐 최종적으로 ID token(authservice_session)을 얻을 수 있다.

        set-cookie:

            authservice_session=MTYzMjkwNjIzN3xOd3dBTkZoQlUw...;

            Path=/;

            Expires=Thu, 30 Sep 2021 09:03:57 GMT;

            Max-Age=86400

   ✓ ID Token은 한번만 조회가 가능한다. 두번째 호출 부터는 "504 Gateway Timeout" 에러가 발생된다.

 

 

6. Dex 사용자 추가 

- Add static users for basic auth

   Kubeflow 1.2 in On-prem 구성 : "2. Kubeflow Install / e. Add static users for basic auth"

- LDAP / Active Directory 

   Kubeflow 1.0 in On-prem 구성 : "2. Kubeflow Install / g. Log in with LDAP / Active Directory"

'Kubeflow > Management' 카테고리의 다른 글

KFServing - Canary rollout 테스트  (0) 2021.10.17
KFServing - Deep dive  (0) 2021.10.14
Kubeflow Jupyter Custom Image 추가  (0) 2021.09.24

관련글

댓글

티스토리툴바