2021.03.24
1. CI (Continuous Integration)
- In software engineering, continuous integration (CI) is the practice of merging all developers' working copies to a shared mainline several times a day
- Each integration is verified by an automated build and automated unit testing.
2. Create and configure a project in Gitlab
a. Create Users & Groups
- Login
URL: http://gitlab.14.52.244.138.sslip.io/
Username: root
- User 생성 (Admin Area > Overview > Users > New user)
Name: Architecture Governance Project
Username: agp
ysjeon71@gmail.com: ysjeon71@gmail.com
- User 암호 설정 (Admin Area > Overview > Users > 2FA Disabled > Edit (“Architecture Governance Project”) > Password)
- Group 생성 (+ > New group)
Group name: agp-grp
Visibility level: Private
- Group Members 변경 (Groups > Your groups > agp-grp > Members > Invite member)
GitLab member or Email address: “Architecture Governance Project”
Choose a role permission: Developer
b. Create and configure a project
- Project 생성 (+ > New project)
Project Name: agp-app1
Visibility Level: Private
Initialize repository with a README: check
▷ Git 주소
http://gitlab.14.52.244.138.sslip.io/agp/agp-app1
- Project 변경 #1 (Project > Your project > agp-app1 > Settings > CI/CD)
Auto DevOps > Expand
Default to Auto DevOps pipeline: Uncheck
- Project 변경 #2 (Project > Your project > agp-app1 > Settings > Repository)
Protected branches > Expand
Keep stable branches secure, and force developers to use merge requests. What are protected branches?
Allowed to push: Developers + Maintainers (선택)
Unprotect
- 미 설징 시 Error message: (Project owner가 아닌 계정에서 push할 경우 발생)
remote: GitLab: You are not allowed to push code to protected branches on this project.
To http://repo.chelsea.kt.co.kr/agp/agp-app1.git
! [remote rejected] master -> master (pre-receive hook declined)
error: failed to push some refs to 'http://apg@repo.chelsea.kt.co.kr/agp/agp-app1.git'
- Project 변경 #3 (Project > Your project > agp-app1 > Members > Invite group)
Select a group to invite: agp-grp
Max access level: Developer
3. Coding with PyCharm and merge in Gitlab
a. Git-flow ?
- https://blog.naver.com/good_ray/221998498778 ★
- https://product.hubspot.com/blog/git-and-github-tutorial-for-beginners
- https://backlog.com/git-tutorial/kr/intro/intro1_1.html
b. Get from VCS
- Repository URL
URL: http://gitlab.14.52.244.138.sslip.io/agp/agp-app1.git
Directory: /Users/yoosungjeon/PycharmProjects/agp-app1
Username: agp
Password: ****
c. Coding
- Example (https://docs.docker.com/language/python/build-images/)
- Dockerfile과 어플리케이션(ex. app.py) 작성
▷ Flask is a micro web framework written in Python.
- requirements.txt 생성
d. create New Branch
- Git > New branch & name: branch-1
▷ “git push” 하기 이전에 아무 단계에서 브랜치를 생성하면 됨.
e. git push
- Git > Add
- Git > Commit Directory
- Git > Push
f. Merge from GitLab UI
- Repository > Branches > branch-1 > Merge request
- Merge immediately
g. git Pull
- Git > Pull
▷ 소스를 수정할 경우 신규 브랜치를 생성하고 작업 시작
h. Dockerfile Test
yoosungjeon@ysjeon-Dev agp-app1 % pwd
/Users/yoosungjeon/PycharmProjects/agp-app1
yoosungjeon@ysjeon-Dev agp-app1 % docker build -t agp-app1:1.0 .
[+] Building 0.2s (9/9) FINISHED
...
yoosungjeon@ysjeon-Dev agp-app1 % docker images | egrep "REPO|agp-app1"
REPOSITORY TAG IMAGE ID CREATED SIZE
agp-app1 1.0 bd248c15973a 31 seconds ago 128MB
yoosungjeon@ysjeon-Dev agp-app1 % docker run -p 5000:5000 --name agp-app1 agp-app1:1.0
* Environment: production
WARNING: This is a development server. Do not use it in a production deployment.
Use a production WSGI server instead.
* Debug mode: off
* Running on http://0.0.0.0:5000/ (Press CTRL+C to quit)
172.17.0.1 - - [24/Mar/2021 11:39:45] "GET / HTTP/1.1" 200 -
yoosungjeon@ysjeon-Dev ~ % curl http://127.0.0.1:5000/
Hello, Docker !
yoosungjeon@ysjeon-Dev ~ %
4. docker build & push using Jenkins
- 본 예제의 Jenkins pipeline은 Kubernetes에서 동작되도록 작성되었음
a. Log in
- AI Core Platform : http://14.52.244.136:31443/
b. create Credential for SCM, Docker registry
Dashboard > Credentials > System > Global credentials (unrestricted) > Add Credentials
- Credential Info. (SCM, Docker registry)
Username: agp, Password: ****, ID: ACP-gitlab-agp, Description: AI Core Platform (Gitlab) / agp
Username: agp, Password: ****, ID: ACP-Harbor-agp, Description: AI Core Platform (Harbor) / agp
c. create Pipeline
- Pipeline 생성 (GitLab의 Project name과 동일하게 설정하였으며, 해당 Project 내에 Jenkinsfile을 작성하였음)
Dashboard -> 새로운 Item
Enter an Item name: "agp-app1"
Type: Pipeline
- Pipeline 설정
Definition: Pipeline script from SCM
SCM: Git
Repository URL:
http://gitlab.14.52.244.138.sslip.io/agp/agp-app1.git
Credentials: "SW Dev Hub (Gitlab) / 10150529"
d. What is Pipeline ?
- Pipeline provides an extensible set of tools for modeling simple-to-complex delivery pipelines "as code" via the Pipeline DSL(Domain-specific language).
- Both Declarative and Scripted Pipeline are DSLs to describe portions of your software delivery pipeline.
✓ Declarative Pipeline (https://www.jenkins.io/doc/book/pipeline/syntax/#declarative-pipeline)
▷ Declarative Pipeline is a relatively recent addition to Jenkins Pipeline which presents a more simplified and opinionated syntax on top of the Pipeline sub-systems.
✓ Scripted Pipeline (https://www.jenkins.io/doc/book/pipeline/syntax/#scripted-pipeline)
▷ Scripted Pipeline, like Declarative Pipeline, is built on top of the underlying Pipeline sub-system.
▷ Unlike Declarative, Scripted Pipeline is effectively a general-purpose DSL built with Groovy.
▷ Most functionality provided by the Groovy language is made available to users of Scripted Pipeline, which means it can be a very expressive and flexible tool with which one can author continuous delivery pipelines.
- A Pipeline can be created in one of the following ways:
✓ Through Blue Ocean - after setting up a Pipeline project in Blue Ocean, the Blue Ocean UI helps you write your Pipeline’s Jenkinsfile and commit it to source control.
✓ Through the classic UI - you can enter a basic Pipeline directly in Jenkins through the classic UI.
✓ In SCM - you can write a Jenkinsfile manually, which you can commit to your project’s source control repository.
e. Jenkinsfile 작성
- PyCharm (Jenkinsfile 작성 > git add > git commit > git branch > git push) >> Gitlab (Merge request > Merge)
- References site
✓ https://akomljen.com/set-up-a-jenkins-ci-cd-pipeline-with-kubernetes/ ***
✓ https://tech.osci.kr/2019/11/21/86026733/
✓ https://www.howtodo.cloud/devops/docker/2019/05/16/devops-application.html
- Jenkinsfile 예제1
✓ SW Dev Hub의 Jenkins에서 빌드 작업 수행
✓ Declarative Pipeline 형식으로 작성
✓ Pipeline
Git clone ▷ Build docker image ▷ Push docker image ▷ Delete docker image ▷ Static analysis ▷ Git clone Gitops ▷ Kustomize ▷Git Push Gitops
✓ 소스: 부록 A. Jenkinsfile
- Jenkinsfile 예제2
✓ AI Core Platform의 Jenkins에서 빌드 작업 수행, Kubernetes 환경에서 동작
✓ Scripted Pipeline 형식으로 작성
✓ Pipeline
Clone repo ▷ Build docker image ▷ Push docker image ▷ Delete docker image ▷ Deploy to Dev
✓ 소스: 부록 B. Jenkinsfile for Kubernetes
f. Build
- Dashboard -> agp-app1 > Build Now
- Build 결과
- Console Output
- Jenkins POD 정보 (Kubernetes에서 Jenkins 운영시)
$ k get pod -n jenkins | grep jenkins-slave-pod
jenkins-slave-pod-94z65-3pt0l 5/5 Running 0 101s
$
5. Docker registry
- Nexus 사용 시
- Harbor 사용 시
Harbor는 Multi-tenant, Vulnerability Scanning를 지원 함
6. SonarQube
a. Jenkins pipeline에서 정적분석 한 결과 확인
- 정적분석
소프트웨어를 실행하지 않고 도구를 이용해서 소스 코드나 바이너리를 분석해서 잠재적인 결함을 찾는 것
- SonarQube 접속 후 프로젝트 선택
b. 소스코드 품질평가 (KT기준)
- KTSSP (KT Standard Software Process): SW개발 프로젝트에서 수행해야 하는 프로세스/산출물/개발환경을 정의함
- KTSSP 적용 프로젝트: 프로젝트 유형(IT프로젝트) & 금액 (1억원 이상)
- 4대 SW 품질평가 항목
- SonarQube에 소스코드 정적 분석을 위한 KT 룰 셋이 적용 되어 있음
c. SonarQube Project 생성절차
- Create new project
Project key는 GiLap 프로젝트, Jenkins pipeline 명, Docker Image 명, Kubernetes Service/Pod 명과 동일하게 설정
- 토큰 생성하기
임의의 문자열 입력
- 프로젝트 분석 실행하기
- Jenkins Pipeline에 반영
"로컬 컴퓨터에서 SonarQube Scanner 실행하기"에서 제공하는 명령어를 Pipeline에 추가
stage('Static analysis') {
...
steps {
script {
try {
echo "\n### Static analysis"
sh """
sonar-scanner \
-Dsonar.projectKey=agp-app1 \
-Dsonar.sources=. \
-Dsonar.host.url=https://swdevhub.kt.co.kr/sonar \
-Dsonar.login=6e7be06949e6d2b763e060d68d8f22959354b3f3
"""
} catch(e) {
echo "Error: " + e.toString()
throw e
}
}
}
}
부록 A. Jenkinsfile
pipeline {
agent {label 'master'}
environment{
imageHubURL = "nexus.kt.co.kr:10000"
newImage = ""
// Nexus, GitLab Information
scmURL = "swdevhub.kt.co.kr" // GitLab URL
scmCredential = "SDH-gitlab-10150529"
scmAccount = "gitlab/10150529" // ACP or Chelsea: agp
scmAppProject = "agp-app1"
scmGitOpsProject = "agp-gitops"
registryURL = "nexus.kt.co.kr:10001" // Nexus URL
//registryCredential = "SDH-nexus-10150529" // unauthorized: access to the requested resource is not authorized
registryCredential = "nexus-certs"
// registryAccount = "agp"
dockerImage = "agp-app1"
}
stages {
stage('Git clone') {
agent {
docker{
image '${imageHubURL}/alpine/git'
args "-it --name gittest --entrypoint="
}
}
steps {
script {
try {
echo "\n### Stage: Git clone"
sh "pwd && ls -a"
withCredentials(bindings: [usernamePassword(credentialsId: "${scmCredential}",
usernameVariable: 'SCM_USER', passwordVariable: 'SCM_PWD')]) {
sh "rm -rf ${scmAppProject}"
//echo "git clone https://$SCM_USER:'$SCM_PWD'@${scmURL}/${scmAccount}/${scmAppProject}.git"
sh "git clone https://$SCM_USER:'$SCM_PWD'@${scmURL}/${scmAccount}/${scmAppProject}.git"
}
} catch(e) {
echo "Error: " + e.toString()
throw e
} finally {
stash includes: "${scmAppProject}/**/*", name: 'app'
}
}
}
}
stage('Build docker image') {
steps {
script{
try {
echo "\n### Stage: Build docker image"
unstash 'app'
newImage = docker.build("${registryURL}/${dockerImage}:${env.BUILD_ID}", "-f ./${scmAppProject}/Dockerfile ./${scmAppProject}")
// this cmd equals = "docker build -t {imageName}:{tag} -f {dockerfile_path} {execution_path}"
sh "docker tag ${registryURL}/${dockerImage}:${env.BUILD_ID} ${registryURL}/${dockerImage}:latest"
} catch(e) {
echo "Error: " + e.toString()
throw e
}
}
}
}
stage('Push docker image'){
steps{
script{
try {
echo "\n### Stage: Push docker image"
docker.withRegistry("https://${registryURL}", "${registryCredential}") {
newImage.push()
}
} catch(e) {
echo "Error: " + e.toString()
throw e
}
}
}
}
stage('Delete docker image') {
steps {
script {
try {
echo "\n### Stage: Delete docker image"
sh "docker rmi -f ${registryURL}/${dockerImage}:${env.BUILD_ID}"
sh "docker rmi -f ${registryURL}/${dockerImage}:latest"
} catch(e) {
echo "Error: " + e.toString()
throw e
}
}
}
}
stage('Static analysis') {
agent {
docker{
image '${imageHubURL}/sonarsource/sonar-scanner-cli'
}
}
steps {
script {
try {
echo "\n### Static analysis"
//sh 'sonar-scanner --version'
sh """
sonar-scanner \
-Dsonar.projectKey=agp-app1 \
-Dsonar.sources=. \
-Dsonar.host.url=https://swdevhub.kt.co.kr/sonar \
-Dsonar.login=6e7be06949e6d2b763e060d68d8f22959354b3f3
"""
} catch(e) {
echo "Error: " + e.toString()
throw e
}
}
}
}
stage('Git clone Gitops') {
agent {
docker {
image '${imageHubURL}/alpine/git'
args "-it --name gittest --entrypoint="
}
}
steps {
script {
try {
echo "\n### Git clone Gitops"
withCredentials(bindings: [usernamePassword(credentialsId: "${scmCredential}",
usernameVariable: 'SCM_USER', passwordVariable: 'SCM_PWD')]) {
sh "rm -rf ${scmGitopsProject}"
sh "git clone https://$SCM_USER:'$SCM_PWD'@${scmURL}/${scmAccount}/${scmGitopsProject}.git"
}
} catch(e) {
echo "Error: " + e.toString()
throw e
}
}
}
}
stage('Kustomize') {
agent {
docker {
image '${imageHubURL}/traherom/kustomize-docker:latest'
args "-it --name kustomizetest --entrypoint="
}
}
steps{
script {
try {
echo "\n### Kustomize"
sh "ls ${scmGitopsProject}/${dockerImage}/overlays/development"
sh """
cd ${scmGitopsProject}/${dockerImage}/overlays/development &&
kustomize edit set image ${registryURL}/${dockerImage}:${env.BUILD_ID}
cat kustomization.yaml
"""
} catch(e) {
echo "Error: " + e.toString()
throw e
}
}
}
}
stage('Git push Gitops'){
agent {
docker {
image '${imageHubURL}/alpine/git'
args "-it --name gittest --entrypoint="
}
}
steps{
script {
try {
echo "\n### Git push Gitops"
sh """
cd ${scmGitopsProject};
git config --global user.email "jenkins@kt.com";
git config --global user.name "Jenkins pipeline";
git add . &&
git commit -m "changed Image tag by Jenkins pipeline" &&
git push
"""
} catch(e) {
echo "Error: " + e.toString()
throw e
}
}
}
}
}
}
부록 B. Jenkinsfile for Kubernetes
podTemplate(label: 'jenkins-slave-pod',
containers: [
containerTemplate(name: 'git' , image: 'alpine/git' , command: 'cat', ttyEnabled: true),
containerTemplate(name: 'docker' , image: 'docker' , command: 'cat', ttyEnabled: true),
containerTemplate(name: 'kustomize', image: 'traherom/kustomize-docker:latest' , command: 'cat', ttyEnabled: true)
// containerTemplate(name: 'kubectl' , image: 'lachlanevenson/k8s-kubectl:v1.16.14', command: 'cat', ttyEnabled: true)
// containerTemplate(name: 'helm' , image: 'lachlanevenson/k8s-helm:latest' , command: 'cat', ttyEnabled: true)
],
volumes: [
hostPathVolume(mountPath: '/var/run/docker.sock', hostPath: '/var/run/docker.sock'),
]
)
{
node('jenkins-slave-pod') {
def dockerImage = "agp-app1"
// Nexus, GitLab Information
def registryURL = "repo.chelsea.kt.co.kr:5002" // Nexus
def registryCredential = "chelsea-nexus-agp"
def registryAccount = "agp"
def scmURL = "scm.chelsea.kt.co.kr" // GitLab
def scmCredential = "chelsea-gitlab-agp"
def scmAccount = "agp"
def scmProject = "agp-gitops"
def registryURLPull = "repo.chelsea.kt.co.kr" // Docker registry의 Pull와 Push 포트가 다른 경우
stage('Clone repository') {
container('git') {
checkout scm
}
}
stage('Build docker image') {
container('docker') {
sh "docker build -t $registryURL/$registryAccount/$dockerImage:${env.BUILD_ID} -f ./Dockerfile ."
sh "docker tag $registryURL/$registryAccount/$dockerImage:${env.BUILD_ID} $registryURL/$registryAccount/$dockerImage:latest"
}
}
stage('Push docker image') {
container('docker') {
withCredentials([ usernamePassword(credentialsId: "$registryCredential",
usernameVariable: 'DOCKER_USER',
passwordVariable: 'DOCKER_PASSWORD')]) {
sh "docker login -u $DOCKER_USER -p $DOCKER_PASSWORD $registryURL"
docker.image("$registryURL/$registryAccount/$dockerImage:${env.BUILD_ID}").push()
docker.image("$registryURL/$registryAccount/$dockerImage:latest").push()
}
}
}
stage('Delete docker image') {
container('docker') {
sh "docker rmi $registryURL/$registryAccount/$dockerImage:${env.BUILD_ID} -f"
sh "docker rmi $registryURL/$registryAccount/$dockerImage:latest -f"
}
}
stage('Deploy to Dev') {
container('git') {
withCredentials([usernamePassword(credentialsId: "$scmCredential",
usernameVariable: 'SCM_USER',
passwordVariable: 'SCM_PASSWORD')]) {
sh """
git clone https://${SCM_USER}:${SCM_PASSWORD}@${scmURL}/${scmAccount}/${scmProject}.git
"""
}
}
container('kustomize') {
sh """
cd $scmProject/$dockerImage/overlays/development
kustomize edit set image $registryURLPull/$registryAccount/$dockerImage:${env.BUILD_ID}
"""
}
container('git') {
withCredentials([usernamePassword(credentialsId: "$scmCredential",
usernameVariable: 'SCM_USER',
passwordVariable: 'SCM_PASSWORD')]) {
sh """
cd $scmProject
git config --global user.email "jenkins@kt.co.kr"
git config --global user.name "Jenkins pipeline"
git add .
git commit -m "changed Image tag by Jenkins pipeline"
git push
"""
}
}
}
}
}
부록 C. Pipeline Use case (KT OOO Project)
a. Jenkins Pipeline UI
b. Blueocean UI
- Blue Ocean rethinks the Jenkins user experience.
부록 D. Version control using “git command”
- https://product.hubspot.com/blog/git-and-github-tutorial-for-beginners
When you clone a remote repository to your local machine, git creates an alias for you.
In nearly all cases this alias is called "origin." It's essentially shorthand for the remote repository's URL.
'Kubernetes > CI-CD' 카테고리의 다른 글
| CI/CD 적용 가이드 #3 (CD-Gitops 편) (0) | 2021.09.26 |
|---|---|
| CI/CD 적용 가이드 #1 (개요) (0) | 2021.09.26 |
| Jenkins (0) | 2021.09.18 |
| Harbor (0) | 2021.09.18 |
| Giblab (0) | 2021.09.17 |
댓글