DevSecOps Project using git, GitHub, jenkins, Maven,Junit, SonarQube, Docker, Trivy, Hashicorp Vault, AWS, Kubernetes
In this project, I created an end-to-end CI/CD pipeline while keeping in mind Securities Best Practices, DevSecOps principles and used all these tools Git, GitHub , Jenkins,Maven, Junit, SonarQube, Docker, Trivy, AWS S3, Docker Hub, Kubernetes , Slack and Hashicorp Vault, to achive the goal.
Push all the web application page code file into github
Jenkins Installation Prequuisities https://www.jenkins.io/doc/book/installing/linux/
Installation guide is available here https://github.com/praveensirvi1212/DevSecOps-project/blob/main/Jenkins_installation.md
After installation, install suggested plugins
Open Jenkins Dashboard and install required plugins – SonarQube Scanner, Hashicorp Vault, Slack
go to manage jenkins > manage pulgins > search for plugins > install without restart
We will required another pulgin called - Kubernetes Continuous Deploy Plugin ( this plugin is deprecated but we can down grade the version for just testing purpose) Download the Plugin file from here https://github.com/praveensirvi1212/DevSecOps-project/blob/main/kubernetes-cd.hpi
Now go to manage jenkins > manage pulgins > Advanced Setting > Deploy Plugin > choose the download file ( kubernetes-cd.hpi) > click on Deploy
I am following Debian/Ubuntu based installation guide you can choose accourding to your os
sudo apt-get install wget apt-transport-https gnupg lsb-release
wget -qO - https://aquasecurity.github.io/trivy-repo/deb/public.key | sudo apt-key add -
echo deb https://aquasecurity.github.io/trivy-repo/deb $(lsb_release -sc) main | sudo tee -a /etc/apt/sources.list.d/trivy.list
sudo apt-get update
sudo apt-get install trivy
In our can we will scan contianer images
trivy image [YOUR_IMAGE_NAME]
HashiCorp Vault is a secret-management tool specifically designed to control access to sensitive credentials in a low-trust environment.
Slack is a workplace communication tool, “a single place for messaging, tools and files.” .
Install Slack from official website of Slack https://slack.com/intl/en-in/downloads/linux
Minikube installation Guide is Available here https://www.linuxtechi.com/how-to-install-minikube-on-ubuntu/
I am assuming that your Vault server is running
Video guide to integrate Hashicorp Vault with Jnekins https://www.youtube.com/watch?v=5-RMu9M_Anc
/etc/vault.d/vault.hcl
storage "raft" {
path = "/opt/vault/data"
node_id = "raft_node_1"
}
listener "tcp" {
address = "0.0.0.0:8200"
tls_disable = 1
}
api_addr = "http://127.0.0.1:8200"
cluster_addr = "https://127.0.0.1:8201"
ui = true
sudo systemctl stop vault
sudo systemctl start vault
export VAULT_ADDR='http://127.0.0.1:8200'
vault operator init
vault operator unseal
vault operator unseal
vault operator unseal
vault login <Initial_Root_Token>
<Initial_Root_Token>
is found in the output of vault operator init
vault auth enable approle
vault auth enable approle
vault write auth/approle/role/jenkins-role token_num_uses=0 secret_id_num_uses=0 policies="jenkins"
vault read auth/approle/role/jenkins-role/role-id
vault write -f auth/approle/role/jenkins-role/secret-id
Open SonarQube and login using admin username and admin password
Create a Project >Enter Project name, Project key > click on setup
Create sonarqube token > and save it soemwhere
click on continue > Run analysis on your project > maven > copy following commands and save it some where
Now go to jenkins >Manage Credentials > system > Add credentials > secret text file > paste token we create in sonarqube and save and apply.
go to manage Jenkins > Configure System > Add SonarQube Server name,url and credentials
go to manage Jenkins > Global tool configuration > Add Maven and SonarQube Scanner
Now go to SonarQube > Quality gates > create your own quality gate
Add conditions with your own requirement
Select your project and Set this quality gate as defalut
Now go to your Project > project setting > webhook
create webhook with your Jenkins url
sudo gpasswd -a jenkins docker
1.Installation Guide is Available here https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html
aws configure
paste your access key and secret key
configure aws cli for jenkins user also
To store secrets into Vault-
docker
secret into Vaultvault secrets enable -path=secrets kv
vault write secrets/creds/docker username=your-dockerhub-username password=token-generated-in-dockerhub
path "secrets/creds/*" {
capabilities = ["read"]
}
vault policy write jenkins jenkins-policy.hcl
Now you have to copy the content of your kubeconfig file of your cluster. for that -
.kube
You will find your kubeconfig like this
apiVersion: v1
clusters:
- cluster:
certificate-authority: /home/praveen/.minikube/ca.crt
extensions:
- extension:
last-update: Fri, 24 Feb 2023 19:17:00 IST
provider: minikube.sigs.k8s.io
version: v1.28.0
name: cluster_info
server: https://192.168.49.2:8443
name: minikube
contexts:
- context:
cluster: minikube
extensions:
- extension:
last-update: Fri, 24 Feb 2023 19:17:00 IST
provider: minikube.sigs.k8s.io
version: v1.28.0
name: context_info
namespace: default
user: minikube
name: minikube
- context:
cluster: ""
namespace: dev
user: ""
name: my-context
current-context: minikube
kind: Config
preferences: {}
users:
- name: minikube
user:
client-certificate: /home/praveen/.minikube/profiles/minikube/client.crt
client-key: home/praveen/.minikube/profiles/minikube/client.key
certificate- authority
to certificate- authority-data
, client-certificate
to client-certificate-data
, client-key
to client-key-data
Now copy the config file data and paste into jenkins > save
vault write secrets/creds/slack secret=your-slack-token
I used Tools, Declarative Pipeline beccause we required build tool called maven
pipeline {
agent any
tools {
maven 'apache-maven-3.0.1'
}
stages {
stage('Example') {
steps {
sh 'mvn --version'
}
}
}
}
stage('Checkout git') {
steps {
git branch: 'main', url: 'https://github.com/praveensirvi1212/DevSecOps-project'
}
}
stage ('Build & JUnit Test') {
steps {
sh 'mvn install'
}
post {
success {
junit 'target/surefire-reports/**/*.xml'
}
}
}
In this stage i used withSonarQubeEnv to Prepare SonarQube Scanner environment and shell command sh
stage('SonarQube Analysis'){
steps{
withSonarQubeEnv('SonarQube-server') {
sh 'mvn clean verify sonar:sonar \
-Dsonar.projectKey=devsecops-project-key \
-Dsonar.host.url=$sonarurl \
-Dsonar.login=$sonarlogin'
}
}
}
This step pauses Pipeline execution and wait for previously submitted SonarQube analysis to be completed and returns quality gate status. Setting the parameter abortPipeline to true will abort the pipeline if quality gate status is not green.
stage("Quality Gate") {
steps {
timeout(time: 1, unit: 'HOURS') {
waitForQualityGate abortPipeline: true
}
}
}
First write your dockerfile to build docker images.I have posted my dockerfile here https://github.com/praveensirvi1212/DevSecOps-project/blob/main/Dockerfile .
In this stage i shell command sh to build docker image
stage('Docker Build') {
steps {
sh 'docker build -t praveensirvi/sprint-boot-app:v1.$BUILD_ID .'
sh 'docker image tag praveensirvi/sprint-boot-app:v1.$BUILD_ID praveensirvi/sprint-boot-app:latest'
}
}
In this stage i trivy shell command sh to scan docker image
stage('Image Scan') {
steps {
sh ' trivy image --format template --template "@/usr/local/share/trivy/templates/html.tpl" -o report.html praveensirvi/sprint-boot-app:latest '
}
}
In this stage i shell command sh to scan docker image
stage('Upload Scan report to AWS S3') {
steps {
sh 'aws s3 cp report.html s3://devsecops-project/'
}
}
For that - S3 plugins should be installed Pipeline Syntax
stage("Upload"){
steps{
withAWS(region:"${region}", credentials:"${aws_credential}){
s3Upload(file:"${TAG_NAME}", bucket:"${bucket}", path:"${TAG_NAME}/")
}
}
}
In this stage i shell command sh to push docker image to docker hub. I stored Credentials into Vault and access into jenkins using vault key. You can store DockerHub credentials into jenkins and use as environment variables
stage('Docker Push') {
steps {
withVault(configuration: [skipSslVerification: true, timeout: 60, vaultCredentialId: 'vault-cred', vaultUrl: 'http://your-vault-server-url:8200'], vaultSecrets: [[path: 'secrets/creds/docker', secretValues: [[vaultKey: 'username'], [vaultKey: 'password']]]]) {
sh "docker login -u ${username} -p ${password} "
sh 'docker push praveensirvi/sprint-boot-app:v1.$BUILD_ID'
sh 'docker push praveensirvi/sprint-boot-app:latest'
sh 'docker rmi praveensirvi/sprint-boot-app:v1.$BUILD_ID praveensirvi/sprint-boot-app:latest'
}
}
}
write your kubernetes deployment and service manifest.Find my kubernetes manifest here https://github.com/praveensirvi1212/DevSecOps-project/blob/main/spring-boot-deployment.yaml .
Now generate pipeline syntax:- For this Kubernetes continuous Deploy plugins should be installed
stage('Deploy to k8s') {
steps {
script{
kubernetesDeploy configs: 'spring-boot-deployment.yaml', kubeconfigId: 'kubernetes'
}
}
}
In post build action i used slack notification . After build jenkins will send notification massage to slack whether your build success or failed.
post{
always{
sendSlackNotifcation()
}
}
sendSlackNotification function
def sendSlackNotifcation()
{
if ( currentBuild.currentResult == "SUCCESS" ) {
buildSummary = "Job_name: ${env.JOB_NAME}\n Build_id: ${env.BUILD_ID} \n Status: *SUCCESS*\n Build_url: ${BUILD_URL}\n Job_url: ${JOB_URL} \n"
slackSend( channel: "#devops", token: 'slack-token', color: 'good', message: "${buildSummary}")
}
else {
buildSummary = "Job_name: ${env.JOB_NAME}\n Build_id: ${env.BUILD_ID} \n Status: *FAILURE*\n Build_url: ${BUILD_URL}\n Job_url: ${JOB_URL}\n \n "
slackSend( channel: "#devops", token: 'slack-token', color : "danger", message: "${buildSummary}")
}
}
After 86th Build my jenkins pipeline became successful.
This Output is the build number 86th. SonarQube Quality gate status is green and passed .
You applied your custom quality gate like : there should be zero ( bug, Vulnerability , code smell ) and your code have greater then 0 (bugs, vulnerability , code smells) . Then your quality gate status will become failure or red. If your quality gate status beome failure , stages after quality gate will be failure.