1 of 100

v0.6

Introduction

Devtron is a tool integration platform for Kubernetes.

Devtron deeply integrates with products across the lifecycle of microservices i.e., CI/CD, security, cost, debugging, and observability via an intuitive web interface. Devtron helps you to deploy, observe, manage & debug the existing Helm apps in all your clusters.

Devtron's Key Features:

No Code Software Delivery Workflow for Kubernetes

Workflow which understands the domain of Kubernetes, testing, CD, SecOps so that you don't have to write scripts
Reusable and composable components so that workflows are easy to construct and reason through

Multi-cloud Deployment

Deploy to multiple Kubernetes clusters on multiple cloud/on-prem from one Devtron setup
Works for all cloud providers and on-premise Kubernetes clusters

Easy DevSecOps Integration

Multi-level security policy at global, cluster, environment, and application-level for efficient hierarchical policy management
Behavior-driven security policy
Define policies and exceptions for Kubernetes resources
Define policies for events for faster resolution

Application Debugging Dashboard

One place for all historical Kubernetes events
Access all manifests securely, such as secret obfuscation
Application metrics for CPU, RAM, HTTP status code, and latency with a comparison between new and old
Advanced logging with grep and JSON search
Intelligent correlation between events, logs for faster triangulation of issue
Auto issue identification

Enterprise-Grade Security and Compliances

Fine-grained access control; control who can edit the configuration and who can deploy.
Audit log to know who did what and when
History of all CI and CD events
Kubernetes events impacting application
Relevant cloud events and their impact on applications
Advanced workflow policies like blackout window, branch environment relationship to secure build and deployment pipelines

Implements GitOps

GitOps exposed through API and UI so that you don't have to interact with git CLI
GitOps backed by Postgres for easy analysis
Enforce finer access control than Git

Operational Insights

Deployment metrics to measure the success of the agile process. It captures MTTR, change failure rate, deployment frequency, and deployment size out of the box.
Audit log to understand the failure causes
Monitor changes across deployments and reverts easily

Compatibility Notes

Application metrics only work for K8s version 1.16+

Contributing Guidelines

Community

Get updates on Devtron's development and chat with the project maintainers, contributors, and community members.

Vulnerability Reporting

We, at Devtron, take security and our users' trust very seriously. If you believe you have found a security issue in Devtron, please responsibly disclose it by contacting us at security@devtron.ai.

Install Devtron

Devtron is installed over a Kubernetes cluster. Once you create a Kubernetes cluster, Devtron can be installed standalone or along with CI/CD integration.

Choose one of the options as per your requirements:

Install Devtron with CI/CD

In this section, we describe the steps in detail on how you can install Devtron with CI/CD integration.

Prerequisites

Run the following command to install AWS EBS CSI driver using Helm:

helm repo add aws-ebs-csi-driver \
https://kubernetes-sigs.github.io/aws-ebs-csi-driver \
helm repo update \
helm upgrade --install aws-ebs-csi-driver \
--namespace kube-system aws-ebs-csi-driver/aws-ebs-csi-driver

Command

Run the following command to install the latest version of Devtron along with the CI/CD module:

helm repo add devtron https://helm.devtron.ai 

helm repo update devtron

helm install devtron devtron/devtron-operator \
--create-namespace --namespace devtroncd \
--set installer.modules={cicd}

Install Multi-Architecture Nodes (ARM and AMD)

To install Devtron on clusters with the multi-architecture nodes (ARM and AMD), append the Devtron installation command with --set installer.arch=multi-arch.

Configure Blob Storage during Installation

Configuring Blob Storage in your Devtron environment allows you to store build logs and cache. In case, if you do not configure the Blob Storage, then:

You will not be able to access the build and deployment logs after an hour.
Build time for commit hash takes longer as cache is not available.
Artifact reports cannot be generated in pre/post build and deployment stages.

Choose one of the options to configure blob storage:

Run the following command to install Devtron along with MinIO for storing logs and cache.

helm repo add devtron https://helm.devtron.ai 

helm repo update devtron

helm install devtron devtron/devtron-operator \
--create-namespace --namespace devtroncd \
--set installer.modules={cicd} \
--set minio.enabled=true

Note: Unlike global cloud providers such as AWS S3 Bucket, Azure Blob Storage and Google Cloud Storage, MinIO can be hosted locally also.

Run the following command to install Devtron along with AWS S3 buckets for storing build logs and cache:

Install using S3 IAM policy.

Note: Please ensure that S3 permission policy to the IAM role attached to the nodes of the cluster if you are using below command.

helm repo add devtron https://helm.devtron.ai

helm repo update devtron

helm install devtron devtron/devtron-operator \
--create-namespace --namespace devtroncd \
--set installer.modules={cicd} \
--set configs.BLOB_STORAGE_PROVIDER=S3 \
--set configs.DEFAULT_CACHE_BUCKET=demo-s3-bucket \
--set configs.DEFAULT_CACHE_BUCKET_REGION=us-east-1 \
--set configs.DEFAULT_BUILD_LOGS_BUCKET=demo-s3-bucket \
--set configs.DEFAULT_CD_LOGS_BUCKET_REGION=us-east-1

Install using access-key and secret-key for AWS S3 authentication:

helm repo add devtron https://helm.devtron.ai

helm repo update devtron

helm install devtron devtron/devtron-operator \
--create-namespace --namespace devtroncd \
--set installer.modules={cicd} \
--set configs.BLOB_STORAGE_PROVIDER=S3 \
--set configs.DEFAULT_CACHE_BUCKET=demo-s3-bucket \
--set configs.DEFAULT_CACHE_BUCKET_REGION=us-east-1 \
--set configs.DEFAULT_BUILD_LOGS_BUCKET=demo-s3-bucket \
--set configs.DEFAULT_CD_LOGS_BUCKET_REGION=us-east-1 \
--set secrets.BLOB_STORAGE_S3_ACCESS_KEY=<access-key> \
--set secrets.BLOB_STORAGE_S3_SECRET_KEY=<secret-key>

Install using S3 compatible storages:

helm repo add devtron https://helm.devtron.ai

helm repo update devtron

helm install devtron devtron/devtron-operator \
--create-namespace --namespace devtroncd \
--set installer.modules={cicd} \
--set configs.BLOB_STORAGE_PROVIDER=S3 \
--set configs.DEFAULT_CACHE_BUCKET=demo-s3-bucket \
--set configs.DEFAULT_CACHE_BUCKET_REGION=us-east-1 \
--set configs.DEFAULT_BUILD_LOGS_BUCKET=demo-s3-bucket \
--set configs.DEFAULT_CD_LOGS_BUCKET_REGION=us-east-1 \
--set secrets.BLOB_STORAGE_S3_ACCESS_KEY=<access-key> \
--set secrets.BLOB_STORAGE_S3_SECRET_KEY=<secret-key> \
--set configs.BLOB_STORAGE_S3_ENDPOINT=<endpoint>

Run the following command to install Devtron along with Azure Blob Storage for storing build logs and cache:

helm repo add devtron https://helm.devtron.ai

helm repo update devtron

helm install devtron devtron/devtron-operator \
--create-namespace --namespace devtroncd \
--set installer.modules={cicd} \
--set secrets.AZURE_ACCOUNT_KEY=xxxxxxxxxx \
--set configs.BLOB_STORAGE_PROVIDER=AZURE \
--set configs.AZURE_ACCOUNT_NAME=test-account \
--set configs.AZURE_BLOB_CONTAINER_CI_LOG=ci-log-container \
--set configs.AZURE_BLOB_CONTAINER_CI_CACHE=ci-cache-container

Run the following command to install Devtron along with Google Cloud Storage for storing build logs and cache:

helm repo add devtron https://helm.devtron.ai

helm repo update devtron

helm install devtron devtron/devtron-operator \
--create-namespace --namespace devtroncd \
--set installer.modules={cicd} \
--set configs.BLOB_STORAGE_PROVIDER=GCP \
--set secrets.BLOB_STORAGE_GCP_CREDENTIALS_JSON=eyJ0eXBlIjogInNlcnZpY2VfYWNjb3VudCIsInByb2plY3RfaWQiOiAiPHlvdXItcHJvamVjdC1pZD4iLCJwcml2YXRlX2tleV9pZCI6ICI8eW91ci1wcml2YXRlLWtleS1pZD4iLCJwcml2YXRlX2tleSI6ICI8eW91ci1wcml2YXRlLWtleT4iLCJjbGllbnRfZW1haWwiOiAiPHlvdXItY2xpZW50LWVtYWlsPiIsImNsaWVudF9pZCI6ICI8eW91ci1jbGllbnQtaWQ+IiwiYXV0aF91cmkiOiAiaHR0cHM6Ly9hY2NvdW50cy5nb29nbGUuY29tL28vb2F1dGgyL2F1dGgiLCJ0b2tlbl91cmkiOiAiaHR0cHM6Ly9vYXV0aDIuZ29vZ2xlYXBpcy5jb20vdG9rZW4iLCJhdXRoX3Byb3ZpZGVyX3g1MDlfY2VydF91cmwiOiAiaHR0cHM6Ly93d3cuZ29vZ2xlYXBpcy5jb20vb2F1dGgyL3YxL2NlcnRzIiwiY2xpZW50X3g1MDlfY2VydF91cmwiOiAiPHlvdXItY2xpZW50LWNlcnQtdXJsPiJ9Cg== \
--set configs.DEFAULT_CACHE_BUCKET=cache-bucket \
--set configs.DEFAULT_BUILD_LOGS_BUCKET=log-bucket

Check Status of Devtron Installation

The installation takes about 15 to 20 minutes to spin up all of the Devtron microservices one by one

Run the following command to check the status of the installation:

kubectl -n devtroncd get installers installer-devtron \
-o jsonpath='{.status.sync.status}'

The command executes with one of the following output messages, indicating the status of the installation:

Status

Description

Downloaded

The installer has downloaded all the manifests, and the installation is in progress.

Applied

The installer has successfully applied all the manifests, and the installation is completed.

Check the Installer Logs

Run the following command to check the installer logs:

kubectl logs -f -l app=inception -n devtroncd

Devtron Dashboard

Run the following command to get the Devtron dashboard URL:

kubectl get svc -n devtroncd devtron-service \
-o jsonpath='{.status.loadBalancer.ingress}'

You will get an output similar to the example shown below:

[map[hostname:aaff16e9760594a92afa0140dbfd99f7-305259315.us-east-1.elb.amazonaws.com]]

Use the hostname aaff16e9760594a92afa0140dbfd99f7-305259315.us-east-1.elb.amazonaws.com (Loadbalancer URL) to access the Devtron dashboard.

If you do not get a hostname or receive a message that says "service doesn't exist," it means Devtron is still installing. Please wait until the installation is completed.

You can also use a CNAME entry corresponding to your domain/subdomain to point to the Loadbalancer URL to access at a customized domain.

Host

Type

Points to

devtron.yourdomain.com

CNAME

aaff16e9760594a92afa0140dbfd99f7-305259315.us-east-1.elb.amazonaws.com

Devtron Admin Credentials

When you install Devtron for the first time, it creates a default admin user and password (with unrestricted access to Devtron). You can use that credentials to log in as an administrator.

After the initial login, we recommend you set up any SSO service like Google, GitHub, etc., and then add other users (including yourself). Subsequently, all the users can use the same SSO (let's say, GitHub) to log in to Devtron's dashboard.

The sections below will help you understand the process of getting the administrator password.

For Devtron version v0.6.0 and higher

Username: admin Password: Run the following command to get the admin password:

kubectl -n devtroncd get secret devtron-secret \
-o jsonpath='{.data.ADMIN_PASSWORD}' | base64 -d

For Devtron version less than v0.6.0

Username: admin Password: Run the following command to get the admin password:

kubectl -n devtroncd get secret devtron-secret \
-o jsonpath='{.data.ACD_PASSWORD}' | base64 -d

Install Devtron without Integrations

Before you begin

Add Helm Repo

Update Helm Repo

Install Helm Dashboard by Devtron

Run the following command to install Helm Dashboard by Devtron:

Install Multi-Architecture Nodes (ARM and AMD)

To install Devtron on clusters with the multi-architecture nodes (ARM and AMD), append the Devtron installation command with --set installer.arch=multi-arch.

Devtron Dashboard

Run the following command to get the dashboard URL:

You will get the result something as shown below:

The hostname aaff16e9760594a92afa0140dbfd99f7-305259315.us-east-1.elb.amazonaws.com as mentioned above is the Loadbalancer URL where you can access the Devtron dashboard.

You can also do a CNAME entry corresponding to your domain/subdomain to point to this Loadbalancer URL to access it at a custom domain.

Devtron Admin credentials

When you install Devtron for the first time, it creates a default admin user and password (with unrestricted access to Devtron). You can use that credentials to log in as an administrator.

The section below will help you understand the process of getting the administrator credentials.

For Devtron version v0.6.0 and higher

Username: admin Password: Run the following command to get the admin password:

For Devtron version less than v0.6.0

Username: admin Password: Run the following command to get the admin password:

Upgrade

Install Devtron on Minikube, Microk8s, K3s, Kind, Cloud VMs

You can install and try Devtron on a high-end machine or a Cloud VM. If you install it on a laptop/PC, it may start to respond slowly, so it is recommended to uninstall Devtron from your system before shutting it down.

Prerequisites

2 vCPUs
4GB+ of free memory
20GB+ free disk space

Before you get started, you must set up a cluster in your server and finish the following actions:

Tutorial

For Minikube, Microk8s, K3s, Kind

To install devtron on Minikube/kind cluster, run the following command:

helm repo add devtron https://helm.devtron.ai

helm repo update devtron

helm install devtron devtron/devtron-operator \
--create-namespace --namespace devtroncd \
--set components.devtron.service.type=NodePort --set installer.arch=multi-arch

To install devtron on k3s cluster, run the following command:

kubectl apply -f https://raw.githubusercontent.com/rancher/local-path-provisioner/master/deploy/local-path-storage.yaml

helm repo add devtron https://helm.devtron.ai

helm repo update devtron

helm install devtron devtron/devtron-operator \
--create-namespace --namespace devtroncd \
--set components.devtron.service.type=NodePort

Access Devtron Dashboard

To access Devtron dashboard when using Minikube as cluster, run the following command:

minikube service devtron-service --namespace devtroncd

To access Devtron dashboard when using Kind/k3s as cluster, run the following command to port forward the devtron service to port 8000:

kubectl -n devtroncd port-forward service/devtron-service 8000:80

Get Admin Credentials

When you install Devtron for the first time, it creates a default admin user and password (with unrestricted access to Devtron). You can use that credentials to log in as an administrator.

The section below will help you understand the process of getting the administrator credentials.

For Devtron version v0.6.0 and higher

Username: admin Password: Run the following command to get the admin password:

kubectl -n devtroncd get secret devtron-secret \
-o jsonpath='{.data.ADMIN_PASSWORD}' | base64 -d

For Devtron version less than v0.6.0

Username: admin Password: Run the following command to get the admin password:

kubectl -n devtroncd get secret devtron-secret \
-o jsonpath='{.data.ACD_PASSWORD}' | base64 -d

For Cloud VM (AWS EC2, Azure VM, GCP VM)

It is recommended to use Cloud VM with 2vCPU+, 4GB+ free memory, 20GB+ storage, Compute Optimized VM type & Ubuntu Flavoured OS.

Create Microk8s Cluster

sudo snap install microk8s --classic --channel=1.22
sudo usermod -a -G microk8s $USER
sudo chown -f -R $USER ~/.kube
newgrp microk8s
microk8s enable dns storage helm3
echo "alias kubectl='microk8s kubectl '" >> .bashrc
echo "alias helm='microk8s helm3 '" >> .bashrc
source .bashrc

Install Devtron

helm repo add devtron https://helm.devtron.ai

helm repo update devtron

helm install devtron devtron/devtron-operator \
--create-namespace --namespace devtroncd \
--set components.devtron.service.type=NodePort

Get devtron-service Port Number

kubectl get svc -n devtroncd devtron-service -o jsonpath='{.spec.ports[0].nodePort}'

Make sure that the port on which the devtron-service runs remain open in the VM's security group or network security group.

Demo on Popular Cloud Providers

Here we have demonstrated the installation of Devtron on popular cloud providers. The videos are easy to follow and provide step-by-step instructions.

Installing on EKS Cluster

Installing on AKS Cluster

Installing on GKE Cluster

Backup for Disaster Recovery

Regular backups for Devtron PostgreSQL and ArgoCD are crucial components of a disaster recovery plan, as they protect against potential data loss due to unforeseen circumstances. This documentation provides instructions on how to take backups of Devtron and store them either on AWS S3 or Azure containers.

Go to the devtron chart store and search for devtron-backups chart.

Select the devtron-backups and click Configure & Deploy.
Now follow either of the options described below according to your Cloud provider.

AWS S3 Backup

To store Devtron backups on AWS S3, please follow these steps:

Create an S3 bucket to store the Devtron backup, you can configure the bucket to delete all the objects older than 15/30 days.
Create a user with sufficient permissions to push to the S3 bucket created in step 1.
Obtain the access key and secret access key for the created user.
Configure the devtron-backups chart for AWS S3 by selecting the appropriate options:

Deploy the chart, and the Devtron backup will be automatically uploaded to the AWS S3 bucket at the scheduled intervals.

Azure Containers Backup

To store Devtron backups on Azure Containers, please follow these steps:

Create a storage account in Azure.
Within the storage account, create two containers for the Devtron backup.
Navigate to Security + Networking > Access Key section in Azure and copy the Access Key:

Configure the devtron-backups chart for Azure Containers by providing the Access Key:

Before deploying the backup chart, ensure that AWS.enabled is set to false. This will ensure that Devtron backup will be automatically uploaded to the configured Azure containers on the scheduled intervals.

By following these steps, you can ensure that your Devtron data is securely backed up and protected against any potential data loss, enabling you to recover quickly in case of emergencies.

Uninstall Devtron

To uninstall Devtron, run the following command:

This command will remove all the namespaces related to Devtron (devtroncd, devtron-cd, devtron-ci etc.).

Devtron Kubernetes Client

Overview

The Kubernetes client by Devtron is a very lightweight dashboard that can be installed on arm64/amd64-based architectures. It comes with the features such as Kubernetes Resources Browser and Cluster Management that can provide control and observability for resources across clouds and clusters.

Devtron Kubernetes client is an intuitive Kubernetes Dashboard or a command line utility installed outside a Kubernetes cluster. The client can be installed on a desktop running on any Operating Systems and interact with all your Kubernetes clusters and workloads through an API server. It is a binary, packaged in a bash script that you can download and install by using the following set of commands.

By installing Devtron Kubernetes Client, you can access:

Here are a few advantages of using Devtron Kubernetes Client:

Managing Kubernetes Resources at scale: Clusters vary on business and architectural needs. Organizations tend to build smaller clusters for more decentralization. This practice leads to the creation of multiple clusters and more nodes. Managing them on a CLI requires multiple files, making it difficult to perform resource operations. But with the Devtron Kubernetes Client, you can gain more visibility into K8s resources easily.
Unifying information in one place: When information is scattered across clusters, and you have to type commands with arguments to fetch desired output, the process becomes slow and error-prone. Without a single point of configuration source, the configurations of different config. files diverge, making them even more challenging to restore and track. The Devtron Kubernetes Client unifies all the information and tools into one interface to perform various contextual tasks.
Accessibility during an outage for troubleshooting: As the Devtron Kubernetes Client runs outside a cluster, you can exercise basic control over their failed resources when there is a cluster-level outage. The Client helps to gather essential logs and data to pinpoint the root cause of the issue and reduce the time to restore service.
Avoiding Kubeconfig version mismatch errors: With the Devtron Kubernetes Client, you can be relieved from maintaining the Kubeconfig versions for the respective clusters (v1.16 - 1.26 i.e, current version) as the Devtron Kubernetes Client performs self kubeconfig version control. Instead of managing multiple kubectl versions manually, it eliminates the chances of errors occurring due to the mismatch in configuration.

Install Devtron Kubernetes Client

Download the bash script using the below URL: https://cdn.devtron.ai/k8s-client/devtron-install.bash
To automatically download the executable file and to open the dashboard in the respective browser, run the following command:

Note: Make sure you place Devtron-install.bash in your current directory before you execute the command.

Devtron Kubernetes Client opens in your browser automatically.

Note: You do not need to have a super admin permission to add a cluster if you install Devtron Kubernetes Client. You can add more than one cluster.

Kubernetes Resource Browser

Kubernetes Resource Browser provides a graphical user interface for interacting and managing all your Kubernetes (k8s) resources across clusters. It also helps you to deploy and manage Kubernetes resources and allows pod operations such as:

View real-time logs
Check manifest and edit live manifests of k8s resources
Executable via terminal
View Events
Or, delete a resource

With Kubernetes Resource browser, you can also perform the following:

Check the real-time health status
Search for any workloads
Manage multiple clusters and change cluster contexts
Deploy multiple K8s manifests through Create UI option.
Perform resource grouping at the cluster level.

Note: You do not need to have a super admin permission to access Kubernetes Resource Browser if you install Devtron Kubernetes Client.

Cluster Management

With the Devtron Kubernetes Client, you can manage all your clusters running on-premises or on a cloud. It is a cluster and cloud agnostic platform where you can add as many clusters as you want, be it a lightweight cluster such as k3s/ microk8s or cloud managed clusters like Amazon EKS.

It enables you to observe and monitor the cluster health and real-time node conditions. The Cluster management feature provides a summary of nodes with all available labels, annotations, taints, and other parameters such as resource usage. In addition to that, it helps you to perform node operations such as:

Debug a node
Cordon a node
Drain a node
Taint a node
Edit a node config
Delete a node

Some Peripheral Commands

In case if you close the browser by mistake, you can open the dashboard by executing the following command. It will open the dashboard through a port in the available web browser and store the Kubernetes client's state.

To stop the dashboard, you can execute the following command:

To update the Devtron Kubernetes Client, use the following command. It will stop the running dashboard and download the latest executable file and open it in the browser.

Configurations

You can configure Devtron by using configuration files. Configuration files are YAML files which are user-friendly. The configuration allows you to quickly roll back a configuration change if necessary. It also aids cluster re-creation and restoration.

There are two ways you can perform configurations while setting up Devtron dashboard:

Installation Configurations

Configure Secrets

For Helm installation this section refers to secrets section of values.yaml.

Configure the following properties:

Configure ConfigMaps

For Helm installation this section refers to configs section of values.yaml.

Configure the following properties:

Configure Resources

Devtron provides ways to control how much memory or CPU can be allocated to each Devtron microservice. You can adjust the resources that are allocated to these microservices based on your requirements. The resource configurations are available in following sizes:

Small: To configure the small resources (e.g. to manage less than 10 apps on Devtron ) based on the requirements, append the Devtron installation command with -f https://raw.githubusercontent.com/devtron-labs/devtron/main/charts/devtron/resources-small.yaml.

Configure Overrides

For Helm installation this section refers to customOverrides section of values.yaml. In this section you can override values of devtron-cm which you want to keep persistent. For example:

You can configure the following properties:

Storage for Logs and Cache

`AWS SPECIFIC`

While installing Devtron and using the AWS-S3 bucket for storing the logs and caches, the below parameters are to be used in the ConfigMap.

NOTE: For using the S3 bucket it is important to add the S3 permission policy to the IAM role attached to the nodes of the cluster.

The below parameters are to be used in the Secrets :

`AZURE SPECIFIC`

While installing Devtron using Azure Blob Storage for storing logs and caches, the below parameters will be used in the ConfigMap.

`GOOGLE CLOUD STORAGE SPECIFIC`

While installing Devtron using Google Cloud Storage for storing logs and caches, the below parameters will be used in the ConfigMap.

To convert string to base64 use the following command:

Note:
Ensure that the cluster has read and write access to the S3 buckets/Azure Blob storage container mentioned in DEFAULT_CACHE_BUCKET, DEFAULT_BUILD_LOGS_BUCKET or AZURE_BLOB_CONTAINER_CI_LOG, or AZURE_BLOB_CONTAINER_CI_CACHE.
Ensure that the cluster has read access to AWS secrets backends (SSM & secrets manager).

We can use the --set flag to override the default values when installing with Helm. For example, to update POSTGRESQL_PASSWORD and BLOB_STORAGE_PROVIDER, use the install command as:

Configuration of Blob Storage

Blob Storage allows users to store large amounts of unstructured data. Unstructured data is a data that does not adhere to a particular data model or definition, such as text or binary data. Configuring blob storage in your Devtron environment allows you to store build logs and cache.

In case, if you do not configure the Blob Storage, then:

You will not be able to access the build and deployment logs after an hour.
Build time for commit hash takes longer as cache is not available.
Artifact reports cannot be generated in pre/post build and deployment stages.

You can configure Blob Storage with one of the following Blob Storage providers given below:

Note: You can also use the respective following command to switch to another Blob Storage provider. As an example, If you are using MinIO Storage and want to switch to Azure Blob Storage, use the command provided on the Azure Blob Storage tab to switch.

Use the following command to configure MinIO for storing logs and cache.

Note: Unlike global cloud providers such as AWS S3 Bucket, Azure Blob Storage and Google Cloud Storage, MinIO can be hosted locally also.

Configure using S3 IAM policy:

NOTE: Pleasee ensure that S3 permission policy to the IAM role attached to the nodes of the cluster if you are using the below command.

Configure using access-key and secret-key for aws S3 authentication:

Configure using S3 compatible storages:

Secrets

ConfigMaps

Dashboard Configurations

Usage

User Permissions

Like any enterprise product, Devtron supports fine grained access control to the resources based on:

Type of action allowed on Devtron resources (Create Vs View)
Sensitivity of the data (Editing image Vs Editing memory)

Role-based Access Levels

Devtron supports the following levels of access:

View only: User with View only access has the least privilege. This user can only view combination of environments, applications and helm charts on which access has been granted to the user. This user cannot view sensitive data like secrets used in applications or charts.
Build and Deploy: In addition to View only access, user with Build and deploy permission can build and deploy the image of the permitted applications and helm charts to the permitted environments.
Admin: User with Admin access can create, edit, delete and view permitted applications in the permitted projects.
Manager: User with Manager access can do everything that an Admin type user can do, in addition, they can also give and revoke access of users for the applications and environments of which they are Manager.
Super Admin: User with Super admin privilege has unrestricted access to all Devtron resources. Super admin can create, modify, delete and view any Devtron resource without any restriction; its like Superman without the weakness of Kryptonite. Super Admin can also add and delete user access across any Devtron resource, add delete git repository credentials, container registry credentials, cluster and environment.

User Roles And Permissions

1. Custom Applications

User Roles

View

Create

Edit

Delete

Build & Deploy

View

Yes

Build and Deploy

Yes

Admin

Yes

Manager

Yes

Super Admin

Yes

2. Helm Charts

User Roles

View

Deploy

Edit

Delete

View Only

Yes

View and Edit

Yes

Admin

Yes

Super Admin

Yes

3. User Access

User Roles

Add User Access

Edit User Access

Delete User Access

Manager

Yes

Super Admin

Yes

4. Global Configurations

User Role

Add Global Config

Edit Global Config

Delete Global Config

Super Admin

Yes

Add User

To add a user, go to the Authorization > User Permissions section of Global Configurations. Click Add user.

There are two types of permissions in Devtron:

Permission Type

Description

Specific permissions

Devtron Apps

Helm Apps

Kubernetes Resources

Chart Groups

Super admin permission

Assign Super admin permission

To assign a super admin access, go to the Authorization > User Permissions section of Global Configurations.

Click Add user.
Provide the email address of a user. You can add more than one email address. Please note that email address must be same as that in the email field in the JWT token returned by OIDC provider.
Select Super admin permission and click Save.

Note:

Only users with Super admin permission can assign super admin permissions to a user.
We suggest that super admin access must be given to the selected users only.

Assign Specific permissions

To assign a specific permission, go to the Authorization > User Permissions section of Global Configurations.

Click Add user.
Provide the email address of a user. You can add more than one email address. Please note that email address must be same as that in the email field in the JWT token returned by OIDC provider.
Select Specific permissions.
Select the group permission from the drop-down list, if required.

Devtron Apps Permissions

In Devtron Apps option, you can provide access to a user to manage permission for custom apps created using Devtron.

Provide the information in the following fields:

Registry Type

Credentials

Project

Select a project from the drop-down list to which you want to give permission to the user. You can select only one project at a time. Note: If you want to select more than one project, then click Add row.

Environment

Select the specific environment or all environments from the drop-down list. Note: If you select All environments option, then a user gets access to all the current environments including any new environment which gets associated with the application later.

Application

Select the specific applications or all applications from the drop-down list corresponding to your selected Environments. Note: If you select All applications option, then a user gets access to all the current applications including any new application which gets associated with the project later .

Role

View only

Build and Deploy

Admin

Manager

You can add multiple rows for Devtron app permission.

Once you have finished assigning the appropriate permissions for the users, Click Save.

Helm Apps Permissions

In Helm Apps option, you can provide access to a user to manage permission for Helm apps deployed from Devtron or outside Devtron.

Provide the information in the following fields:

Registry Type

Credentials

Project

Environment or cluster/namespace

Select the specific environment or all existing environments in default cluster from the drop-down list. Note: If you select all existing + future environments in default cluster option, then a user gets access to all the current environments including any new environment which gets associated with the application later.

Application

Select the specific application or all applications from the drop-down list corresponding to your selected Environments. Note: If All applications option is selected, then a user gets access to all the current applications including any new application which gets associated with the project later .

Role

View only

View & Edit

Admin

You can add multiple rows for Devtron app permission.

Once you have finished assigning the appropriate permissions for the users, Click Save.

Kubernetes Resources Permissions

Note: Only super admin users will be able to see Kubernetes Resources tab and provide permission to other users to access Resource Browser.

To provide Kubernetes resource permission, click Add permission.

On the Kubernetes resource permission, provide the information in the following fields:

Registry Type

Credentials

Cluster

Select a cluster from the drop-down list to which you want to give permission to the user. You can select only one cluster at a time. Note: To add another cluster, then click Add another.

Namespace

Select the namespace from the drop-down list.

API Group

Select the specific API group or All API groups from the drop-down list corresponding to the K8s resource.

Kind

Select the kind or All kind from the drop-down list corresponding to the K8s resource.

Resource name

Select the resource name or All resources from the drop-down list to which you want to give permission to the user.

Role

View

Admin

You can add multiple rows for Kubernetes resource permission.

Once you have finished assigning the appropriate permissions for the users, Click Save.

Chart Group Permissions

In Chart group permission option, you can manage the access of users for Chart Groups in your project.

NOTE: You can only give users the ability to create or edit, not both.

Action

Permissions

View

Enable View to view chart groups only.

Create

Enable Create if you want the users to create, view, edit or delete the chart groups.

Edit

Deny: Select Deny option from the drop-down list to restrict the users to edit the chart groups.
Specific chart groups: Select the Specific Charts Groups option from the drop-down list and then select the chart group for which you want to allow users to edit.

Click Saveonce you have configured all the required permissions for the users.

Edit User Permissions

You can edit the user permissions by clicking on the downward arrow.

Edit the user permissions.

After you have done editing the user permissions, click Save.

If you want to delete the user/users with particular permissions, click Delete.

Deployment

This chart creates a deployment that runs multiple replicas of your application and automatically replaces any instances that fail or become unresponsive. It does not support Blue/Green and Canary deployments.

This is the default deployment chart. You can select Deployment chart when you want to use only basic use cases which contain the following:

Create a Deployment to rollout a ReplicaSet. The ReplicaSet creates Pods in the background. Check the status of the rollout to see if it succeeds or not.
Declare the new state of the Pods. A new ReplicaSet is created and the Deployment manages moving the Pods from the old ReplicaSet to the new one at a controlled rate. Each new ReplicaSet updates the revision of the Deployment.
Rollback to an earlier Deployment revision if the current state of the Deployment is not stable. Each rollback updates the revision of the Deployment.
Scale up the Deployment to facilitate more load.
Use the status of the Deployment as an indicator that a rollout has stuck.
Clean up older ReplicaSets that you do not need anymore.

You can define application behavior by providing information in the following sections:

1. Yaml File

Container Ports

This defines ports on which application services will be exposed to other services

EnvVariables

To set environment variables for the containers that run in the Pod.

EnvVariablesFromFieldPath

To set environment variables for the containers and fetching their values from pod-level fields.

Liveness Probe

If this check fails, kubernetes restarts the pod. This should return error code in case of non-recoverable error.

MaxUnavailable

The maximum number of pods that can be unavailable during the update process. The value of "MaxUnavailable: " can be an absolute number or percentage of the replicas count. The default value of "MaxUnavailable: " is 25%.

MaxSurge

The maximum number of pods that can be created over the desired number of pods. For "MaxSurge: " also, the value can be an absolute number or percentage of the replicas count. The default value of "MaxSurge: " is 25%.

Min Ready Seconds

This specifies the minimum number of seconds for which a newly created Pod should be ready without any of its containers crashing, for it to be considered available. This defaults to 0 (the Pod will be considered available as soon as it is ready).

Readiness Probe

If this check fails, kubernetes stops sending traffic to the application. This should return error code in case of errors which can be recovered from if traffic is stopped.

Pod Disruption Budget

You can create PodDisruptionBudget for each application. A PDB limits the number of pods of a replicated application that are down simultaneously from voluntary disruptions. For example, an application would like to ensure the number of replicas running is never brought below the certain number.

You can specify either maxUnavailable or minAvailable in a PodDisruptionBudget and it can be expressed as integers or as a percentage.

Ambassador Mappings

You can create ambassador mappings to access your applications from outside the cluster. At its core a Mapping resource maps a resource to a service.

Autoscaling

This is connected to HPA and controls scaling up and down in response to request load.

Flagger

You can use flagger for canary releases with deployment objects. It supports flexible traffic routing with istio service mesh as well.

Fullname Override

fullnameOverride replaces the release fullname created by default by devtron, which is used to construct Kubernetes object names. By default, devtron uses {app-name}-{environment-name} as release fullname.

Image

Image is used to access images in kubernetes, pullpolicy is used to define the instances calling the image, here the image is pulled when the image is not present,it can also be set as "Always".

imagePullSecrets

imagePullSecrets contains the docker credentials that are used for accessing a registry.

serviceAccount

HostAliases

the hostAliases field is used in a Pod specification to associate additional hostnames with the Pod's IP address. This can be helpful in scenarios where you need to resolve specific hostnames to the Pod's IP within the Pod itself.

Ingress

This allows public access to the url, please ensure you are using right nginx annotation for nginx class, its default value is nginx

Legacy deployment-template ingress format

Ingress Internal

This allows private access to the url, please ensure you are using right nginx annotation for nginx class, its default value is nginx

Init Containers

Specialized containers that run before app containers in a Pod. Init containers can contain utilities or setup scripts not present in an app image. One can use base image inside initContainer by setting the reuseContainerImage flag to true.

Pause For Seconds Before Switch Active

To wait for given period of time before switch active the container.

Resources

These define minimum and maximum RAM and CPU available to the application.

Resources are required to set CPU and memory usage.

Limits

Limits make sure a container never goes above a certain value. The container is only allowed to go up to the limit, and then it is restricted.

Requests

Requests are what the container is guaranteed to get.

Service

This defines annotations and the type of service, optionally can define name also.

Volumes

It is required when some values need to be read from or written to an external disk.

Volume Mounts

It is used to provide mounts to the volume.

Affinity and anti-affinity

Spec is used to define the desire state of the given container.

Node Affinity allows you to constrain which nodes your pod is eligible to schedule on, based on labels of the node.

Inter-pod affinity allow you to constrain which nodes your pod is eligible to be scheduled based on labels on pods.

Key

Key part of the label for node selection, this should be same as that on node. Please confirm with devops team.

Values

Value part of the label for node selection, this should be same as that on node. Please confirm with devops team.

Tolerations

Taints are the opposite, they allow a node to repel a set of pods.

A given pod can access the given node and avoid the given taint only if the given pod satisfies a given taint.

Taints and tolerations are a mechanism which work together that allows you to ensure that pods are not placed on inappropriate nodes. Taints are added to nodes, while tolerations are defined in the pod specification. When you taint a node, it will repel all the pods except those that have a toleration for that taint. A node can have one or many taints associated with it.

Arguments

This is used to give arguments to command.

Command

It contains the commands for the server.

Containers

Containers section can be used to run side-car containers along with your main container within same pod. Containers running within same pod can share volumes and IP Address and can address each other @localhost. We can use base image inside container by setting the reuseContainerImage flag to true.

Container Lifecycle Hooks

Container lifecycle hooks are mechanisms that allow users to define custom actions to be performed at specific stages of a container's lifecycle i.e. PostStart or PreStop.

Prometheus

It is a kubernetes monitoring tool and the name of the file to be monitored as monitoring in the given case. It describes the state of the Prometheus.

rawYaml

Accepts an array of Kubernetes objects. You can specify any kubernetes yaml here and it will be applied when your app gets deployed.

Grace Period

Kubernetes waits for the specified time called the termination grace period before terminating the pods. By default, this is 30 seconds. If your pod usually takes longer than 30 seconds to shut down gracefully, make sure you increase the GracePeriod.

A Graceful termination in practice means that your application needs to handle the SIGTERM message and begin shutting down when it receives it. This means saving all data that needs to be saved, closing down network connections, finishing any work that is left, and other similar tasks.

There are many reasons why Kubernetes might terminate a perfectly healthy container. If you update your deployment with a rolling update, Kubernetes slowly terminates old pods while spinning up new ones. If you drain a node, Kubernetes terminates all pods on that node. If a node runs out of resources, Kubernetes terminates pods to free those resources. It’s important that your application handle termination gracefully so that there is minimal impact on the end user and the time-to-recovery is as fast as possible.

Server

It is used for providing server configurations.

Deployment

It gives the details for deployment.

Service Monitor

It gives the set of targets to be monitored.

Db Migration Config

It is used to configure database migration.

Istio

These Istio configurations collectively provide a comprehensive set of tools for controlling access, authenticating requests, enforcing security policies, and configuring traffic behavior within a microservices architecture. The specific settings you choose would depend on your security and traffic management requirements.

KEDA Autoscaling

Example for autosccaling with KEDA using Prometheus metrics is given below:

Example for autosccaling with KEDA based on kafka is given below :

NetworkPolicy

Kubernetes NetworkPolicies control pod communication by defining rules for incoming and outgoing traffic.

Winter-Soldier

Winter Soldier can be used to

cleans up (delete) Kubernetes resources
reduce workload pods to 0

Given below is template values you can give in winter-soldier:

here is an example,

Above settings will take action on Sat and Sun from 00:00 to 23:59:59, and on Mon-Fri from 00:00 to 08:00 and 20:00 to 23:59:59. If action:sleep then runs hibernate at timeFrom and unhibernate at timeTo. If action: delete then it will delete workloads at timeFrom and timeTo. Here the action:scale thus it scale the number of resource replicas to targetReplicas: [1,1,1]. Here each element of targetReplicas array is mapped with the corresponding elements of array timeRangesWithZone/timeRanges. Thus make sure the length of both array is equal, otherwise the cnages cannot be observed.

The above example will select the application objects which have been created 10 hours ago across all namespaces excluding application's namespace. Winter soldier exposes following functions to handle time, cpu and memory.

ParseTime - This function can be used to parse time. For eg to parse creationTimestamp use ParseTime({{metadata.creationTimestamp}}, '2006-01-02T15:04:05Z')
AddTime - This can be used to add time. For eg AddTime(ParseTime({{metadata.creationTimestamp}}, '2006-01-02T15:04:05Z'), '-10h') ll add 10h to the time. Use d for day, h for hour, m for minutes and s for seconds. Use negative number to get earlier time.
Now - This can be used to get current time.
CpuToNumber - This can be used to compare CPU. For eg any({{spec.containers.#.resources.requests}}, { MemoryToNumber(.memory) < MemoryToNumber('60Mi')}) will check if any resource.requests is less than 60Mi.

Security Context

A security context defines privilege and access control settings for a Pod or Container.

To add a security context for main container:

To add a security context on pod level:

Topology Spread Constraints

You can use topology spread constraints to control how Pods are spread across your cluster among failure-domains such as regions, zones, nodes, and other user-defined topology domains. This can help to achieve high availability as well as efficient resource utilization.

Deployment Metrics

It gives the realtime metrics of the deployed applications

2. Show application metrics

If you want to see application metrics like different HTTP status codes metrics, application throughput, latency, response time. Enable the Application metrics from below the deployment template Save button. After enabling it, you should be able to see all metrics on App detail page. By default it remains disabled.

Helm Chart Json Schema

Other Validations in Json Schema

The values of CPU and Memory in limits must be greater than or equal to in requests respectively. Similarly, In case of envoyproxy, the values of limits are greater than or equal to requests as mentioned below.

CI Pipeline

Creating CI Pipeline

A CI Workflow can be created in one of the following ways:

Sync with Environment
Create a Job

Each method has different use-cases that can be tailored according the needs of the organization.

1. Build and Deploy from Source Code

Build and Deploy from Source Code workflow allows you to build the container image from a source code repository.

From the Applications menu, select your application.
On the App Configuration page, select Workflow Editor.
Select + New Workflow.
Select Build and Deploy from Source Code.
Enter the following fields on the Create build pipeline window:

Advanced Options

The Advanced CI Pipeline includes the following stages:

Pre-build stage: The tasks in this stage are executed before the image is built.
Build stage: In this stage, the build is triggered from the source code that you provide.
Post-build stage: The tasks in this stage will be triggered once the build is complete.

Build Stage

Go to the Build stage tab.

Source type

Branch Fixed

This allows you to trigger a CI build whenever there is a code change on the specified branch.

Enter the Branch Name of your code repository.

Branch Regex

Branch Regex allows users to easily switch between branches matching the configured Regex before triggering the build pipeline. In case of Branch Fixed, users cannot change the branch name in ci-pipeline unless they have admin access for the app. So, if users with Build and Deploy access should be allowed to switch branch name before triggering ci-pipeline, Branch Regex should be selected as source type by a user with Admin access.

For example if the user sets the Branch Regex as feature-*, then users can trigger from branches such as feature-1450, feature-hot-fix etc.

Pull Request

This allows you to trigger the CI build when a pull request is created in your repository.

Prerequisites

To trigger the build from specific PRs, you can filter the PRs based on the following keys:

Select the appropriate filter and pass the matching condition as a regular expression (regex).

Select Create Pipeline.

Tag Creation

This allows you to trigger the CI build whenever a new tag is created.

Prerequisites

To trigger the build from specific tags, you can filter the tags based on the author and/or the tag name.

Select the appropriate filter and pass the matching condition as a regular expression (regex).

Select Create Pipeline.

Scan for Vulnerabilities

Prerequisite

Install any one of the following integrations from Devtron Stack Manager:

Trivy

Custom Image Tag Pattern

This feature helps you apply custom tags (e.g., v1.0.0) to readily distinguish container images within your repository.

Enable the toggle button as shown below.
You can write an alphanumeric pattern for your image tag, e.g., test-v1.0.{x}. Here, 'x' is a mandatory variable whose value will incrementally increase with every build. You can also define the value of 'x' for the next build trigger in case you want to change it.

Click Update Pipeline.
Now, go to Build & Deploy tab of your application, and click Select Material in the CI pipeline.
Choose the git commit you wish to use for building the container image. Click Start Build.
The build will initiate and once it is successful the image tag would reflect at all relevant screens:
- Build History
- Docker Registry
- CD Pipeline (Image Selection)

Build will fail if the resulting image tag has already been built in the past. This means if there is an existing image with tag test-v1.0.0, you cannot build another image having the same tag test-v1.0.0 in the same CI pipeline. This error might occur when you reset the value of the variable x or when you disable/enable the toggle button for Custom image tag pattern.

2. Linked Build Pipeline

If one code is shared across multiple applications, Linked Build Pipeline can be used, and only one image will be built for multiple applications because if there is only one build, it is not advisable to create multiple CI Pipelines.

From the Applications menu, select your application.
On the App Configuration page, select Workflow Editor.
Select + New Workflow.
Select Linked Build Pipeline.
On the Create linked build pipeline screen:
- Search for the application in which the source CI pipeline is present.
- Select the source CI pipeline from the application that you selected above.
- Enter a new name for the linked CI pipeline.
Click Create Linked CI Pipeline.

Thereafter, the source CI pipeline will indicate the number of Linked CI pipelines. Upon clicking it, it will display the child information as shown below. It reveals the applications and environments where Linked CI is used for deployment.

After creating a linked CI pipeline, you can create a CD pipeline.

Linked CI pipelines can't trigger builds. They rely on the source CI pipeline to build images. Trigger a build in the source CI pipeline to see the images available for deployment in the linked CI pipeline's CD stage.

3. Deploy Image from External Service

For CI pipeline, you can receive container images from an external services via webhook API.

You can use Devtron for deployments on Kubernetes while using an external CI tool such as Jenkins or CircleCI. External CI feature can be used when the CI tool is hosted outside the Devtron platform. However, by using an external CI, you will not be able to use some of the Devtron features such as Image scanning and security policies, configuring pre-post CI stages etc.

To configure Git Repository, you can add any Git repository account (e.g., dummy account) and click Next.
To configure the Container Registry and Container Repository, you can leave the fields blank or simply add any test repository and click Save & Next.
On the Workflow Editor page, click New Workflow and select Deploy image from external service.
On the Deploy image from external source page, provide the information in the following fields:

Click Create Pipeline. A new CI pipeline will be created for the external source. To get the webhook URL and JSON sample payload to be used in external CI pipeline, click Show webhook details.

On the Webhook Details page, you have to authenticate via API token to allow requests from an external service (e.g. Jenkins or CircleCI).
For authentication, only users with super-admin permissions can select or generate an API token:
- Or use Auto-generate token to generate the API token with the required permissions. Make sure to enter the token name in the Token name field.
To allow requests from the external source, you can request the API by using:
- Webhook URL
- cURL Request

Webhook URL

HTTP Method: POST

API Endpoint: https://{domain-name}/orchestrator/webhook/ext-ci/{pipeline-id}

JSON Payload:

You can also select metadata to send to Devtron. Sample JSON will be generated accordingly. You can send the Payload script to your CI tools such as Jenkins and Devtron will receive the build image every time the CI pipeline is triggered or you can use the Webhook URL, which will build an image every time CI pipeline is triggered using Devtron Dashboard.

Sample cURL Request

Response Codes

Integrate with External Sources - Jenkins or CircleCI

On the Jenkins dashboard, select the Jenkins job which you want to integrate with the Devtron dashboard.
Go to the Configuration > Build Steps, click Add build step, and then click Execute Shell.

Enter the cURL request command.
Make sure to enter the API token and dockerImage in your cURL command and click Save.

Now, you can access the images on the Devtron dashboard and deploy manually. In case, if you select Automatic deployment option, then your application will be deployed automatically everytime a new image is received.

Similarly, you can also integrate with external source such as CircleCI by:

Select the job on the CircleCI dashboard and click Configuration File.
On the respective job, enter the cURL command and update the API token and dockerImage in your cURL command.

Updating CI Pipeline

You can update the configurations of an existing CI Pipeline except for the pipeline's name. To update a pipeline, select your CI pipeline. In the Edit build pipeline window, edit the required stages and select Update Pipeline.

Deleting CI Pipeline

You can only delete a CI pipeline if there is no CD pipeline created in your workflow.

To delete a CI pipeline, go to App Configurations > Workflow Editor and select Delete Pipeline.

Extras

Configuring Webhook

For GitHub

Go to the Settings page of your repository and select Webhooks.
Select Add webhook.
In the Payload URL field, enter the Webhook URL that you get on selecting the source type as "Pull Request" or "Tag Creation" in Devtron the dashboard.
Change the Content-type to application/json.
In the Secret field, enter the secret from Devtron the dashboard when you select the source type as "Pull Request" or "Tag Creation".

Under Which events would you like to trigger this webhook?, select Let me select individual events. to trigger the webhook to build CI Pipeline.
Select Branch or tag creation and Pull Requests.
Select Add webhook.

For Bitbucket Cloud

Go to the Repository settings page of your Bitbucket repository.
Select Webhooks and then select Add webhook.

Enter a Title for the webhook.
In the URL field, enter the Webhook URL that you get on selecting the source type as "Pull Request" or "Tag Creation" in the Devtron dashboard.
Select the event triggers for which you want to trigger the webhook.
Select Save to save your configurations.

CI Pipeline (Legacy)

Each of these methods have different use-cases which can be used according to the needs of the organization. Let’s begin with Continuous Integration.

A. Continuous Integration

Click on Continuous Integration, a prompt comes up in which we need to provide our custom configurations. Below is the description of some configurations which are required.

Key

Description

Pipeline Name

Name of the pipeline

Pipeline Execution (Advanced)

Select from automatic or manual execution depending upon your use-case

Source Type

Select the source through which the CI Pipeline will be triggered

Stages (Advanced)

1.Pre-build Stages- Scripts to be executed before building an image. 2.Docker build Stages- Provide a new argument and override an old argument in key-value pair. 3. Post-build Stages- Scripts to be executed after building image

Scan for vulnerabilities (Advanced)

It will scan your image and find if any vulnerabilities present

[Note] Options such as pipeline execution, stages and scan for vulnerabilities, will be visible after clicking on advanced options present in the bottom left corner.

I. Pipeline Name

Pipeline name is an auto-generated name which can also be renamed by clicking on Advanced options.

II. Pipeline Execution

You can select the method you want to execute the pipeline. By default the value is automatic. In this case it will get automatically triggered if any changes are made to the respective git repository. You can set it to manual if you want to trigger the pipeline manually.

III. Source Type

In source type, we can observe that we have three types of mechanisms which can be used for building your CI Pipeline. In the drop-down you can observe we have Branch Fixed, Pull Request and Tag Creation.

i) Branch Fixed

If you select the Branch Fixed as your source type for building CI Pipeline, then you need to provide the corresponding Branch Name.

Branch Name is the name of the corresponding branch (eg. main or master, or any other branch)

ii) Pull Request

If you select the Pull Request option, you can configure the CI Pipeline using the generated PR. For this mechanism you need to configure a webhook for the repository added in the Git Material.

Prerequisites for Pull Request

If using GitHub - To use this mechanism, as stated above you need to create a webhook for the corresponding repository of your Git Provider. In Github to create a webhook for the repository -

Go to settings of that particular repository
Click on webhook section under options tab
In the Payload URL section, please copy paste the Webhook URL which can be found at Devtron Dashboard when you select source type as Pull Request as seen in above image.
Change content type to - application/json
Copy paste the Secret as well from the Dashboard when you select the source type as Pull Request

Now, scroll down and select the custom events for which you want to trigger the webhook to build CI Pipeline -

Check the radio button for Let me select individual events
Then, check the Branch or Tag Creation and Pull Request radio buttons under the individual events as mentioned in image below.

[Note] If you select Branch or Tag Creation, it will work for the Tag Creation mechanism as well.

After selecting the respective options, click on the generate the webhook button to create a webhook for your respective repository.

If using Bitbucket Cloud - If you are using Bitbucket cloud as your git provider, you need to create a webhook for that as we created for Github in the above section. Follow the steps to create webhook -

Go to Repository Settings on left sidebar of repository window
Click on Webhooks and then click on Add webhook as shown in the image.

Give any appropriate title as per your choice and then copy-paste the url which you can get from Devtron Dashboard when you select Pull Request as source type in case of Bitbucket Cloud as Git Host.
Check the Pull Request events for which you want to trigger the webhook and then save the configurations.

Filters

Now, coming back to the Pull Request mechanism, you can observe we have the option to add filters. In a single repository we have multiple PRs generated, so to have the exact PR for which you want to build the CI Pipeline, we have this feature of filters.

You can add a few filters which can be seen in the dropdown to sort the exact PR which you want to use for building the pipeline.

Below are the details of different filters which you can use as per your requirement. Please select any of the filters and pass the value in regex format as one has already given for example and then click on Create Pipeline.

Key

Description

Source branch name

Branch from which the Pull Request is generated.

Target branch name

Branch to which the Pull request will be merged.

Author

The one who created the Pull Request.

Title

Title provided to the Pull Request.

State

It shows the state of PR and as of now it is fixed to Open which cannot be changed.

iii) Tag Creation

The third option i.e, Tag Creation. In this mechanism you need to provide the tag name or author to specify the exact tag for which you want to build the CI Pipeline. To work with this feature as well, you need to configure the webhook for either Github or Bitbucket as we did in the previous mechanism i.e, Pull Request.

In this process as well you can find the option to filter the specific tags with certain filter parameters. Select the appropriate filter as per your requirement and pass the value in form of regex, one of the examples is already given.

Filter

Description

Author

The one who created the tag.

Tag name

Name of the tag for which the webhook will be triggered.

Select the appropriate filter and pass the value in the form of regex and then click on Create Pipeline.

Advanced Options

When you click on the advanced options button which can be seen at the bottom-left of the screen, you can see some more configuration options which includes pipeline execution, stages and scan for vulnerabilities.

Stages:

There are 3 dropdowns given below:

Pre-build
Docker build
Post-build

(a) Pre-build

This section is used for those steps which you want to execute before building the Docker image. To add a Pre-build stage, click on Add Stage and provide a name to your pre-stage and write your script as per your requirement. These stages will run in sequence before the docker image is built. Optionally, you can also provide the path of the directory where the output of the script will be stored locally.

You can add one or more than one stage in a CI Pipeline.

(b) Docker build

Though we have the option available in the Docker build configuration section to add arguments in key-value pairs for the docker build image. But one can also provide docker build arguments here as well. This is useful, in case you want to override them or want to add new arguments to build your docker image.

(c) Post-build

The post-build stage is similar to the pre-build stage. The difference between the post-build stage and the pre-build stage is that the post-build will run when your CI pipeline will be executed successfully.

Adding a post-build stage is similar to adding a pre-build stage. Click on Add Stage and provide a name to your post-stage. Here you can write your script as per your requirement, which will run in sequence after the docker image is built. You can also provide the path of the directory in which the output of the script will be stored in the Remote Directory column. And this is optional to fill because many times you run scripts that do not provide any output.

NOTE:

(a) You can provide pre-build and post-build stages via the Devtron tool’s console or can also provide these details by creating a file devtron-ci.yaml inside your repository. There is a pre-defined format to write this file. And we will run these stages using this YAML file. You can also provide some stages on the Devtron tool’s console and some stages in the devtron-ci.yaml file. But stages defined through the Devtron dashboard are first executed then the stages defined in the devtron-ci.yaml file.

(b) The total timeout for the execution of the CI pipeline is by default set as 3600 seconds. This default timeout is configurable according to the use-case. The timeout can be edited in the configmap of the orchestrator service in the env variable env:"DEFAULT_TIMEOUT" envDefault:"3600"

Scan for vulnerabilities

Scan for vulnerabilities adds a security feature to your application. If you enable this option, your code will be scanned for any vulnerabilities present in your code. And you will be informed about these vulnerabilities. For more details please check doc

You have provided all the details required to create a CI pipeline, now click on Create Pipeline.

Update CI Pipeline

You can also update any configuration of an already created CI Pipeline, except the pipeline name. The pipeline name can not be edited.

Click on your CI pipeline, to update your CI Pipeline. A window will be popped up with all the details of the current pipeline.

Make your changes and click on Update Pipeline at the bottom to update your Pipeline.

Delete CI Pipeline

You can only delete CI pipeline if you have no CD pipeline created in your workflow.

To delete a CI pipeline, go to the App Configurations and then click on Workflow editor

Click on Delete Pipeline at the bottom to delete the CD pipeline

Automated Test suite integration in the CI step using devtron-ci.yaml

The test cases given in the script will run before the test cases given in the devtron.ci.yaml

Field

Description

version

specify the version of yaml

appliesTo

applies the changes to a specified branch

type

branch type on which changes are to be applied, it can be BRANCH_FIXED or TAG_PATTERN

value

branch name on which changes are to be applied, it can take a value as the name of branch (“master”) or as a regular expression ("%d.%d.%d-rc")

script

A script which you want to execute, you can also execute the docker commands here

beforeDockerBuildStages

script to run before the docker build step

afterDockerBuildStages

script to run after the docker build step

outputLocation

The location where you want to see the output of the report of Test cases

B. Linked CI Pipeline

If one code is shared across multiple applications, Linked CI Pipeline can be used, and only one image will be built for multiple applications because if there is only one build, it is not advisable to create multiple CI Pipelines.

To create a Linked CI Pipeline, please follow the steps mentioned below :

Click on + New Build Pipeline button.

Select Linked CI Pipeline.

Select the application in which the source CI pipeline is present.
Select the source CI pipeline.
Provide a name for linked CI pipeline.
Click on Create Linked CI Pipeline button to create linked CI pipeline.

After creating a linked CI pipeline, you can create a CD pipeline. You cannot trigger build from linked CI pipeline, it can be triggered only from source CI pipeline. Initially you will not see any images to deploy in CD pipeline created from linked CI pipeline. Trigger build in source CI pipeline to see the images in CD pipeline of linked CI pipeline. After this, whenever you trigger build in source CI pipeline, the build images will be listed in CD pipeline of linked CI pipeline too.

C. Incoming Webhook

You can use Devtron for deployments on Kubernetes while using your own CI tool such as Jenkins. External CI features can be used for cases where the CI tool is hosted outside the Devtron platform.

You can send the ‘Payload script’ to your CI tools such as Jenkins and Devtron will receive the build image every time the CI Service is triggered or you can use the Webhook URL which will build an image every time CI Service is triggered using Devtron Dashboard.

Key

Description

Pipeline Name

Name of the pipeline

Source Type

‘Branch Fixed’ or ‘Tag Regex’

Branch Name

Name of the branch

CD Pipeline

After your CI pipeline is ready, you can start building your CD pipeline. Devtron enables you to design your CD pipeline in a way that fully automates your deployments. Images from CI stage can be deployed to one or more environments through dedicated CD pipelines.

Creating CD Pipeline

Click the '+' sign on CI Pipeline to attach a CD Pipeline to it.

A basic Create deployment pipeline window will pop up.

Here, you get three sections:

Deploy to Environment

This section expects four inputs from you:

Setting

Description

Options

Environment

Select the environment where you want to deploy your application

(List of available environments)

Namespace

Automatically populated based on the selected environment

Not Applicable

Trigger

When to execute the deployment pipeline

Automatic: Deployment triggers automatically when a new image completes the previous stage (build pipeline or another deployment pipeline) Manual: Deployment is not initiated automatically. You can trigger deployment with a desired image.

Deployment Approach

How to deploy the application

Deployment Strategy

Advanced Options

This option is available at the bottom of the Create deployment pipeline window.

Now, the window will have 3 distinct tabs, and you will see the following additions:

You can create or edit a deployment strategy in Advanced Options. Remember, only the default strategy will be used for deployment, so use the SET DEFAULT button to mark your preferred strategy as default after creating it.

Pre-Deployment Stage

If your deployment requires prior actions like DB migration, code quality check (QC), etc., you can use the Pre-deployment stage to configure such tasks.

Tasks

Here you can add one or more tasks. The tasks can be re-arranged using drag-and-drop and they will be executed sequentially.

Trigger Pre-Deployment Stage

ConfigMaps & Secrets

Prerequisites

If you want to use some configuration files and secrets in pre-deployment stages or post-deployment stages, then you can use the ConfigMaps & Secrets options. You will get them as a drop-down in the pre-deployment stage.

Execute tasks in application environment

These Pre-deployment CD / Post-deployment CD pods can be created in your deployment cluster or the devtron build cluster. If your scripts/tasks has some dependency on the deployment environment, you may run these pods in the deployment cluster. Thus, your scripts (if any) can interact with the cluster services that may not be publicly exposed.

Some tasks require extra permissions for the node where Devtron is installed. However, if the node already has the necessary permissions for deploying applications, there is no need to assign them again. Instead, you can enable the Execute tasks in application environment option for the pre-CD or post-CD steps. By default, this option is disabled.

To enable the Execute tasks in application environment option, follow these steps:

Go to the chart store and search for the devtron-in-clustercd chart.
Configure the chart according to your requirements and deploy it in the target cluster.
After the deployment, edit the devtron-cm configmap and add the following key-value pair:
```
ORCH_HOST: <host_url>/orchestrator/webhook/msg/nats

Example:

ORCH_HOST: http://xyz.devtron.com/orchestrator/webhook/msg/nats
```
ORCH_HOST value should be same as of CD_EXTERNAL_LISTENER_URL value which is passed in values.yaml.
Delete the Devtron pod using the following command:
```
kubectl delete pod -l app=devtron -n devtroncd
```
Again navigate to the chart store and search for the "migration-incluster-cd" chart.
Edit the cluster-name and secret name values within the chart. The cluster name refers to the name used when adding the cluster in the global configuration and for which you are going to enable Execute tasks in application environment option.
Deploy the chart in any environment within the Devtron cluster. Now you should be able to enable Execute tasks in application environment option for an environment of target cluster.

Deployment Stage

Pipeline Name

Pipeline name will be auto-generated; however, you are free to modify the name as per your requirement.

If you want only approved images to be eligible for deployment, enable the Manual approval for deployment option in the respective deployment pipeline. By doing so, unapproved images would be prevented from being deployed for that deployment pipeline.

Currently, only super-admins can enable or disable this option.

Users can also specify the number of approvals required for each deployment, where the permissible limit ranges from one approval (minimum) to six approvals (maximum). In other words, if the image doesn't get the specified number of approvals, it will not be eligible for deployment

To enable manual approval for deployment, follow these steps:

Click the deployment pipeline for which you want to enable manual approval.
Turn on the ‘Manual approval for deployment’ toggle button.
Select the number of approvals required for each deployment.

Custom Image Tag Pattern

Enable the toggle button as shown below.
Click the edit icon.
You can write an alphanumeric pattern for your image tag, e.g., prod-v1.0.{x}. Here, 'x' is a mandatory variable whose value will incrementally increase with every pre or post deployment trigger (that option is also available to you). You can also define the value of 'x' for the next trigger in case you want to change it.

Ensure your custom tag do not start or end with a period (.) or comma (,)

Click Update Pipeline.

Pull Container Image with Image Digest

Therefore, to eliminate the possibility of pulling an unintended image, Devtron offers the option to pull container images using digest and image tag.

An image digest is a unique and immutable SHA-256 string returned by the container registry when you push an image. So the image referenced by the digest will never change.

Who Can Perform This Action?

Post-Deployment Stage

If you need to run any actions for e.g., closure of Jira ticket, load testing or performance testing, you can configure such actions in the post-deployment stages.

Post-deployment stages are similar to pre-deployment stages. The difference is, pre-deployment executes before the deployment, while post-deployment occurs after.

Updating CD Pipeline

You can update the deployment stages and the deployment strategy of the CD Pipeline whenever you require it. However, you cannot change the name of a CD Pipeline or its Deployment Environment. If you want a new CD pipeline for the same environment, first delete the previous CD pipeline.

To update a CD Pipeline, go to the App Configurations section, Click on Workflow editor and then click on the CD Pipeline you want to Update.

Make changes as needed and click on Update Pipeline to update this CD Pipeline.

Deleting CD Pipeline

If you no longer require the CD Pipeline, you can also delete the Pipeline.

To delete a CD Pipeline, go to the App Configurations and then click on the Workflow editor. Now click on the pipeline you wish to delete. A pop-up having the CD details will appear. Verify the name and the details to ensure that you are not accidentally deleting the wrong CD pipeline and then click Delete Pipeline to delete it.

Deleting a CD pipeline also deletes all the K8s resources associated with it and will bring a disruption in the deployed micro-service. Before deleting a CD pipeline, please ensure that the associated resources are not being used in any production workload.

Extras

Deployment Strategies

A deployment strategy is a method of updating, downgrading, or creating new versions of an application. The options you see under deployment strategy depend on the selected chart type (see fig 2). Below are some deployment configuration-based strategies.

Blue-Green Strategy

Blue-green deployments involve running two versions of an application at the same time and moving traffic from the in-production version (the green version) to the newer version (the blue version).

blueGreen:
  autoPromotionSeconds: 30
  scaleDownDelaySeconds: 30
  previewReplicaCount: 1
  autoPromotionEnabled: false

Key

Description

autoPromotionSeconds

It will make the rollout automatically promote the new ReplicaSet to active Service after this time has passed

scaleDownDelaySeconds

It is used to delay scaling down the old ReplicaSet after the active Service is switched to the new ReplicaSet

previewReplicaCount

It will indicate the number of replicas that the new version of an application should run

autoPromotionEnabled

It will make the rollout automatically promote the new ReplicaSet to the active service

Rolling Strategy

A rolling deployment slowly replaces instances of the previous version of an application with instances of the new version of the application. Rolling deployment typically waits for new pods to become ready via a readiness check before scaling down the old components. If a significant issue occurs, the rolling deployment can be aborted.

rolling:
  maxSurge: "25%"
  maxUnavailable: 1

Key

Description

maxSurge

No. of replicas allowed above the scheduled quantity

maxUnavailable

Maximum number of pods allowed to be unavailable

Canary Strategy

Canary deployments are a pattern for rolling out releases to a subset of users or servers. The idea is to first deploy the change to a small subset of servers, test it, and then roll the change out to the rest of the servers. The canary deployment serves as an early warning indicator with less impact on downtime: if the canary deployment fails, the rest of the servers aren't impacted.

canary:
  maxSurge: "25%"
  maxUnavailable: 1
  steps:
    - setWeight: 25
    - pause:
        duration: 15 # 1 min
    - setWeight: 50
    - pause:
        duration: 15 # 1 min
    - setWeight: 75
    - pause:
        duration: 15 # 1 min

Key

Description

maxSurge

It defines the maximum number of replicas the rollout can create to move to the correct ratio set by the last setWeight

maxUnavailable

The maximum number of pods that can be unavailable during the update

setWeight

It is the required percent of pods to move to the next step

duration

It is used to set the duration to wait to move to the next step

Recreate Strategy

The recreate strategy is a dummy deployment that consists of shutting down version 'A' and then deploying version 'B' after version 'A' is turned off.

A recreate deployment incurs downtime because, for a brief period, no instances of your application are running. However, your old code and new code do not run at the same time. It terminates the old version and releases the new one.

recreate:

Unlike other strategies mentioned above, 'Recreate' strategy doesn't contain keys for you to configure.

Creating Sequential Pipelines

Devtron supports attaching multiple deployment pipelines to a single build pipeline, in its workflow editor. This feature lets you deploy an image first to stage, run tests and then deploy the same image to production.

Please follow the steps mentioned below to create sequential pipelines:

After creating CI/build pipeline, create a CD pipeline by clicking on the + sign on CI pipeline and configure the CD pipeline as per your requirements.
To add another CD Pipeline sequentially after previous one, again click on + sign on the last CD pipeline.
Similarly, you can add multiple CD pipelines by clicking + sign of the last CD pipeline, each deploying in different environments.

Scoped Variables

Introduction

In any piece of software or code, variables are used for holding data such as numbers or strings. Variables are created by declaring them, which involves specifying the variable's name and type, followed by assigning it a value.

Devtron offers super-admins the capability to define scoped variables (key-value pairs). It means, while the key remains the same, its value may change depending on the following context:

Global: Variable value will be universally same throughout Devtron.

Advantages of using scoped variables

Reduces repeatability: Configuration management team can centrally maintain the static data.
Simplifies bulk edits: All the places that use a variable get updated when you change the value of the variable.
Keeps data secure: You can decide the exposure of a variable's value to prevent misuse or leakage of sensitive data.

How to Define a Scoped Variable

On Devtron, a super-admin can download a YAML template. It will contain a schema for defining the variables.

Download the Template

From the left sidebar, go to Global Configurations → Scoped Variables
Click Download template.
Open the downloaded template using any code editor (say VS Code).

Enter the Values

The YAML file contains key-value pairs that follow the below schema:

Field

Type

Description

apiVersion

string

The API version of the resource (comes pre-filled)

kind

string

The kind of resource (i.e. Variable, comes pre-filled)

spec

object

The complete specification object containing all the variables

spec.name

string

Unique name of the variable, e.g. DB_URL

spec.shortDescription

string

A short description of the variable (up to 120 characters)

spec.notes

string

Additional details about the variable (will not be shown on UI)

spec.isSensitive

boolean

Whether the variable value is confidential (will not be shown on UI if true)

spec.values

array

The complete values object containing all the variable values as per context

The spec.values array further contains the following elements:

Field

Type

Description

category

string

The context, e.g., Global, Cluster, Application, Env, ApplicationEnv

value

string

The value of the variable

selectors

object

A set of selectors that restrict the scope of the variable

selectors.attributeSelectors

object

A map of attribute selectors to values

selectors.attributeSelectors.<selector_key>

string

The key of the attribute selector, e.g., ApplicationName, EnvName, ClusterName

selectors.attributeSelectors.<selector_value>

string

The value of the attribute selector

Here's a truncated template containing the specification of two variables for your understanding:

apiVersion: devtron.ai/v1beta1
kind: Variable
spec:

# First example of a variable
  - name: DB_URL
    shortDescription: My application's customers are stored
    notes: The DB is a MySQL DB running version 7.0. The DB contains confidential
      information.
    isSensitive: true
    values:
      - category: Global
        value: mysql.example.com

# Second example of a variable
  - name: DB_Name
    shortDescription: My database name to recognize the DB
    notes: NA
    isSensitive: false
    values:
      - category: Global
        value: Devtron
      - category: ApplicationEnv
        value: app1-p
        selectors:
          attributeSelectors:
            ApplicationName: MyFirstApplication
            EnvName: prod

Upload the Template

Once you save the YAML file, go back to the screen where you downloaded the template.
Use the file uploader utility to upload your YAML file.
The content of the file will be uploaded for you to review and edit. Click Review Changes.
You may check the changes between the last saved file and the current one before clicking Save.

How to Edit an Existing Scoped Variable

Only a super-admin can edit existing scoped variables.

Option 1: Directly edit using the UI

Option 2: Reupload the updated YAML file

Reuploading the YAML file will replace the previous file, so any variable that existed in the previous file but not in the latest one will be lost

How to Use a Scoped Variable

Once a variable is defined, it can be used by your authorized users on Devtron. A scoped variable widget would appear only on the screens that support its usage.

Workflow Editor → Edit build pipeline → Pre-build stage (tab)
Workflow Editor → Edit build pipeline → Post-build stage (tab)
Workflow Editor → Edit deployment pipeline → Post-Deployment stage (tab)
Workflow Editor → Edit deployment pipeline → Post-Deployment stage (tab)
Deployment Template
ConfigMaps
Secrets

Upon clicking on the widget, a list of variables will be visible.

Use the copy button to copy a relevant variable of your choice.

It would appear in the following format upon pasting it within an input field: @{{variable-name}}

Order of Precedence

When multiple values are associated with a scoped variable, the precedence order is as follows, with the highest priority at the top:

Global

Example

Environment + App: This is the most specific scope, and it will take precedence over all other scopes. For example, the value of DB name variable for the app1 application in the prod environment would be app1-p, even though there is a global DB name variable set to Devtron. If a variable value for this scope is not defined, the App scope will be checked.
App: This is the next most specific scope, and it will take precedence over the Environment, Cluster, and Global scopes. For example, the value of DB name variable for the app1 application would be project-tahiti, even though the value of DB name exists in lower scopes. If a variable value for this scope is not defined, the Environment scope will be checked.
Environment: This is the next most specific scope, and it will take precedence over the Cluster and Global scopes. For example, the value of DB name variable in the prod environment would be devtron-prod, even though the value of DB name exists in lower scopes. If a variable value for this scope is not defined, the Cluster scope will be checked.
Cluster: This is the next most specific scope, and it will take precedence over the Global scope. For example, the value of DB name variable in the gcp-gke cluster would be Devtron-gcp, even though there is a global DB name variable set to Devtron-gcp. If a variable value for this scope is not defined, the Global scope will be checked.
Global: This is the least specific scope, and it will only be used if no variable values are found in other higher scopes. The value of DB name variable would be Devtron.

List of Predefined Variables

There are some system variables that exist by default in Devtron that you can readily use if needed:

DEVTRON_IMAGE: Provides full image path of the container image, e.g., gcr.io/k8s-minikube/kicbase:v0.0.39

Currently, these variables do not appear in the scoped variable widget, but you may use them.

Rollout Deployment

The Rollout Deployment chart deploys an advanced version of deployment that supports Blue/Green and Canary deployments. For functioning, it requires a rollout controller to run inside the cluster.

You can define application behavior by providing information in the following sections:

1. Chart version

Key

Descriptions

Chart Version

Select the Chart Version using which you want to deploy the application.

Devtron uses helm charts for the deployments. And we are having multiple chart versions based on features it is supporting with every chart version.

One can see multiple chart version options available in the drop-down. you can select any chart version as per your requirements. By default, the latest version of the helm chart is selected in the chart version option.

Every chart version has its own YAML file. Helm charts are used to provide specifications for your application. To make it easy to use, we have created templates for the YAML file and have added some variables inside the YAML. You can provide or change the values of these variables as per your requirement.

Note: Application Metrics are not supported for the Chart version older than 3.7 version.

2. Basic Configuration

Some of the use-cases which are defined on the Deployment Template (YAML file) may not be applicable to configure for your application. In such cases, you can do the basic deployment configuration for your application on the Basic GUI section instead of configuring the YAML file.

The following fields are provided on the Basic GUI section:

Fields

Description

Port

The internal HTTP port.

HTTP Request Routes

Enable the HTTP Request Routes to define Host and Path. By default, it is in disabled state.

Host: Domain name of the server.

Path: Path of the specific component in the host that the HTTP wants to access.

You can define multiple paths as required by clicking Add path.

CPU

The CPU resource as per the application.

RAM

The RAM resource as per the application.

Environment Variables (Key/Value)

Define key/value by clicking Add variable.

Key: Define the key of the environment.

Value: Define the value of the environment.

You can define multiple env variables by clicking Add variable.

Click Save Changes.

If you want to do additional configurations, then click Advanced (YAML) for modifications.

Note: If you change any values in the Basic GUI, then the corresponding values will be changed in YAML file also.

3. Advanced (YAML)

Container Ports

This defines the ports on which application services will be exposed to other services.

ContainerPort:
  - envoyPort: 8799
    envoyTimeout: 15s
    idleTimeout:
    name: app
    port: 8080
    servicePort: 80
    supportStreaming: true
    useHTTP2: true

Key

Description

envoyPort

envoy port for the container.

envoyTimeout

envoy Timeout for the container,envoy supports a wide range of timeouts that may need to be configured depending on the deployment.By default the envoytimeout is 15s.

idleTimeout

the duration of time that a connection is idle before the connection is terminated.

name

name of the port.

port

port for the container.

servicePort

port of the corresponding kubernetes service.

supportStreaming

Used for high performance protocols like grpc where timeout needs to be disabled.

useHTTP2

Envoy container can accept HTTP2 requests.

EnvVariables

EnvVariables: []

EnvVariables provide run-time information to containers and allow to customize how the application works and the behavior of the applications on the system.

Here we can pass the list of env variables , every record is an object which contain the name of variable along with value.

To set environment variables for the containers that run in the Pod.

Example of EnvVariables

IMP Docker image should have env variables, whatever we want to set.

EnvVariables: 
  - name: HOSTNAME
    value: www.xyz.com
  - name: DB_NAME
    value: mydb
  - name: USER_NAME
    value: xyz

But ConfigMap and Secret are the preferred way to inject env variables. You can create this in App Configuration Section.

ConfigMap

It is a centralized storage, specific to k8s namespace where key-value pairs are stored in plain text.

Secret

It is a centralized storage, specific to k8s namespace where we can store the key-value pairs in plain text as well as in encrypted(Base64) form.

IMP All key-values of Secret and CofigMap will reflect to your application.

Liveness Probe

If this check fails, kubernetes restarts the pod. This should return error code in case of non-recoverable error.

LivenessProbe:
  Path: ""
  port: 8080
  initialDelaySeconds: 20
  periodSeconds: 10
  successThreshold: 1
  timeoutSeconds: 5
  failureThreshold: 3
  command:
    - python
    - /etc/app/healthcheck.py
  httpHeaders:
    - name: Custom-Header
      value: abc
  scheme: ""
  tcp: true

Key

Description

Path

It define the path where the liveness needs to be checked.

initialDelaySeconds

It defines the time to wait before a given container is checked for liveliness.

periodSeconds

It defines how often (in seconds) to perform the liveness probe.

successThreshold

It defines the number of successes required before a given container is said to fulfil the liveness probe.

timeoutSeconds

The maximum time (in seconds) for the probe to complete.

failureThreshold

The number of consecutive failures required to consider the probe as failed.

command

The mentioned command is executed to perform the livenessProbe. If the command returns a non-zero value, it's equivalent to a failed probe.

httpHeaders

Custom headers to set in the request. HTTP allows repeated headers,You can override the default headers by defining .httpHeaders for the probe.

scheme

Scheme to use for connecting to the host (HTTP or HTTPS). Defaults to HTTP.

tcp

The kubelet will attempt to open a socket to your container on the specified port. If it can establish a connection, the container is considered healthy.

MaxUnavailable

  MaxUnavailable: 0

MaxSurge

MaxSurge: 1

Min Ready Seconds

MinReadySeconds: 60

Readiness Probe

If this check fails, kubernetes stops sending traffic to the application. This should return error code in case of errors which can be recovered from if traffic is stopped.

ReadinessProbe:
  Path: ""
  port: 8080
  initialDelaySeconds: 20
  periodSeconds: 10
  successThreshold: 1
  timeoutSeconds: 5
  failureThreshold: 3
  command:
    - python
    - /etc/app/healthcheck.py
  httpHeaders:
    - name: Custom-Header
      value: abc
  scheme: ""
  tcp: true

Key

Description

Path

It define the path where the readiness needs to be checked.

initialDelaySeconds

It defines the time to wait before a given container is checked for readiness.

periodSeconds

It defines how often (in seconds) to perform the readiness probe.

successThreshold

It defines the number of successes required before a given container is said to fulfill the readiness probe.

timeoutSeconds

The maximum time (in seconds) for the probe to complete.

failureThreshold

The number of consecutive failures required to consider the probe as failed.

command

The mentioned command is executed to perform the readinessProbe. If the command returns a non-zero value, it's equivalent to a failed probe.

httpHeaders

Custom headers to set in the request. HTTP allows repeated headers,You can override the default headers by defining .httpHeaders for the probe.

scheme

Scheme to use for connecting to the host (HTTP or HTTPS). Defaults to HTTP.

tcp

The kubelet will attempt to open a socket to your container on the specified port. If it can establish a connection, the container is considered healthy.

Startup Probe

Startup Probe in Kubernetes is a type of probe used to determine when a container within a pod is ready to start accepting traffic. It is specifically designed for applications that have a longer startup time.

StartupProbe:
  Path: ""
  port: 8080
  initialDelaySeconds: 20
  periodSeconds: 10
  successThreshold: 1
  timeoutSeconds: 5
  failureThreshold: 3
  httpHeaders:
    - name: Custom-Header
      value: abc
  command:
    - python
    - /etc/app/healthcheck.py
  tcp: false

Key

Description

Path

It define the path where the startup needs to be checked.

initialDelaySeconds

It defines the time to wait before a given container is checked for startup.

periodSeconds

It defines how often (in seconds) to perform the startup probe.

successThreshold

The number of consecutive successful probe results required to mark the container as ready.

timeoutSeconds

The maximum time (in seconds) for the probe to complete.

failureThreshold

The number of consecutive failures required to consider the probe as failed.

command

The mentioned command is executed to perform the startup probe. If the command returns a non-zero value, it's equivalent to a failed probe.

httpHeaders

Custom headers to set in the request. HTTP allows repeated headers,You can override the default headers by defining .httpHeaders for the probe.

tcp

The kubelet will attempt to open a socket to your container on the specified port. If it can establish a connection, the container is considered healthy.

Autoscaling

This is connected to HPA and controls scaling up and down in response to request load.

autoscaling:
  enabled: false
  MinReplicas: 1
  MaxReplicas: 2
  TargetCPUUtilizationPercentage: 90
  TargetMemoryUtilizationPercentage: 80
  extraMetrics: []

Key

Description

enabled

Set true to enable autoscaling else set false.

MinReplicas

Minimum number of replicas allowed for scaling.

MaxReplicas

Maximum number of replicas allowed for scaling.

TargetCPUUtilizationPercentage

The target CPU utilization that is expected for a container.

TargetMemoryUtilizationPercentage

The target memory utilization that is expected for a container.

extraMetrics

Used to give external metrics for autoscaling.

Fullname Override

fullnameOverride: app-name

Image

image:
  pullPolicy: IfNotPresent

Image is used to access images in kubernetes, pullpolicy is used to define the instances calling the image, here the image is pulled when the image is not present,it can also be set as "Always".

serviceAccount

serviceAccount:
  create: false
  name: ""
  annotations: {}

Key

Description

enabled

Determines whether to create a ServiceAccount for pods or not. If set to true, a ServiceAccount will be created.

name

Specifies the name of the ServiceAccount to use.

annotations

Specify annotations for the ServiceAccount.

imagePullSecrets

imagePullSecrets contains the docker credentials that are used for accessing a registry.

imagePullSecrets:
  - regcred

HostAliases

  hostAliases:
  - ip: "192.168.1.10"
    hostnames:
    - "hostname1.example.com"
    - "hostname2.example.com"
  - ip: "192.168.1.11"
    hostnames:
    - "hostname3.example.com"

Ingress

This allows public access to the url. Please ensure you are using the right nginx annotation for nginx class. The default value is nginx.

ingress:
  enabled: false
  # For K8s 1.19 and above use ingressClassName instead of annotation kubernetes.io/ingress.class:
  className: nginx
  annotations: {}
  hosts:
      - host: example1.com
        pathType: "ImplementationSpecific"
        paths:
            - /example
      - host: example2.com
        pathType: "ImplementationSpecific"
        paths:
            - /example2
            - /example2/healthz
  tls: []

Legacy deployment-template ingress format

ingress:
  enabled: false
  # For K8s 1.19 and above use ingressClassName instead of annotation kubernetes.io/ingress.class:
  ingressClassName: nginx-internal
  annotations: {}
  path: ""
  host: ""
  tls: []

Key

Description

enabled

Enable or disable ingress

annotations

To configure some options depending on the Ingress controller

host

Host name

pathType

Path in an Ingress is required to have a corresponding path type. Supported path types are ImplementationSpecific, Exact and Prefix.

path

Path name

tls

It contains security details

Ingress Internal

This allows private access to the url, please ensure you are using right nginx annotation for nginx class, its default value is nginx

ingressInternal:
  enabled: false
  # For K8s 1.19 and above use ingressClassName instead of annotation kubernetes.io/ingress.class:
  ingressClassName: nginx-internal
  annotations: {}
  hosts:
      - host: example1.com
        pathType: "ImplementationSpecific"
        paths:
            - /example
      - host: example2.com
        pathType: "ImplementationSpecific"
        paths:
            - /example2
            - /example2/healthz
  tls: []

Key

Description

enabled

Enable or disable ingress

annotations

To configure some options depending on the Ingress controller

host

Host name

pathType

Path in an Ingress is required to have a corresponding path type. Supported path types are ImplementationSpecific, Exact and Prefix.

path

Path name

pathType

Supported path types are ImplementationSpecific, Exact and Prefix.

tls

It contains security details

Init Containers

initContainers: 
  - reuseContainerImage: true
    securityContext:
      runAsUser: 1000
      runAsGroup: 3000
      fsGroup: 2000
    volumeMounts:
     - mountPath: /etc/ls-oms
       name: ls-oms-cm-vol
   command:
     - flyway
     - -configFiles=/etc/ls-oms/flyway.conf
     - migrate

  - name: nginx
    image: nginx:1.14.2
    securityContext:
      privileged: true
    ports:
    - containerPort: 80
    command: ["/usr/local/bin/nginx"]
    args: ["-g", "daemon off;"]

Pause For Seconds Before Switch Active

pauseForSecondsBeforeSwitchActive: 30

To wait for given period of time before switch active the container.

Resources

These define minimum and maximum RAM and CPU available to the application.

resources:
  limits:
    cpu: "1"
    memory: "200Mi"
  requests:
    cpu: "0.10"
    memory: "100Mi"

Resources are required to set CPU and memory usage.

Limits

Limits make sure a container never goes above a certain value. The container is only allowed to go up to the limit, and then it is restricted.

Requests

Requests are what the container is guaranteed to get.

Service

This defines annotations and the type of service, optionally can define name also.

  service:
    type: ClusterIP
    annotations: {}

Key

Description

type

Select the type of service, default ClusterIP

annotations

Annotations are widely used to attach metadata and configs in Kubernetes.

name

Optional field to assign name to service

loadBalancerSourceRanges

If service type is LoadBalancer, Provide a list of whitelisted IPs CIDR that will be allowed to use the Load Balancer.

Note - If loadBalancerSourceRanges is not set, Kubernetes allows traffic from 0.0.0.0/0 to the LoadBalancer / Node Security Group(s).

Volumes

volumes:
  - name: log-volume
    emptyDir: {}
  - name: logpv
    persistentVolumeClaim:
      claimName: logpvc

It is required when some values need to be read from or written to an external disk.

Volume Mounts

volumeMounts:
  - mountPath: /var/log/nginx/
    name: log-volume 
  - mountPath: /mnt/logs
    name: logpvc
    subPath: employee

It is used to provide mounts to the volume.

Affinity and anti-affinity

Spec:
  Affinity:
    Key:
    Values:

Spec is used to define the desire state of the given container.

Node Affinity allows you to constrain which nodes your pod is eligible to schedule on, based on labels of the node.

Inter-pod affinity allow you to constrain which nodes your pod is eligible to be scheduled based on labels on pods.

Key

Key part of the label for node selection, this should be same as that on node. Please confirm with devops team.

Values

Value part of the label for node selection, this should be same as that on node. Please confirm with devops team.

Tolerations

tolerations:
 - key: "key"
   operator: "Equal"
   value: "value"
   effect: "NoSchedule|PreferNoSchedule|NoExecute(1.6 only)"

Taints are the opposite, they allow a node to repel a set of pods.

A given pod can access the given node and avoid the given taint only if the given pod satisfies a given taint.

Arguments

args:
  enabled: false
  value: []

This is used to give arguments to command.

Command

command:
  enabled: false
  value: []
  workingDir: {}

It contains the commands to run inside the container.

Key

Description

enabled

To enable or disable the command.

value

It contains the commands.

workingDir

It is used to specify the working directory where commands will be executed.

Containers

    containers:
      - name: nginx
        image: nginx:1.14.2
        ports:
        - containerPort: 80
        command: ["/usr/local/bin/nginx"]
        args: ["-g", "daemon off;"]
      - reuseContainerImage: true
        securityContext:
          runAsUser: 1000
          runAsGroup: 3000
          fsGroup: 2000
        volumeMounts:
        - mountPath: /etc/ls-oms
          name: ls-oms-cm-vol
        command:
          - flyway
          - -configFiles=/etc/ls-oms/flyway.conf
          - migrate

Prometheus

  prometheus:
    release: monitoring

It is a kubernetes monitoring tool and the name of the file to be monitored as monitoring in the given case.It describes the state of the prometheus.

rawYaml

rawYaml: 
  - apiVersion: v1
    kind: Service
    metadata:
      name: my-service
    spec:
      selector:
        app: MyApp
      ports:
        - protocol: TCP
          port: 80
          targetPort: 9376
      type: ClusterIP

Accepts an array of Kubernetes objects. You can specify any kubernetes yaml here and it will be applied when your app gets deployed.

Grace Period

GracePeriod: 30

Server

server:
  deployment:
    image_tag: 1-95a53
    image: ""

It is used for providing server configurations.

Deployment

It gives the details for deployment.

Key

Description

image_tag

It is the image tag

image

It is the URL of the image

Service Monitor

servicemonitor:
      enabled: true
      path: /abc
      scheme: 'http'
      interval: 30s
      scrapeTimeout: 20s
      metricRelabelings:
        - sourceLabels: [namespace]
          regex: '(.*)'
          replacement: myapp
          targetLabel: target_namespace

It gives the set of targets to be monitored.

Db Migration Config

dbMigrationConfig:
  enabled: false

It is used to configure database migration.

Istio

istio:
  enable: true

  gateway:
    enabled: true
    labels:
      app: my-gateway
    annotations:
      description: "Istio Gateway for external traffic"
    host: "example.com"
    tls:
      enabled: true
      secretName: my-tls-secret

  virtualService:
    enabled: true
    labels:
      app: my-service
    annotations:
      description: "Istio VirtualService for routing"
    gateways:
      - my-gateway
    hosts:
      - "example.com"
    http:
      - match:
          - uri:
              prefix: /v1
        route:
          - destination:
              host: my-service-v1
              subset: version-1
      - match:
          - uri:
              prefix: /v2
        route:
          - destination:
              host: my-service-v2
              subset: version-2

  destinationRule:
    enabled: true
    labels:
      app: my-service
    annotations:
      description: "Istio DestinationRule for traffic policies"
    subsets:
      - name: version-1
        labels:
          version: "v1"
      - name: version-2
        labels:
          version: "v2"
    trafficPolicy:
      connectionPool:
        tcp:
          maxConnections: 100
      outlierDetection:
        consecutiveErrors: 5
        interval: 30s
        baseEjectionTime: 60s

  peerAuthentication:
    enabled: true
    labels:
      app: my-service
    annotations:
      description: "Istio PeerAuthentication for mutual TLS"
    selector:
      matchLabels:
        version: "v1"
    mtls:
      mode: STRICT
    portLevelMtls:
      8080:
        mode: DISABLE

  requestAuthentication:
    enabled: true
    labels:
      app: my-service
    annotations:
      description: "Istio RequestAuthentication for JWT validation"
    selector:
      matchLabels:
        version: "v1"
    jwtRules:
      - issuer: "issuer-1"
        jwksUri: "https://issuer-1/.well-known/jwks.json"

  authorizationPolicy:
    enabled: true
    labels:
      app: my-service
    annotations:
      description: "Istio AuthorizationPolicy for access control"
    action: ALLOW
    provider:
      name: jwt
      kind: Authorization
    rules:
      - from:
          - source:
              requestPrincipals: ["*"]
        to:
          - operation:
              methods: ["GET"]

Key

Description

istio

Istio enablement. When istio.enable set to true, Istio would be enabled for the specified configurations

authorizationPolicy

It allows you to define access control policies for service-to-service communication.

action

Determines whether to ALLOW or DENY the request based on the defined rules.

provider

Authorization providers are external systems or mechanisms used to make access control decisions.

rules

List of rules defining the authorization policy. Each rule can specify conditions and requirements for allowing or denying access.

destinationRule

It allows for the fine-tuning of traffic policies and load balancing for specific services. You can define subsets of a service and apply different traffic policies to each subset.

subsets

Specifies subsets within the service for routing and load balancing.

trafficPolicy

Policies related to connection pool size, outlier detection, and load balancing.

gateway

Allowing external traffic to enter the service mesh through the specified configurations.

host

The external domain through which traffic will be routed into the service mesh.

tls

Traffic to and from the gateway should be encrypted using TLS.

secretName

Specifies the name of the Kubernetes secret that contains the TLS certificate and private key. The TLS certificate is used for securing the communication between clients and the Istio gateway.

peerAuthentication

It allows you to enforce mutual TLS and control the authentication between services.

mtls

Mutual TLS. Mutual TLS is a security protocol that requires both client and server, to authenticate each other using digital certificates for secure communication.

mode

Mutual TLS mode, specifying how mutual TLS should be applied. Modes include STRICT, PERMISSIVE, and DISABLE.

portLevelMtls

Configures port-specific mTLS settings. Allows for fine-grained control over the application of mutual TLS on specific ports.

selector

Configuration for selecting workloads to apply PeerAuthentication.

requestAuthentication

Defines rules for authenticating incoming requests.

jwtRules

Rules for validating JWTs (JSON Web Tokens). It defines how incoming JWTs should be validated for authentication purposes.

selector

Specifies the conditions under which the RequestAuthentication rules should be applied.

virtualService

Enables the definition of rules for how traffic should be routed to different services within the service mesh.

gateways

Specifies the gateways to which the rules defined in the VirtualService apply.

hosts

List of hosts (domains) to which this VirtualService is applied.

http

Configuration for HTTP routes within the VirtualService. It define routing rules based on HTTP attributes such as URI prefixes, headers, timeouts, and retry policies.

Application Metrics

Application metrics can be enabled to see your application's metrics-CPU Service Monitor usage, Memory Usage, Status, Throughput and Latency.

Deployment Metrics

It gives the realtime metrics of the deployed applications

Key

Description

Deployment Frequency

It shows how often this app is deployed to production

Change Failure Rate

It shows how often the respective pipeline fails.

Mean Lead Time

It shows the average time taken to deliver a change to production.

Mean Time to Recovery

It shows the average time taken to fix a failed pipeline.

Addon features in Deployment Template Chart version 3.9.0

Service Account

serviceAccountName: orchestrator

A service account provides an identity for the processes that run in a Pod.

When you access the cluster, you are authenticated by the API server as a particular User Account. Processes in containers inside pod can also contact the API server. When you are authenticated as a particular Service Account.

When you create a pod, if you do not create a service account, it is automatically assigned the default service account in the namespace.

Pod Disruption Budget

podDisruptionBudget: 
     minAvailable: 1

podDisruptionBudget: 
     maxUnavailable: 50%

You can specify either maxUnavailable or minAvailable in a PodDisruptionBudget and it can be expressed as integers or as a percentage.

Key

Description

minAvailable

Evictions are allowed as long as they leave behind 1 or more healthy pods of the total number of desired replicas.

maxUnavailable

Evictions are allowed as long as at most 1 unhealthy replica among the total number of desired replicas.

Application metrics Envoy Configurations

envoyproxy:
  image: envoyproxy/envoy:v1.14.1
  configMapName: ""
  resources:
    limits:
      cpu: "50m"
      memory: "50Mi"
    requests:
      cpu: "50m"
      memory: "50Mi"

Envoy is attached as a sidecar to the application container to collect metrics like 4XX, 5XX, Throughput and latency. You can now configure the envoy settings such as idleTimeout, resources etc.

Prometheus Rule

prometheusRule:
  enabled: true
  additionalLabels: {}
  namespace: ""
  rules:
    - alert: TooMany500s
      expr: 100 * ( sum( nginx_ingress_controller_requests{status=~"5.+"} ) / sum(nginx_ingress_controller_requests) ) > 5
      for: 1m
      labels:
        severity: critical
      annotations:
        description: Too many 5XXs
        summary: More than 5% of the all requests did return 5XX, this require your attention

Alerting rules allow you to define alert conditions based on Prometheus expressions and to send notifications about firing alerts to an external service.

In this case, Prometheus will check that the alert continues to be active during each evaluation for 1 minute before firing the alert. Elements that are active, but not firing yet, are in the pending state.

Pod Labels

Labels are key/value pairs that are attached to pods. Labels are intended to be used to specify identifying attributes of objects that are meaningful and relevant to users, but do not directly imply semantics to the core system. Labels can be used to organize and to select subsets of objects.

podLabels:
  severity: critical

Pod Annotations

Pod Annotations are widely used to attach metadata and configs in Kubernetes.

podAnnotations:
  fluentbit.io/exclude: "true"

Custom Metrics in HPA

autoscaling:
  enabled: true
  MinReplicas: 1
  MaxReplicas: 2
  TargetCPUUtilizationPercentage: 90
  TargetMemoryUtilizationPercentage: 80
  behavior:
    scaleDown:
      stabilizationWindowSeconds: 300
      policies:
      - type: Percent
        value: 100
        periodSeconds: 15
    scaleUp:
      stabilizationWindowSeconds: 0
      policies:
      - type: Percent
        value: 100
        periodSeconds: 15
      - type: Pods
        value: 4
        periodSeconds: 15
      selectPolicy: Max

HPA, by default is configured to work with CPU and Memory metrics. These metrics are useful for internal cluster sizing, but you might want to configure wider set of metrics like service latency, I/O load etc. The custom metrics in HPA can help you to achieve this.

Wait For Seconds Before Scaling Down

waitForSecondsBeforeScalingDown: 30

Wait for given period of time before scaling down the container.

4. Show Application Metrics

Helm Chart Json Schema Table

Helm Chart json schema is used to validate the deployment template values.

Chart Version

Link

reference-chart_3-12-0

reference-chart_3-11-0

reference-chart_3-10-0

reference-chart_3-9-0

Other Validations in Json Schema

resources.limits.cpu >= resources.requests.cpu
resources.limits.memory >= resources.requests.memory
envoyproxy.resources.limits.cpu >= envoyproxy.resources.requests.cpu
envoyproxy.resources.limits.memory >= envoyproxy.resources.requests.memory

Addon features in Deployment Template Chart version 4.11.0

KEDA Autoscaling

KEDA Helm repo : https://kedacore.github.io/charts

Example for autoscaling with KEDA using Prometheus metrics is given below:

kedaAutoscaling:
  enabled: true
  minReplicaCount: 1
  maxReplicaCount: 2
  idleReplicaCount: 0
  pollingInterval: 30
  advanced:
    restoreToOriginalReplicaCount: true
    horizontalPodAutoscalerConfig:
      behavior:
        scaleDown:
          stabilizationWindowSeconds: 300
          policies:
          - type: Percent
            value: 100
            periodSeconds: 15
  triggers: 
    - type: prometheus
      metadata:
        serverAddress:  http://<prometheus-host>:9090
        metricName: http_request_total
        query: envoy_cluster_upstream_rq{appId="300", cluster_name="300-0", container="envoy",}
        threshold: "50"
  triggerAuthentication:
    enabled: false
    name:
    spec: {}
  authenticationRef: {}

Example for autosccaling with KEDA based on kafka is given below :

kedaAutoscaling:
  enabled: true
  minReplicaCount: 1
  maxReplicaCount: 2
  idleReplicaCount: 0
  pollingInterval: 30
  advanced: {}
  triggers: 
    - type: kafka
      metadata:
        bootstrapServers: b-2.kafka-msk-dev.example.c2.kafka.ap-southeast-1.amazonaws.com:9092,b-3.kafka-msk-dev.example.c2.kafka.ap-southeast-1.amazonaws.com:9092,b-1.kafka-msk-dev.example.c2.kafka.ap-southeast-1.amazonaws.com:9092
        topic: Orders-Service-ESP.info
        lagThreshold: "100"
        consumerGroup: oders-remove-delivered-packages
        allowIdleConsumers: "true"
  triggerAuthentication:
    enabled: true
    name: keda-trigger-auth-kafka-credential
    spec:
      secretTargetRef:
        - parameter: sasl
          name: keda-kafka-secrets
          key: sasl
        - parameter: username
          name: keda-kafka-secrets
          key: username
  authenticationRef: 
    name: keda-trigger-auth-kafka-credential

NetworkPolicy

Kubernetes NetworkPolicies control pod communication by defining rules for incoming and outgoing traffic.

networkPolicy:
  enabled: false
  annotations: {}
  labels: {}
  podSelector:
    matchLabels:
      role: db
  policyTypes:
    - Ingress
    - Egress
  ingress:
    - from:
        - ipBlock:
            cidr: 172.17.0.0/16
            except:
              - 172.17.1.0/24
        - namespaceSelector:
            matchLabels:
              project: myproject
        - podSelector:
            matchLabels:
              role: frontend
      ports:
        - protocol: TCP
          port: 6379
  egress:
    - to:
        - ipBlock:
            cidr: 10.0.0.0/24
      ports:
        - protocol: TCP
          port: 5978

Key

Description

enabled

Enable or disable NetworkPolicy.

annotations

Additional metadata or information associated with the NetworkPolicy.

labels

Labels to apply to the NetworkPolicy.

podSelector

Each NetworkPolicy includes a podSelector which selects the grouping of pods to which the policy applies. The example policy selects pods with the label "role=db". An empty podSelector selects all pods in the namespace.

policyTypes

Each NetworkPolicy includes a policyTypes list which may include either Ingress, Egress, or both.

Ingress

Controls incoming traffic to pods.

Egress

Controls outgoing traffic from pods.

Winter-Soldier

Winter Soldier can be used to

cleans up (delete) Kubernetes resources
reduce workload pods to 0

Given below is template values you can give in winter-soldier:

winterSoldier:
  enabled: false
  apiVersion: pincher.devtron.ai/v1alpha1
  action: sleep
  timeRangesWithZone:
    timeZone: "Asia/Kolkata"
    timeRanges: []
  targetReplicas: []
  fieldSelector: []

Key

values

Description

enabled

false,true

decide the enabling factor

apiVersion

pincher.devtron.ai/v1beta1, pincher.devtron.ai/v1alpha1

specific api version

action

sleep,delete, scale

This specify the action need to perform.

timeRangesWithZone:timeZone

eg:- "Asia/Kolkata","US/Pacific"

timeRangesWithZone:timeRanges

array of [ timeFrom, timeTo, weekdayFrom, weekdayTo]

It use to define time period/range on which the user need to perform the specified action. you can have multiple timeRanges. These settings will take action on Sat and Sun from 00:00 to 23:59:59,

targetReplicas

[n] : n - number of replicas to scale.

These is mandatory field when the action is scale Default value is [].

fieldSelector

- AfterTime(AddTime( ParseTime({{metadata.creationTimestamp}}, '2006-01-02T15:04:05Z'), '5m'), Now())

These value will take a list of methods to select the resources on which we perform specified action .

here is an example,

winterSoldier:
  apiVersion: pincher.devtron.ai/v1alpha1 
  enabled: true
  annotations: {}
  labels: {}
  timeRangesWithZone:
    timeZone: "Asia/Kolkata"
    timeRanges: 
      - timeFrom: 00:00
        timeTo: 23:59:59
        weekdayFrom: Sat
        weekdayTo: Sun
      - timeFrom: 00:00
        timeTo: 08:00
        weekdayFrom: Mon
        weekdayTo: Fri
      - timeFrom: 20:00
        timeTo: 23:59:59
        weekdayFrom: Mon
        weekdayTo: Fri
  action: scale
  targetReplicas: [1,1,1]
  fieldSelector: 
    - AfterTime(AddTime( ParseTime({{metadata.creationTimestamp}}, '2006-01-02T15:04:05Z'), '10h'), Now())

ParseTime - This function can be used to parse time. For eg to parse creationTimestamp use ParseTime({{metadata.creationTimestamp}}, '2006-01-02T15:04:05Z')
AddTime - This can be used to add time. For eg AddTime(ParseTime({{metadata.creationTimestamp}}, '2006-01-02T15:04:05Z'), '-10h') ll add 10h to the time. Use d for day, h for hour, m for minutes and s for seconds. Use negative number to get earlier time.
Now - This can be used to get current time.
CpuToNumber - This can be used to compare CPU. For eg any({{spec.containers.#.resources.requests}}, { MemoryToNumber(.memory) < MemoryToNumber('60Mi')}) will check if any resource.requests is less than 60Mi.

Security Context

A security context defines privilege and access control settings for a Pod or Container.

To add a security context for main container:

containerSecurityContext:
  allowPrivilegeEscalation: false

To add a security context on pod level:

podSecurityContext:
  runAsUser: 1000
  runAsGroup: 3000
  fsGroup: 2000

Topology Spread Constraints

topologySpreadConstraints:
  - maxSkew: 1
    topologyKey: zone
    whenUnsatisfiable: DoNotSchedule
    autoLabelSelector: true
    customLabelSelector: {}

StatefulSets

The StatefulSet chart in Devtron allows you to deploy and manage stateful applications. StatefulSet is a Kubernetes resource that provides guarantees about the ordering and uniqueness of Pods during deployment and scaling.

It supports only ONDELETE and ROLLINGUPDATE deployment strategy.

You can select StatefulSet chart when you want to use only basic use cases which contain the following:

Managing Stateful Applications: StatefulSets are ideal for managing stateful applications, such as databases or distributed systems, that require stable network identities and persistent storage for each Pod.
Ordered Pod Management: StatefulSets ensure ordered and predictable management of Pods by providing each Pod with a unique and stable hostname based on a defined naming convention and ordinal index.
Updating and Scaling Stateful Applications: StatefulSets support updating and scaling stateful applications by creating new versions of the StatefulSet and performing rolling updates or scaling operations in a controlled manner, ensuring minimal disruption to the application.
Persistent Storage: StatefulSets have built-in mechanisms for handling persistent volumes, allowing each Pod to have its own unique volume claim and storage. This ensures data persistence even when Pods are rescheduled or restarted.
Maintaining Pod Identity: StatefulSets guarantee consistent identity for each Pod throughout its lifecycle. This stability is maintained even if the Pods are rescheduled, allowing applications to rely on stable network identities.
Rollback Capability: StatefulSets provide the ability to rollback to a previous version in case the current state of the application is unstable or encounters issues, ensuring a known working state for the application.
Status Monitoring: StatefulSets offer status information that can be used to monitor the deployment, including the current version, number of replicas, and the readiness of each Pod. This helps in tracking the health and progress of the StatefulSet deployment.
Resource Cleanup: StatefulSets allow for easy cleanup of older versions by deleting StatefulSets and their associated Pods and persistent volumes that are no longer needed, ensuring efficient resource utilization.

1. Yaml File

Container Ports

This defines ports on which application services will be exposed to other services

ContainerPort:
  - envoyPort: 8799
    idleTimeout:
    name: app
    port: 8080
    servicePort: 80
    nodePort: 32056
    supportStreaming: true
    useHTTP2: true

Key

Description

envoyPort

envoy port for the container.

idleTimeout

the duration of time that a connection is idle before the connection is terminated.

name

name of the port.

port

port for the container.

servicePort

port of the corresponding kubernetes service.

nodePort

nodeport of the corresponding kubernetes service.

supportStreaming

Used for high performance protocols like grpc where timeout needs to be disabled.

useHTTP2

Envoy container can accept HTTP2 requests.

EnvVariables

EnvVariables: []

EnvVariablesFromSecretKeys

EnvVariablesFromSecretKeys: 
  - name: ENV_NAME
    secretName: SECRET_NAME
    keyName: SECRET_KEY

It is used to get the name of Environment Variable name, Secret name and the Key name from which we are using the value in that corresponding Environment Variable.

EnvVariablesFromConfigMapKeys

EnvVariablesFromConfigMapKeys: 
  - name: ENV_NAME
    configMapName: CONFIG_MAP_NAME
    keyName: CONFIG_MAP_KEY

It is used to get the name of Environment Variable name, Config Map name and the Key name from which we are using the value in that corresponding Environment Variable.

To set environment variables for the containers that run in the Pod.

StatefulSetConfig

These are all the configuration settings for the StatefulSet.

statefulSetConfig:
  labels:
    app: my-statefulset
    environment: production
  annotations:
    example.com/version: "1.0"
  serviceName: "my-statefulset-service"
  podManagementPolicy: "Parallel"
  revisionHistoryLimit: 5
  mountPath: "/data"
  volumeClaimTemplates:
    - apiVersion: v1
      kind: PersistentVolumeClaim
      metadata:
        labels:
          app: my-statefulset
      spec:
        accessModes:
          - ReadWriteOnce
        dataSource:
          kind: Snapshot
          apiGroup: snapshot.storage.k8s.io
          name: my-snapshot
        resources:
          requests:
            storage: 5Gi
          limits:
            storage: 10Gi
        storageClassName: my-storage-class
        selector:
          matchLabels:
            app: my-statefulset
        volumeMode: Filesystem
        volumeName: my-pv
  - apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
      name: pvc-logs
      labels:
        app: myapp
    spec:
      accessModes:
        - ReadWriteMany
      dataSourceRef:
        kind: Secret
        apiGroup: v1
        name: my-secret
      resources:
        requests:
          storage: 5Gi
      storageClassName: my-storage-class
      selector:
        matchExpressions:
          - {key: environment, operator: In, values: [production]}
      volumeMode: Block
      volumeName: my-pv

Mandatoryfields in statefulSetConfig is

statefulSetConfig:
  mountPath: /tmp
  volumeClaimTemplates:
  - spec:
      accessModes: 
        - ReadWriteOnce
      resources: 
        requests:
            storage: 2Gi

Here is an explanation of each field in the statefulSetConfig :

Key

Description

labels

set of key-value pairs used to identify the StatefulSet .

annotations

A map of key-value pairs that are attached to the stateful set as metadata.

serviceName

The name of the Kubernetes Service that the StatefulSet should create.

podManagementPolicy

A policy that determines how Pods are created and deleted by the StatefulSet. In this case, the policy is set to "Parallel", which means that all Pods are created at once.

revisionHistoryLimit

The number of revisions that should be stored for each replica of the StatefulSet.

updateStrategy

The update strategy used by the StatefulSet when rolling out changes.

mountPath

The path where the volume should be mounted in the container.

volumeClaimTemplates: An array of volume claim templates that are used to create persistent volumes for the StatefulSet. Each volume claim template specifies the storage class, access mode, storage size, and other details of the persistent volume.

Key

Description

apiVersion

The API version of the PVC .

kind

The type of object that the PVC is.

metadata

Metadata that is attached to the resource being created.

labels

A set of key-value pairs used to label the object for identification and selection.

spec

The specification of the object, which defines its desired state and behavior.

accessModes

A list of access modes for the PersistentVolumeClaim, such as "ReadWriteOnce" or "ReadWriteMany".

dataSource

A data source used to populate the PersistentVolumeClaim, such as a Snapshot or a StorageClass.

kind

specifies the kind of the snapshot, in this case Snapshot.

apiGroup

specifies the API group of the snapshot API, in this case snapshot.storage.k8s.io.

name

specifies the name of the snapshot, in this case my-snapshot.

dataSourceRef

A reference to a data source used to create the persistent volume. In this case, it's a secret.

updateStrategy

The update strategy used by the StatefulSet when rolling out changes.

resources

The resource requests and limits for the PersistentVolumeClaim, which define the minimum and maximum amount of storage it can use.

requests

The amount of storage requested by the PersistentVolumeClaim.

limits

The maximum amount of storage that the PersistentVolumeClaim can use.

storageClassName

The name of the storage class to use for the persistent volume.

selector

The selector used to match a persistent volume to a persistent volume claim.

matchLabels

a map of key-value pairs to match the labels of the corresponding PersistentVolume.

matchExpressions

A set of requirements that the selected object must meet to be considered a match.

key

The key of the label or annotation to match.

operator

The operator used to compare the key-value pairs (in this case, "In" specifies a set membership test).

values

A list of values that the selected object's label or annotation must match.

volumeMode

The mode of the volume, either "Filesystem" or "Block".

volumeName

The name of the PersistentVolume that is created for the PersistentVolumeClaim.

Liveness Probe

If this check fails, kubernetes restarts the pod. This should return error code in case of non-recoverable error.

LivenessProbe:
  Path: ""
  port: 8080
  initialDelaySeconds: 20
  periodSeconds: 10
  successThreshold: 1
  timeoutSeconds: 5
  failureThreshold: 3
  httpHeaders:
    - name: Custom-Header
      value: abc
  scheme: ""
  tcp: true

Key

Description

Path

It define the path where the liveness needs to be checked.

initialDelaySeconds

It defines the time to wait before a given container is checked for liveliness.

periodSeconds

It defines the time to check a given container for liveness.

successThreshold

It defines the number of successes required before a given container is said to fulfil the liveness probe.

timeoutSeconds

It defines the time for checking timeout.

failureThreshold

It defines the maximum number of failures that are acceptable before a given container is not considered as live.

httpHeaders

Custom headers to set in the request. HTTP allows repeated headers,You can override the default headers by defining .httpHeaders for the probe.

scheme

Scheme to use for connecting to the host (HTTP or HTTPS). Defaults to HTTP.

tcp

The kubelet will attempt to open a socket to your container on the specified port. If it can establish a connection, the container is considered healthy.

MaxUnavailable

  MaxUnavailable: 0

MaxSurge

MaxSurge: 1

Min Ready Seconds

MinReadySeconds: 60

Readiness Probe

If this check fails, kubernetes stops sending traffic to the application. This should return error code in case of errors which can be recovered from if traffic is stopped.

ReadinessProbe:
  Path: ""
  port: 8080
  initialDelaySeconds: 20
  periodSeconds: 10
  successThreshold: 1
  timeoutSeconds: 5
  failureThreshold: 3
  httpHeaders:
    - name: Custom-Header
      value: abc
  scheme: ""
  tcp: true

Key

Description

Path

It define the path where the readiness needs to be checked.

initialDelaySeconds

It defines the time to wait before a given container is checked for readiness.

periodSeconds

It defines the time to check a given container for readiness.

successThreshold

It defines the number of successes required before a given container is said to fulfill the readiness probe.

timeoutSeconds

It defines the time for checking timeout.

failureThreshold

It defines the maximum number of failures that are acceptable before a given container is not considered as ready.

httpHeaders

Custom headers to set in the request. HTTP allows repeated headers,You can override the default headers by defining .httpHeaders for the probe.

scheme

Scheme to use for connecting to the host (HTTP or HTTPS). Defaults to HTTP.

tcp

The kubelet will attempt to open a socket to your container on the specified port. If it can establish a connection, the container is considered healthy.

Ambassador Mappings

You can create ambassador mappings to access your applications from outside the cluster. At its core a Mapping resource maps a resource to a service.

ambassadorMapping:
  ambassadorId: "prod-emissary"
  cors: {}
  enabled: true
  hostname: devtron.example.com
  labels: {}
  prefix: /
  retryPolicy: {}
  rewrite: ""
  tls:
    context: "devtron-tls-context"
    create: false
    hosts: []
    secretName: ""

Key

Description

enabled

Set true to enable ambassador mapping else set false.

ambassadorId

used to specify id for specific ambassador mappings controller.

cors

used to specify cors policy to access host for this mapping.

weight

used to specify weight for canary ambassador mappings.

hostname

used to specify hostname for ambassador mapping.

prefix

used to specify path for ambassador mapping.

labels

used to provide custom labels for ambassador mapping.

retryPolicy

used to specify retry policy for ambassador mapping.

corsPolicy

Provide cors headers on flagger resource.

rewrite

used to specify whether to redirect the path of this mapping and where.

tls

used to create or define ambassador TLSContext resource.

extraSpec

used to provide extra spec values which not present in deployment template for ambassador resource.

Autoscaling

This is connected to HPA and controls scaling up and down in response to request load.

autoscaling:
  enabled: false
  MinReplicas: 1
  MaxReplicas: 2
  TargetCPUUtilizationPercentage: 90
  TargetMemoryUtilizationPercentage: 80
  extraMetrics: []

Key

Description

enabled

Set true to enable autoscaling else set false.

MinReplicas

Minimum number of replicas allowed for scaling.

MaxReplicas

Maximum number of replicas allowed for scaling.

TargetCPUUtilizationPercentage

The target CPU utilization that is expected for a container.

TargetMemoryUtilizationPercentage

The target memory utilization that is expected for a container.

extraMetrics

Used to give external metrics for autoscaling.

Fullname Override

fullnameOverride: app-name

Image

image:
  pullPolicy: IfNotPresent

Image is used to access images in kubernetes, pullpolicy is used to define the instances calling the image, here the image is pulled when the image is not present,it can also be set as "Always".

imagePullSecrets

imagePullSecrets contains the docker credentials that are used for accessing a registry.

imagePullSecrets:
  - regcred

Ingress

This allows public access to the url, please ensure you are using right nginx annotation for nginx class, its default value is nginx

ingress:
  enabled: false
  # For K8s 1.19 and above use ingressClassName instead of annotation kubernetes.io/ingress.class:
  className: nginx
  annotations: {}
  hosts:
      - host: example1.com
        paths:
            - /example
      - host: example2.com
        paths:
            - /example2
            - /example2/healthz
  tls: []

Legacy deployment-template ingress format

ingress:
  enabled: false
  # For K8s 1.19 and above use ingressClassName instead of annotation kubernetes.io/ingress.class:
  ingressClassName: nginx-internal
  annotations: {}
  path: ""
  host: ""
  tls: []

Key

Description

enabled

Enable or disable ingress

annotations

To configure some options depending on the Ingress controller

path

Path name

host

Host name

tls

It contains security details

Ingress Internal

This allows private access to the url, please ensure you are using right nginx annotation for nginx class, its default value is nginx

ingressInternal:
  enabled: false
  # For K8s 1.19 and above use ingressClassName instead of annotation kubernetes.io/ingress.class:
  ingressClassName: nginx-internal
  annotations: {}
  hosts:
      - host: example1.com
        paths:
            - /example
      - host: example2.com
        paths:
            - /example2
            - /example2/healthz
  tls: []

Key

Description

enabled

Enable or disable ingress

annotations

To configure some options depending on the Ingress controller

path

Path name

host

Host name

tls

It contains security details

Init Containers

initContainers: 
  - reuseContainerImage: true
    securityContext:
      runAsUser: 1000
      runAsGroup: 3000
      fsGroup: 2000
    volumeMounts:
     - mountPath: /etc/ls-oms
       name: ls-oms-cm-vol
   command:
     - flyway
     - -configFiles=/etc/ls-oms/flyway.conf
     - migrate

  - name: nginx
    image: nginx:1.14.2
    securityContext:
      privileged: true
    ports:
    - containerPort: 80
    command: ["/usr/local/bin/nginx"]
    args: ["-g", "daemon off;"]

Istio

Istio is a service mesh which simplifies observability, traffic management, security and much more with it's virtual services and gateways.

istio:
  enable: true
  gateway:
    annotations: {}
    enabled: false
    host: example.com
    labels: {}
    tls:
      enabled: false
      secretName: example-tls-secret
  virtualService:
    annotations: {}
    enabled: false
    gateways: []
    hosts: []
    http:
      - corsPolicy:
          allowCredentials: false
          allowHeaders:
            - x-some-header
          allowMethods:
            - GET
          allowOrigin:
            - example.com
          maxAge: 24h
        headers:
          request:
            add:
              x-some-header: value
        match:
          - uri:
              prefix: /v1
          - uri:
              prefix: /v2
        retries:
          attempts: 2
          perTryTimeout: 3s
        rewriteUri: /
        route:
          - destination:
              host: service1
              port: 80
        timeout: 12s
      - route:
          - destination:
              host: service2
    labels: {}

Key

Description

istio

Istio enablement. When istio.enable set to true, Istio would be enabled for the specified configurations

gateway

Allowing external traffic to enter the service mesh through the specified configurations.

host

The external domain through which traffic will be routed into the service mesh.

tls

Traffic to and from the gateway should be encrypted using TLS.

secretName

Specifies the name of the Kubernetes secret that contains the TLS certificate and private key. The TLS certificate is used for securing the communication between clients and the Istio gateway.

virtualService

Enables the definition of rules for how traffic should be routed to different services within the service mesh.

gateways

Specifies the gateways to which the rules defined in the VirtualService apply.

hosts

List of hosts (domains) to which this VirtualService is applied.

http

Configuration for HTTP routes within the VirtualService. It define routing rules based on HTTP attributes such as URI prefixes, headers, timeouts, and retry policies.

corsPolicy

Cross-Origin Resource Sharing (CORS) policy configuration.

headers

Additional headers to be added to the HTTP request.

match

Conditions that need to be satisfied for this route to be used.

uri

This specifies a match condition based on the URI of the incoming request.

prefix

It specifies that the URI should have the specified prefix.

retries

Retry configuration for failed requests.

attempts

It specifies the number of retry attempts for failed requests.

perTryTimeout

sets the timeout for each individual retry attempt.

rewriteUri

Rewrites the URI of the incoming request.

route

List of destination rules for routing traffic.

Pause For Seconds Before Switch Active

pauseForSecondsBeforeSwitchActive: 30

To wait for given period of time before switch active the container.

Resources

These define minimum and maximum RAM and CPU available to the application.

resources:
  limits:
    cpu: "1"
    memory: "200Mi"
  requests:
    cpu: "0.10"
    memory: "100Mi"

Resources are required to set CPU and memory usage.

Limits

Limits make sure a container never goes above a certain value. The container is only allowed to go up to the limit, and then it is restricted.

Requests

Requests are what the container is guaranteed to get.

Service

This defines annotations and the type of service, optionally can define name also.

  service:
    type: ClusterIP
    annotations: {}

Volumes

volumes:
  - name: log-volume
    emptyDir: {}
  - name: logpv
    persistentVolumeClaim:
      claimName: logpvc

It is required when some values need to be read from or written to an external disk.

Volume Mounts

volumeMounts:
  - mountPath: /var/log/nginx/
    name: log-volume 
  - mountPath: /mnt/logs
    name: logpvc
    subPath: employee

It is used to provide mounts to the volume.

Affinity and anti-affinity

Spec:
  Affinity:
    Key:
    Values:

Spec is used to define the desire state of the given container.

Node Affinity allows you to constrain which nodes your pod is eligible to schedule on, based on labels of the node.

Inter-pod affinity allow you to constrain which nodes your pod is eligible to be scheduled based on labels on pods.

Key

Key part of the label for node selection, this should be same as that on node. Please confirm with devops team.

Values

Value part of the label for node selection, this should be same as that on node. Please confirm with devops team.

Tolerations

tolerations:
 - key: "key"
   operator: "Equal"
   value: "value"
   effect: "NoSchedule|PreferNoSchedule|NoExecute(1.6 only)"

Taints are the opposite, they allow a node to repel a set of pods.

A given pod can access the given node and avoid the given taint only if the given pod satisfies a given taint.

Arguments

args:
  enabled: false
  value: []

This is used to give arguments to command.

Command

command:
  enabled: false
  value: []

It contains the commands for the server.

Key

Description

enabled

To enable or disable the command.

value

It contains the commands.

Containers

    containers:
      - name: nginx
        image: nginx:1.14.2
        ports:
        - containerPort: 80
        command: ["/usr/local/bin/nginx"]
        args: ["-g", "daemon off;"]
      - reuseContainerImage: true
        securityContext:
          runAsUser: 1000
          runAsGroup: 3000
          fsGroup: 2000
        volumeMounts:
        - mountPath: /etc/ls-oms
          name: ls-oms-cm-vol
        command:
          - flyway
          - -configFiles=/etc/ls-oms/flyway.conf
          - migrate

Prometheus

  prometheus:
    release: monitoring

It is a kubernetes monitoring tool and the name of the file to be monitored as monitoring in the given case.It describes the state of the prometheus.

rawYaml

rawYaml: 
  - apiVersion: v1
    kind: Service
    metadata:
      name: my-service
    spec:
      selector:
        app: MyApp
      ports:
        - protocol: TCP
          port: 80
          targetPort: 9376
      type: ClusterIP

Accepts an array of Kubernetes objects. You can specify any kubernetes yaml here and it will be applied when your app gets deployed.

Grace Period

GracePeriod: 30

Server

server:
  deployment:
    image_tag: 1-95a53
    image: ""

It is used for providing server configurations.

Deployment

It gives the details for deployment.

Key

Description

image_tag

It is the image tag

image

It is the URL of the image

Service Monitor

servicemonitor:
      enabled: true
      path: /abc
      scheme: 'http'
      interval: 30s
      scrapeTimeout: 20s
      metricRelabelings:
        - sourceLabels: [namespace]
          regex: '(.*)'
          replacement: myapp
          targetLabel: target_namespace

It gives the set of targets to be monitored.

Db Migration Config

dbMigrationConfig:
  enabled: false

It is used to configure database migration.

KEDA Autoscaling

Example for autosccaling with KEDA using Prometheus metrics is given below:

kedaAutoscaling:
  enabled: true
  minReplicaCount: 1
  maxReplicaCount: 2
  idleReplicaCount: 0
  pollingInterval: 30
  advanced:
    restoreToOriginalReplicaCount: true
    horizontalPodAutoscalerConfig:
      behavior:
        scaleDown:
          stabilizationWindowSeconds: 300
          policies:
          - type: Percent
            value: 100
            periodSeconds: 15
  triggers: 
    - type: prometheus
      metadata:
        serverAddress:  http://<prometheus-host>:9090
        metricName: http_request_total
        query: envoy_cluster_upstream_rq{appId="300", cluster_name="300-0", container="envoy",}
        threshold: "50"
  triggerAuthentication:
    enabled: false
    name:
    spec: {}
  authenticationRef: {}

Example for autosccaling with KEDA based on kafka is given below :

kedaAutoscaling:
  enabled: true
  minReplicaCount: 1
  maxReplicaCount: 2
  idleReplicaCount: 0
  pollingInterval: 30
  advanced: {}
  triggers: 
    - type: kafka
      metadata:
        bootstrapServers: b-2.kafka-msk-dev.example.c2.kafka.ap-southeast-1.amazonaws.com:9092,b-3.kafka-msk-dev.example.c2.kafka.ap-southeast-1.amazonaws.com:9092,b-1.kafka-msk-dev.example.c2.kafka.ap-southeast-1.amazonaws.com:9092
        topic: Orders-Service-ESP.info
        lagThreshold: "100"
        consumerGroup: oders-remove-delivered-packages
        allowIdleConsumers: "true"
  triggerAuthentication:
    enabled: true
    name: keda-trigger-auth-kafka-credential
    spec:
      secretTargetRef:
        - parameter: sasl
          name: keda-kafka-secrets
          key: sasl
        - parameter: username
          name: keda-kafka-secrets
          key: username
  authenticationRef: 
    name: keda-trigger-auth-kafka-credential

Winter-Soldier

Winter Soldier can be used to

cleans up (delete) Kubernetes resources
reduce workload pods to 0

Given below is template values you can give in winter-soldier:

winterSoilder:
  enable: false
  apiVersion: pincher.devtron.ai/v1alpha1
  action: sleep
  timeRangesWithZone:
    timeZone: "Asia/Kolkata"
    timeRanges: []
  targetReplicas: []
  fieldSelector: []

Here,

Key

values

Description

enable

false,true

decide the enabling factor

apiVersion

pincher.devtron.ai/v1beta1, pincher.devtron.ai/v1alpha1

specific api version

action

sleep,delete, scale

This specify the action need to perform.

timeRangesWithZone:timeZone

eg:- "Asia/Kolkata","US/Pacific"

timeRangesWithZone:timeRanges

array of [ timeFrom, timeTo, weekdayFrom, weekdayTo]

It use to define time period/range on which the user need to perform the specified action. you can have multiple timeRanges. These settings will take action on Sat and Sun from 00:00 to 23:59:59,

targetReplicas

[n] : n - number of replicas to scale.

These is mandatory field when the action is scale Default value is [].

fieldSelector

- AfterTime(AddTime( ParseTime({{metadata.creationTimestamp}}, '2006-01-02T15:04:05Z'), '5m'), Now())

These value will take a list of methods to select the resources on which we perform specified action .

here is an example,

winterSoilder:
  apiVersion: pincher.devtron.ai/v1alpha1 
  enable: true
  annotations: {}
  labels: {}
  timeRangesWithZone:
    timeZone: "Asia/Kolkata"
    timeRanges: 
      - timeFrom: 00:00
        timeTo: 23:59:59
        weekdayFrom: Sat
        weekdayTo: Sun
      - timeFrom: 00:00
        timeTo: 08:00
        weekdayFrom: Mon
        weekdayTo: Fri
      - timeFrom: 20:00
        timeTo: 23:59:59
        weekdayFrom: Mon
        weekdayTo: Fri
  action: scale
  targetReplicas: [1,1,1]
  fieldSelector: 
    - AfterTime(AddTime( ParseTime({{metadata.creationTimestamp}}, '2006-01-02T15:04:05Z'), '10h'), Now())

ParseTime - This function can be used to parse time. For eg to parse creationTimestamp use ParseTime({{metadata.creationTimestamp}}, '2006-01-02T15:04:05Z')
AddTime - This can be used to add time. For eg AddTime(ParseTime({{metadata.creationTimestamp}}, '2006-01-02T15:04:05Z'), '-10h') ll add 10h to the time. Use d for day, h for hour, m for minutes and s for seconds. Use negative number to get earlier time.
Now - This can be used to get current time.
CpuToNumber - This can be used to compare CPU. For eg any({{spec.containers.#.resources.requests}}, { MemoryToNumber(.memory) < MemoryToNumber('60Mi')}) will check if any resource.requests is less than 60Mi.

Security Context

A security context defines privilege and access control settings for a Pod or Container.

To add a security context for main container:

containerSecurityContext:
  allowPrivilegeEscalation: false

To add a security context on pod level:

podSecurityContext:
  runAsUser: 1000
  runAsGroup: 3000
  fsGroup: 2000

Topology Spread Constraints

topologySpreadConstraints:
  - maxSkew: 1
    topologyKey: zone
    whenUnsatisfiable: DoNotSchedule
    autoLabelSelector: true
    customLabelSelector: {}

Deployment Metrics

It gives the realtime metrics of the deployed applications

Key

Description

Deployment Frequency

It shows how often this app is deployed to production

Change Failure Rate

It shows how often the respective pipeline fails.

Mean Lead Time

It shows the average time taken to deliver a change to production.

Mean Time to Recovery

It shows the average time taken to fix a failed pipeline.

2. Show application metrics

Helm Chart Json Schema

Other Validations in Json Schema

resources.limits.cpu >= resources.requests.cpu
resources.limits.memory >= resources.requests.memory
envoyproxy.resources.limits.cpu >= envoyproxy.resources.requests.cpu
envoyproxy.resources.limits.memory >= envoyproxy.resources.requests.memory