This section includes information about the minimum requirements you need to install and use Devtron.

Devtron is installed over a Kubernetes cluster. Once you create a Kubernetes cluster, Devtron can be installed standalone or along with CI/CD integration:

• Devtron with CI/CD: Devtron installation with the CI/CD integration is used to perform CI/CD, security scanning, GitOps, debugging, and observability.

• Helm Dashboard by Devtron: The Helm Dashboard by Devtron, which is a standalone installation, includes functionalities to deploy, observe, manage, and debug existing Helm applications in multiple clusters. You can also install integrations from Devtron Stack Manager.

In this section, we will cover the basic details on how you can quickly get started with Devtron. First, lets see what are the prerequisite requirements before you install Devtron.

Prerequisites

• Create a Kubernetes cluster, preferably K8s version 1.16 or higher

• Helm Installation

• Recommended Resources

Create a Kubernetes Cluster

You can create any Kubernetes cluster (preferably K8s version 1.16 or higher) for installing Devtron.

You can create a cluster using one of the following cloud providers as per your requirements:

Install Helm

Make sure to install helm.

Recommended Resources

The minimum requirements for installing Helm Dashboard by Devtron and Devtron with CI/CD as per the number of applications you want to manage on Devtron are provided below:

• For configuring small resources (to manage not more than 5 apps on Devtron):

• For configuring medium/larger resources (to manage more than 5 apps on Devtron):

Refer to the Override Configurations section for more information.

Note:

• Please make sure that the recommended resources are available on your Kubernetes cluster before you proceed with Devtron installation.

• It is NOT recommended to use brustable CPU VMs (T series in AWS, B Series in Azure and E2/N1 in GCP) for Devtron installation to experience consistency in performance.

Installation of Devtron

You can install Devtron standalone (Helm Dashboard by Devtron) or along with CI/CD integration. Or, you can upgrade Devtron to the latest version.

Choose one of the options as per your requirements:

Devtron is installed over a Kubernetes cluster. Once you create a Kubernetes cluster, Devtron can be installed standalone or along with CI/CD integration.

Choose one of the options as per your requirements:

In this section, we describe on how you can install Helm Dashboard by Devtron without any integrations. Integrations can be added later using Devtron Stack Manager.

If you want to install Devtron on Minikube, Microk8s, K3s, Kind, refer this section.

Before you begin

Install Helm if you have not installed it.

Add Helm Repo

helm repo add devtron https://helm.devtron.ai

Update Helm Repo

helm repo update devtron

Install Helm Dashboard by Devtron

Note: This installation command will not install CI/CD integration. For CI/CD, refer install Devtron with CI/CD section.

Run the following command to install Helm Dashboard by Devtron:

helm install devtron devtron/devtron-operator \
--create-namespace --namespace devtroncd

Install Multi-Architecture Nodes (ARM and AMD)

To install Devtron on clusters with the multi-architecture nodes (ARM and AMD), append the Devtron installation command with --set installer.arch=multi-arch.

Devtron Dashboard

Run the following command to get the dashboard URL:

kubectl get svc -n devtroncd devtron-service -o jsonpath='{.status.loadBalancer.ingress}'

You will get the result something as shown below:

[test2@server ~]$ kubectl get svc -n devtroncd devtron-service -o jsonpath='{.status.loadBalancer.ingress}'
[map[hostname:aaff16e9760594a92afa0140dbfd99f7-305259315.us-east-1.elb.amazonaws.com]]

The hostname aaff16e9760594a92afa0140dbfd99f7-305259315.us-east-1.elb.amazonaws.com as mentioned above is the Loadbalancer URL where you can access the Devtron dashboard.

You can also do a CNAME entry corresponding to your domain/subdomain to point to this Loadbalancer URL to access it at a custom domain.

Devtron Admin credentials

When you install Devtron for the first time, it creates a default admin user and password (with unrestricted access to Devtron). You can use that credentials to log in as an administrator.

After the initial login, we recommend you set up any SSO service like Google, GitHub, etc., and then add other users (including yourself). Subsequently, all the users can use the same SSO (let's say, GitHub) to log in to Devtron's dashboard.

The section below will help you understand the process of getting the administrator credentials.

For Devtron version v0.6.0 and higher

Username: admin Password: Run the following command to get the admin password:

kubectl -n devtroncd get secret devtron-secret \
-o jsonpath='{.data.ADMIN_PASSWORD}' | base64 -d

kubectl -n devtroncd get secret devtron-secret \
-o jsonpath='{.data.ACD_PASSWORD}' | base64 -d

Note: If you want to uninstall Devtron or clean Devtron helm installer, refer our uninstall Devtron.

Upgrade

To use the CI/CD capabilities with Devtron, you can Install the Devtron with CI/CD or Devtron with CI/CD along with GitOps (Argo CD).

Regular backups for Devtron PostgreSQL and ArgoCD are crucial components of a disaster recovery plan, as they protect against potential data loss due to unforeseen circumstances. This documentation provides instructions on how to take backups of Devtron and store them either on AWS S3 or Azure containers.

1. Go to the devtron chart store and search for devtron-backups chart.

1. Select the devtron-backups and click Configure & Deploy.

2. Now follow either of the options described below according to your Cloud provider.

AWS S3 Backup

To store Devtron backups on AWS S3, please follow these steps:

1. Create an S3 bucket to store the Devtron backup, you can configure the bucket to delete all the objects older than 15/30 days.

2. Create a user with sufficient permissions to push to the S3 bucket created in step 1.

3. Obtain the access key and secret access key for the created user.

4. Configure the devtron-backups chart for AWS S3 by selecting the appropriate options:

1. Deploy the chart, and the Devtron backup will be automatically uploaded to the AWS S3 bucket at the scheduled intervals.

Azure Containers Backup

To store Devtron backups on Azure Containers, please follow these steps:

1. Create a storage account in Azure.

2. Within the storage account, create two containers for the Devtron backup.

3. Navigate to Security + Networking > Access Key section in Azure and copy the Access Key:

1. Configure the devtron-backups chart for Azure Containers by providing the Access Key:

1. Before deploying the backup chart, ensure that AWS.enabled is set to false. This will ensure that Devtron backup will be automatically uploaded to the configured Azure containers on the scheduled intervals.

By following these steps, you can ensure that your Devtron data is securely backed up and protected against any potential data loss, enabling you to recover quickly in case of emergencies.

In this section, we describe the steps in detail on how you can install Devtron with CI/CD by enabling GitOps during the installation.

Before you begin

Install Helm if you have not installed it.

Install Devtron with CI/CD along with GitOps (Argo CD)

Run the following command to install the latest version of Devtron with CI/CD along with GitOps (Argo CD) module:

helm repo add devtron https://helm.devtron.ai

helm repo update devtron

helm install devtron devtron/devtron-operator \
--create-namespace --namespace devtroncd \
--set installer.modules={cicd} \
--set argo-cd.enabled=true

Note: If you want to configure Blob Storage during the installation, refer configure blob storage duing installation.

Install Multi-Architecture Nodes (ARM and AMD)

To install Devtron on clusters with the multi-architecture nodes (ARM and AMD), append the Devtron installation command with --set installer.arch=multi-arch.

Note:

• If you want to install Devtron for production deployments, please refer to our recommended overrides for Devtron Installation.

Configure Blob Storage during Installation

Configuring Blob Storage in your Devtron environment allows you to store build logs and cache. In case, if you do not configure the Blob Storage, then:

• You will not be able to access the build and deployment logs after an hour.

• Build time for commit hash takes longer as cache is not available.

• Artifact reports cannot be generated in pre/post build and deployment stages.

Choose one of the options to configure blob storage:

helm repo add devtron https://helm.devtron.ai 

helm repo update devtron

helm install devtron devtron/devtron-operator \
--create-namespace --namespace devtroncd \
--set installer.modules={cicd} \
--set minio.enabled=true \
--set argo-cd.enabled=true

helm repo add devtron https://helm.devtron.ai

helm repo update devtron

helm install devtron devtron/devtron-operator \
--create-namespace --namespace devtroncd \
--set installer.modules={cicd} \
--set configs.BLOB_STORAGE_PROVIDER=S3 \
--set configs.DEFAULT_CACHE_BUCKET=demo-s3-bucket \
--set configs.DEFAULT_CACHE_BUCKET_REGION=us-east-1 \
--set configs.DEFAULT_BUILD_LOGS_BUCKET=demo-s3-bucket \
--set configs.DEFAULT_CD_LOGS_BUCKET_REGION=us-east-1 \
--set argo-cd.enabled=true

helm repo add devtron https://helm.devtron.ai

helm repo update devtron

helm install devtron devtron/devtron-operator \
--create-namespace --namespace devtroncd \
--set installer.modules={cicd} \
--set configs.BLOB_STORAGE_PROVIDER=S3 \
--set configs.DEFAULT_CACHE_BUCKET=demo-s3-bucket \
--set configs.DEFAULT_CACHE_BUCKET_REGION=us-east-1 \
--set configs.DEFAULT_BUILD_LOGS_BUCKET=demo-s3-bucket \
--set configs.DEFAULT_CD_LOGS_BUCKET_REGION=us-east-1 \
--set secrets.BLOB_STORAGE_S3_ACCESS_KEY=<access-key> \
--set secrets.BLOB_STORAGE_S3_SECRET_KEY=<secret-key> \
--set argo-cd.enabled=true

helm repo add devtron https://helm.devtron.ai

helm repo update devtron

helm install devtron devtron/devtron-operator \
--create-namespace --namespace devtroncd \
--set installer.modules={cicd} \
--set configs.BLOB_STORAGE_PROVIDER=S3 \
--set configs.DEFAULT_CACHE_BUCKET=demo-s3-bucket \
--set configs.DEFAULT_CACHE_BUCKET_REGION=us-east-1 \
--set configs.DEFAULT_BUILD_LOGS_BUCKET=demo-s3-bucket \
--set configs.DEFAULT_CD_LOGS_BUCKET_REGION=us-east-1 \
--set secrets.BLOB_STORAGE_S3_ACCESS_KEY=<access-key> \
--set secrets.BLOB_STORAGE_S3_SECRET_KEY=<secret-key> \
--set configs.BLOB_STORAGE_S3_ENDPOINT=<endpoint> \
--set argo-cd.enabled=true

helm repo add devtron https://helm.devtron.ai

helm repo update devtron

helm install devtron devtron/devtron-operator \
--create-namespace --namespace devtroncd \
--set installer.modules={cicd} \
--set secrets.AZURE_ACCOUNT_KEY=xxxxxxxxxx \
--set configs.BLOB_STORAGE_PROVIDER=AZURE \
--set configs.AZURE_ACCOUNT_NAME=test-account \
--set configs.AZURE_BLOB_CONTAINER_CI_LOG=ci-log-container \
--set configs.AZURE_BLOB_CONTAINER_CI_CACHE=ci-cache-container \
--set argo-cd.enabled=true

helm repo add devtron https://helm.devtron.ai

helm repo update devtron

helm install devtron devtron/devtron-operator \
--create-namespace --namespace devtroncd \
--set installer.modules={cicd} \
--set configs.BLOB_STORAGE_PROVIDER=GCP \
--set secrets.BLOB_STORAGE_GCP_CREDENTIALS_JSON=eyJ0eXBlIjogInNlcnZpY2VfYWNjb3VudCIsInByb2plY3RfaWQiOiAiPHlvdXItcHJvamVjdC1pZD4iLCJwcml2YXRlX2tleV9pZCI6ICI8eW91ci1wcml2YXRlLWtleS1pZD4iLCJwcml2YXRlX2tleSI6ICI8eW91ci1wcml2YXRlLWtleT4iLCJjbGllbnRfZW1haWwiOiAiPHlvdXItY2xpZW50LWVtYWlsPiIsImNsaWVudF9pZCI6ICI8eW91ci1jbGllbnQtaWQ+IiwiYXV0aF91cmkiOiAiaHR0cHM6Ly9hY2NvdW50cy5nb29nbGUuY29tL28vb2F1dGgyL2F1dGgiLCJ0b2tlbl91cmkiOiAiaHR0cHM6Ly9vYXV0aDIuZ29vZ2xlYXBpcy5jb20vdG9rZW4iLCJhdXRoX3Byb3ZpZGVyX3g1MDlfY2VydF91cmwiOiAiaHR0cHM6Ly93d3cuZ29vZ2xlYXBpcy5jb20vb2F1dGgyL3YxL2NlcnRzIiwiY2xpZW50X3g1MDlfY2VydF91cmwiOiAiPHlvdXItY2xpZW50LWNlcnQtdXJsPiJ9Cg== \
--set configs.DEFAULT_CACHE_BUCKET=cache-bucket \
--set configs.DEFAULT_BUILD_LOGS_BUCKET=log-bucket \
--set argo-cd.enabled=true

Check Status of Devtron Installation

Note: The installation takes about 15 to 20 minutes to spin up all of the Devtron microservices one by one.

Run the following command to check the status of the installation:

kubectl -n devtroncd get installers installer-devtron \
-o jsonpath='{.status.sync.status}'

The command executes with one of the following output messages, indicating the status of the installation:

Check the installer logs

Run the following command to check the installer logs:

kubectl logs -f -l app=inception -n devtroncd

Devtron dashboard

Run the following command to get the Devtron dashboard URL:

kubectl get svc -n devtroncd devtron-service \
-o jsonpath='{.status.loadBalancer.ingress}'

You will get an output similar to the example shown below:

[map[hostname:aaff16e9760594a92afa0140dbfd99f7-305259315.us-east-1.elb.amazonaws.com]]

Use the hostname aaff16e9760594a92afa0140dbfd99f7-305259315.us-east-1.elb.amazonaws.com (Loadbalancer URL) to access the Devtron dashboard.

Note: If you do not get a hostname or receive a message that says "service doesn't exist," it means Devtron is still installing. Please wait until the installation is completed.

Note: You can also use a CNAME entry corresponding to your domain/subdomain to point to the Loadbalancer URL to access at a customized domain.

Devtron Admin credentials

When you install Devtron for the first time, it creates a default admin user and password (with unrestricted access to Devtron). You can use that credentials to log in as an administrator.

After the initial login, we recommend you set up any SSO service like Google, GitHub, etc., and then add other users (including yourself). Subsequently, all the users can use the same SSO (let's say, GitHub) to log in to Devtron's dashboard.

The section below will help you understand the process of getting the administrator credentials.

For Devtron version v0.6.0 and higher

Username: admin Password: Run the following command to get the admin password:

kubectl -n devtroncd get secret devtron-secret \
-o jsonpath='{.data.ADMIN_PASSWORD}' | base64 -d

kubectl -n devtroncd get secret devtron-secret \
-o jsonpath='{.data.ACD_PASSWORD}' | base64 -d

• If you want to uninstall Devtron or clean Devtron helm installer, refer our uninstall Devtron.

• Related to installation, please also refer FAQ section also.

Overview

The Kubernetes client by Devtron is a very lightweight dashboard that can be installed on arm64/amd64-based architectures. It comes with the features such as Kubernetes Resources Browser and Cluster Management that can provide control and observability for resources across clouds and clusters.

Devtron Kubernetes client is an intuitive Kubernetes Dashboard or a command line utility installed outside a Kubernetes cluster. The client can be installed on a desktop running on any Operating Systems and interact with all your Kubernetes clusters and workloads through an API server. It is a binary, packaged in a bash script that you can download and install by using the following set of commands.

By installing Devtron Kubernetes Client, you can access:

• Kubernetes Resource Browser

• Clusters Management Feature

Here are a few advantages of using Devtron Kubernetes Client:

• Managing Kubernetes Resources at scale: Clusters vary on business and architectural needs. Organizations tend to build smaller clusters for more decentralization. This practice leads to the creation of multiple clusters and more nodes. Managing them on a CLI requires multiple files, making it difficult to perform resource operations. But with the Devtron Kubernetes Client, you can gain more visibility into K8s resources easily.

• Unifying information in one place: When information is scattered across clusters, and you have to type commands with arguments to fetch desired output, the process becomes slow and error-prone. Without a single point of configuration source, the configurations of different config. files diverge, making them even more challenging to restore and track. The Devtron Kubernetes Client unifies all the information and tools into one interface to perform various contextual tasks.

• Accessibility during an outage for troubleshooting: As the Devtron Kubernetes Client runs outside a cluster, you can exercise basic control over their failed resources when there is a cluster-level outage. The Client helps to gather essential logs and data to pinpoint the root cause of the issue and reduce the time to restore service.

• Avoiding Kubeconfig version mismatch errors: With the Devtron Kubernetes Client, you can be relieved from maintaining the Kubeconfig versions for the respective clusters (v1.16 - 1.26 i.e, current version) as the Devtron Kubernetes Client performs self kubeconfig version control. Instead of managing multiple kubectl versions manually, it eliminates the chances of errors occurring due to the mismatch in configuration.

Install Devtron Kubernetes Client

• Download the bash script using the below URL: https://cdn.devtron.ai/k8s-client/devtron-install.bash

• To automatically download the executable file and to open the dashboard in the respective browser, run the following command:

   sh devtron-install.bash start  

Note: Make sure you place Devtron-install.bash in your current directory before you execute the command.

• Devtron Kubernetes Client opens in your browser automatically.

• You must add your cluster to make your cluster visible on the Kubernetes Resource Browser and Clusters section. To add a cluster, go to the Global Configurations and click Add Cluster. Refer documentation on how to add a cluster.

Note: You do not need to have a super admin permission to add a cluster if you install Devtron Kubernetes Client. You can add more than one cluster.

Kubernetes Resource Browser

Kubernetes Resource Browser provides a graphical user interface for interacting and managing all your Kubernetes (k8s) resources across clusters. It also helps you to deploy and manage Kubernetes resources and allows pod operations such as:

• View real-time logs

• Check manifest and edit live manifests of k8s resources

• Executable via terminal

• View Events

• Or, delete a resource

With Kubernetes Resource browser, you can also perform the following:

• Check the real-time health status

• Search for any workloads

• Manage multiple clusters and change cluster contexts

• Deploy multiple K8s manifests through Create UI option.

• Perform resource grouping at the cluster level.

After your cluster is added via Global Configurations, go to the Kubernetes Resource Browser page and select your cluster. Refer Resource Browser documentation for detail and its operations.

Note: You do not need to have a super admin permission to access Kubernetes Resource Browser if you install Devtron Kubernetes Client.

Cluster Management

With the Devtron Kubernetes Client, you can manage all your clusters running on-premises or on a cloud. It is a cluster and cloud agnostic platform where you can add as many clusters as you want, be it a lightweight cluster such as k3s/ microk8s or cloud managed clusters like Amazon EKS.

It enables you to observe and monitor the cluster health and real-time node conditions. The Cluster management feature provides a summary of nodes with all available labels, annotations, taints, and other parameters such as resource usage. In addition to that, it helps you to perform node operations such as:

• Debug a node

• Cordon a node

• Drain a node

• Taint a node

• Edit a node config

• Delete a node

With its rich features and intuitive interface, you can easily manage and debug clusters through cluster terminal access and use any CLI debugging tools like busybox, kubectl, netshoot or any custom CLI tools like k9s.

After your cluster is added via Global Configurations, go to the Clusters page and search or select your cluster. Refer Clusters documentation for detail and its operations.

Some Peripheral Commands

• In case if you close the browser by mistake, you can open the dashboard by executing the following command. It will open the dashboard through a port in the available web browser and store the Kubernetes client's state.

sh devtron-install.bash open 

• To stop the dashboard, you can execute the following command:

sh devtron-install.bash stop

• To update the Devtron Kubernetes Client, use the following command. It will stop the running dashboard and download the latest executable file and open it in the browser.

sh devtron-install.bash upgrade

Here we have demonstrated the installation of Devtron on popular cloud providers. The videos are easy to follow and provide step-by-step instructions.

Installing on EKS Cluster

Cloud Provider: Amazon Web Services (AWS)

Installing on AKS Cluster

Cloud Provider: Microsoft Azure

Installing on GKE Cluster

Cloud Provider: Google Cloud Platform (GCP)

To uninstall Devtron, run the following command:

This command will remove all the namespaces related to Devtron (devtroncd, devtron-cd, devtron-ci etc.).

helm uninstall devtron --namespace devtroncd

kubectl delete -n devtroncd -f https://raw.githubusercontent.com/devtron-labs/charts/main/charts/devtron/crds/crd-devtron.yaml

kubectl delete -n argo -f https://raw.githubusercontent.com/devtron-labs/devtron/main/manifests/yamls/workflow.yaml

kubectl delete ns devtroncd devtron-cd devtron-ci devtron-demo argo

You can configure Devtron by using configuration files. Configuration files are YAML files which are user-friendly. The configuration allows you to quickly roll back a configuration change if necessary. It also aids cluster re-creation and restoration.

There are two ways you can perform configurations while setting up Devtron dashboard:

• Installation Configurations

• Override Configurations

You can also setup ingress while setting up Devtron dashboard. Refer here for ingress setup.

Configure Secrets

For Helm installation this section refers to secrets section of values.yaml.

Configure the following properties:

Configure ConfigMaps

For Helm installation this section refers to configs section of values.yaml.

Configure the following properties:

Configure Resources

Devtron provides ways to control how much memory or CPU can be allocated to each Devtron microservice. You can adjust the resources that are allocated to these microservices based on your requirements. The resource configurations are available in following sizes:

Small: To configure the small resources (e.g. to manage less than 10 apps on Devtron ) based on the requirements, append the Devtron installation command with -f https://raw.githubusercontent.com/devtron-labs/devtron/main/charts/devtron/resources-small.yaml.

Configure Overrides

For Helm installation this section refers to customOverrides section of values.yaml. In this section you can override values of devtron-cm which you want to keep persistent. For example:

You can configure the following properties:

Storage for Logs and Cache

AWS SPECIFIC

While installing Devtron and using the AWS-S3 bucket for storing the logs and caches, the below parameters are to be used in the ConfigMap.

NOTE: For using the S3 bucket it is important to add the S3 permission policy to the IAM role attached to the nodes of the cluster.

The below parameters are to be used in the Secrets :

AZURE SPECIFIC

While installing Devtron using Azure Blob Storage for storing logs and caches, the below parameters will be used in the ConfigMap.

GOOGLE CLOUD STORAGE SPECIFIC

While installing Devtron using Google Cloud Storage for storing logs and caches, the below parameters will be used in the ConfigMap.

To convert string to base64 use the following command:

echo -n "string" | base64

Note:

1. Ensure that the cluster has read and write access to the S3 buckets/Azure Blob storage container mentioned in DEFAULT_CACHE_BUCKET, DEFAULT_BUILD_LOGS_BUCKET or AZURE_BLOB_CONTAINER_CI_LOG, or AZURE_BLOB_CONTAINER_CI_CACHE.

2. Ensure that the cluster has read access to AWS secrets backends (SSM & secrets manager).

The following tables contain parameters and their details for Secrets and ConfigMaps that are configured during the installation of Devtron. If the installation is done using Helm, the values can be tweaked in values.yaml file.

We can use the --set flag to override the default values when installing with Helm. For example, to update POSTGRESQL_PASSWORD and BLOB_STORAGE_PROVIDER, use the install command as:

helm install devtron devtron/devtron-operator --create-namespace --namespace devtroncd \
--set secrets.POSTGRESQL_PASSWORD=change-me \
--set configs.BLOB_STORAGE_PROVIDER=S3

Configuration of Blob Storage

Blob Storage allows users to store large amounts of unstructured data. Unstructured data is a data that does not adhere to a particular data model or definition, such as text or binary data. Configuring blob storage in your Devtron environment allows you to store build logs and cache.

In case, if you do not configure the Blob Storage, then:

• You will not be able to access the build and deployment logs after an hour.

• Build time for commit hash takes longer as cache is not available.

• Artifact reports cannot be generated in pre/post build and deployment stages.

You can configure Blob Storage with one of the following Blob Storage providers given below:

Note: You can also use the respective following command to switch to another Blob Storage provider. As an example, If you are using MinIO Storage and want to switch to Azure Blob Storage, use the command provided on the Azure Blob Storage tab to switch.

helm repo update

helm upgrade devtron devtron/devtron-operator --namespace devtroncd \
--reuse-values \
--set installer.modules={cicd} \
--set minio.enabled=true

helm repo update
helm upgrade devtron devtron/devtron-operator --namespace devtroncd \
--reuse-values \
--set installer.modules={cicd} \
--set configs.BLOB_STORAGE_PROVIDER=S3 \
--set configs.DEFAULT_CACHE_BUCKET=demo-s3-bucket \
--set configs.DEFAULT_CACHE_BUCKET_REGION=us-east-1 \
--set configs.DEFAULT_BUILD_LOGS_BUCKET=demo-s3-bucket \
--set configs.DEFAULT_CD_LOGS_BUCKET_REGION=us-east-1

helm repo update

helm upgrade devtron devtron/devtron-operator --namespace devtroncd \
--reuse-values \
--set installer.modules={cicd} \
--set configs.BLOB_STORAGE_PROVIDER=S3 \
--set configs.DEFAULT_CACHE_BUCKET=demo-s3-bucket \
--set configs.DEFAULT_CACHE_BUCKET_REGION=us-east-1 \
--set configs.DEFAULT_BUILD_LOGS_BUCKET=demo-s3-bucket \
--set configs.DEFAULT_CD_LOGS_BUCKET_REGION=us-east-1 \
--set secrets.BLOB_STORAGE_S3_ACCESS_KEY=<access-key> \
--set secrets.BLOB_STORAGE_S3_SECRET_KEY=<secret-key>

helm repo update

helm upgrade devtron devtron/devtron-operator --namespace devtroncd \
--reuse-values \
--set installer.modules={cicd} \
--set configs.BLOB_STORAGE_PROVIDER=S3 \
--set configs.DEFAULT_CACHE_BUCKET=demo-s3-bucket \
--set configs.DEFAULT_CACHE_BUCKET_REGION=us-east-1 \
--set configs.DEFAULT_BUILD_LOGS_BUCKET=demo-s3-bucket \
--set configs.DEFAULT_CD_LOGS_BUCKET_REGION=us-east-1 \
--set secrets.BLOB_STORAGE_S3_ACCESS_KEY=<access-key> \
--set secrets.BLOB_STORAGE_S3_SECRET_KEY=<secret-key> \
--set configs.BLOB_STORAGE_S3_ENDPOINT=<endpoint>

helm repo update
helm upgrade devtron devtron/devtron-operator --namespace devtroncd \
--reuse-values \
--set installer.modules={cicd} \
--set secrets.AZURE_ACCOUNT_KEY=xxxxxxxxxx \
--set configs.BLOB_STORAGE_PROVIDER=AZURE \
--set configs.AZURE_ACCOUNT_NAME=test-account \
--set configs.AZURE_BLOB_CONTAINER_CI_LOG=ci-log-container \
--set configs.AZURE_BLOB_CONTAINER_CI_CACHE=ci-cache-container

helm repo update

helm upgrade devtron devtron/devtron-operator --namespace devtroncd \
--reuse-values \
--set installer.modules={cicd} \
--set configs.BLOB_STORAGE_PROVIDER=GCP \
--set secrets.BLOB_STORAGE_GCP_CREDENTIALS_JSON=eyJ0eXBlIjogInNlcnZpY2VfYWNjb3VudCIsInByb2plY3RfaWQiOiAiPHlvdXItcHJvamVjdC1pZD4iLCJwcml2YXRlX2tleV9pZCI6ICI8eW91ci1wcml2YXRlLWtleS1pZD4iLCJwcml2YXRlX2tleSI6ICI8eW91ci1wcml2YXRlLWtleT4iLCJjbGllbnRfZW1haWwiOiAiPHlvdXItY2xpZW50LWVtYWlsPiIsImNsaWVudF9pZCI6ICI8eW91ci1jbGllbnQtaWQ+IiwiYXV0aF91cmkiOiAiaHR0cHM6Ly9hY2NvdW50cy5nb29nbGUuY29tL28vb2F1dGgyL2F1dGgiLCJ0b2tlbl91cmkiOiAiaHR0cHM6Ly9vYXV0aDIuZ29vZ2xlYXBpcy5jb20vdG9rZW4iLCJhdXRoX3Byb3ZpZGVyX3g1MDlfY2VydF91cmwiOiAiaHR0cHM6Ly93d3cuZ29vZ2xlYXBpcy5jb20vb2F1dGgyL3YxL2NlcnRzIiwiY2xpZW50X3g1MDlfY2VydF91cmwiOiAiPHlvdXItY2xpZW50LWNlcnQtdXJsPiJ9Cg== \
--set configs.DEFAULT_CACHE_BUCKET=cache-bucket \
--set configs.DEFAULT_BUILD_LOGS_BUCKET=log-bucket

Secrets

ConfigMaps

Dashboard Configurations

RECOMMEND_SECURITY_SCANNING=false
FORCE_SECURITY_SCANNING=false
HIDE_DISCORD=false

In certain cases, you may want to override default configurations provided by Devtron. For example, for deployments or statefulsets you may want to change the memory or CPU requests or limit or add node affinity or taint tolerance. Say, for ingress, you may want to add annotations or host. Samples are available inside the manifests/updates directory.

To modify a particular object, it looks in namespace devtroncd for the corresponding configmap as mentioned in the mapping below:

Let's take an example to understand how to override specific values. Say, you want to override annotations and host in the ingress, i.e., you want to change devtronIngress, copy the file devtron-ingress-override.yaml. This file contains a configmap to modify devtronIngress as mentioned above. Please note the structure of this configmap, data should have the key override with a multiline string as a value.

apiVersion, kind, metadata.name in the multiline string is used to match the object which needs to be modified. In this particular case it will look for apiVersion: extensions/v1beta1, kind: Ingress and metadata.name: devtron-ingress and will apply changes mentioned inside update: as per the example inside the metadata: it will add annotations owner: app1 and inside spec.rules.http.host it will add http://change-me.

In case you want to change multiple objects, for eg in argocd you want to change the config of argocd-dex-server as well as argocd-redis then follow the example in devtron-argocd-override.yaml.

Once we have made these changes in our local system we need to apply them to a Kubernetes cluster on which Devtron is installed currently using the below command:

kubectl apply -f file-name -n devtroncd

Run the following command to make these changes take effect:

kubectl patch -n devtroncd installer installer-devtron --type='json' -p='[{"op": "add", "path": "/spec/reSync", "value": true }]'

Our changes would have been propagated to Devtron after 20-30 minutes.

Recommended Resources for Production use

To use Devtron for production deployments, use our recommended production overrides located in manifests/updates/production. This configuration should be enough for handling up to 200 microservices.

The overall resources required for the recommended production overrides are:

The production overrides can be applied as pre-devtron installation as well as post-devtron installation in the respective namespace.

Pre-Devtron Installation

If you want to install a new Devtron instance for production-ready deployments, this is the best option for you.

Create the namespace and apply the overrides files as stated above:

kubectl create ns devtroncd

After files are applied, you are ready to install your Devtron instance with production-ready resources.

Post-Devtron Installation

If you have an existing Devtron instance and want to migrate it for production-ready deployments, this is the right option for you.

In the existing namespace, apply the production overrides as we do it above.

kubectl apply -f prod-configs -n devtroncd

After Devtron is installed, Devtron is accessible through service devtron-service. If you want to access Devtron through ingress, edit devtron-service and change the loadbalancer to ClusterIP. You can do this using kubectl patch command:

kubectl patch -n devtroncd svc devtron-service -p '{"spec": {"ports": [{"port": 80,"targetPort": "devtron","protocol": "TCP","name": "devtron"}],"type": "ClusterIP","selector": {"app": "devtron"}}}'

After this, create ingress by applying the ingress yaml file. You can use this yaml file to create ingress to access Devtron:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations: 
    nginx.ingress.kubernetes.io/app-root: /dashboard
  labels:
    app: devtron
    release: devtron
  name: devtron-ingress
  namespace: devtroncd
spec:
  ingressClassName: nginx
  rules:
  - http:
      paths:
      - backend:
          service:
            name: devtron-service
            port:
              number: 80
        path: /orchestrator
        pathType: ImplementationSpecific 
      - backend:
          service:
            name: devtron-service
            port:
              number: 80
        path: /dashboard
        pathType: ImplementationSpecific
      - backend:
          service:
            name: devtron-service
            port:
              number: 80
        path: /grafana
        pathType: ImplementationSpecific  

You can access Devtron from any host after applying this yaml. For k8s versions <1.19, apply this yaml:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations: 
    nginx.ingress.kubernetes.io/app-root: /dashboard
  labels:
    app: devtron
    release: devtron
  name: devtron-ingress
  namespace: devtroncd
spec:
  rules:
  - http:
      paths:
      - backend:
          serviceName: devtron-service
          servicePort: 80
        path: /orchestrator
      - backend:
          serviceName: devtron-service
          servicePort: 80
        path: /dashboard
        pathType: ImplementationSpecific  

Optionally, you also can access Devtron through a specific host by running the following YAML file:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations: 
    nginx.ingress.kubernetes.io/app-root: /dashboard
  labels:
    app: devtron
    release: devtron
  name: devtron-ingress
  namespace: devtroncd
spec:
  ingressClassName: nginx
  rules:
    - host: devtron.example.com
      http:
        paths:
          - backend:
              service:
                name: devtron-service
                port:
                  number: 80
            path: /orchestrator
            pathType: ImplementationSpecific
          - backend:
              service:
                name: devtron-service
                port:
                  number: 80
            path: /dashboard
            pathType: ImplementationSpecific
          - backend:
              service:
                name: devtron-service
                port:
                  number: 80
            path: /grafana
            pathType: ImplementationSpecific

Enable HTTPS For Devtron

Once ingress setup for devtron is done and you want to run Devtron over https, you need to add different annotations for different ingress controllers and load balancers.

1. Nginx Ingress Controller

In case of nginx ingress controller, add the following annotations under service.annotations under nginx ingress controller to run devtron over https.

(i) Amazon Web Services (AWS)

If you are using AWS cloud, add the following annotations under service.annotations under nginx ingress controller.

  annotations:
    service.beta.kubernetes.io/aws-load-balancer-backend-protocol: "http"
    service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "443"
    nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
    service.beta.kubernetes.io/aws-load-balancer-ssl-cert: "<acm-arn-here>"

(ii) Digital Ocean

If you are using Digital Ocean cloud, add the following annotations under service.annotations under nginx ingress controller.

annotations:
  service.beta.kubernetes.io/do-loadbalancer-protocol: "http"
  service.beta.kubernetes.io/do-loadbalancer-tls-ports: "443"
  service.beta.kubernetes.io/do-loadbalancer-certificate-id: "<your-certificate-id>"
  service.beta.kubernetes.io/do-loadbalancer-redirect-http-to-https: "true"

2. AWS Application Load Balancer (AWS ALB)

In case of AWS application load balancer, add following annotations under ingress.annotations to run devtron over https.

  annotations:
    kubernetes.io/ingress.class: alb
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS": 443}]'
    alb.ingress.kubernetes.io/certificate-arn: "<acm-arn-here>"

3. Azure Application Gateway

In case of AWS application load balancer, the following annotations need to be added under ingress.annotations to run devtron over https.

 annotations:
  kubernetes.io/ingress.class: "azure/application-gateway"
  appgw.ingress.kubernetes.io/backend-protocol: "http"
  appgw.ingress.kubernetes.io/ssl-redirect: "true"
  appgw.ingress.kubernetes.io/appgw-ssl-certificate: "<name-of-appgw-installed-certificate>"

For an Ingress resource to be observed by AGIC (Application Gateway Ingress Controller) must be annotated with kubernetes.io/ingress.class: azure/application-gateway. Only then AGIC will work with the Ingress resource in question.

Note: Make sure NOT to use port 80 with HTTPS and port 443 with HTTP on the Pods.

Host URL is the domain address at which your devtron dashboard can be reached.

Add Host URL

To add host URL, go to the Host URL section of Global Configurations.

On the Host URL page:

• Enter the host URL in the Host URL field.

• Or, you can select auto-detect from your browser.

• Next, click Update.

While container registries are typically used for storing images built by the CI Pipeline, an OCI registry can store container images as well as other artifacts such as helm charts. In other words, all container registries are OCI registries, but not all OCI registries are container registries.

You can configure a container registry using any registry provider of your choice. It allows you to build, deploy, and manage your container images or charts with easy-to-use UI.

Add Container Registry

1. From the left sidebar, go to Global Configurations → Container/OCI Registry.
2. Click Add Registry.
3. Choose a provider from the Registry provider dropdown. View the Supported Registry Providers.

4. Choose the Registry type:

   • Private Registry: Choose this if your images or artifacts are hosted or should be hosted on a private registry restricted to authenticated users of that registry. Selecting this option requires you to enter your registry credentials (username and password/token).

   • Public Registry: Unlike private registry, this doesn't require your registry credentials. Only the registry URL and repository name(s) would suffice.

5. Assuming your registry type is private, here are few of the common fields you can expect:

   Fields

   Description

   Name

   Registry URL

   Provide the URL of your registry in case it doesn't come prefilled (do not include oci://, http://, or /https:// in the URL)

   Authentication Type

   Push container images

   Push helm packages

   Tick this checkbox if you wish to push helm charts to your registry

   Use as chart repository

   Tick this checkbox if you want Devtron to pull helm charts from your registry and display them on its chart store. Also, you will have to provide a list of repositories (present within your registry) for Devtron to successfully pull the helm charts.

   Set as default registry

   Tick this checkbox to set your registry as the default registry hub for your images or artifacts

6. Click Save.

Supported Registry Providers

ECR

Amazon ECR is an AWS-managed container image registry service. The ECR provides resource-based permissions to the private repositories using AWS Identity and Access Management (IAM). ECR allows both Key-based and Role-based authentications.

Before you begin, create an IAM user and attach the ECR policy according to the authentication type.

Provide the following additional information apart from the common fields:

Docker

Provide the following additional information apart from the common fields:

Azure

For Azure, the service principal authentication method can be used to authenticate with username and password. Visit this link to get the username and password for this registry.

Provide the following additional information apart from the common fields:

Artifact Registry (GCP)

JSON key file authentication method can be used to authenticate with username and service account JSON file. Visit this link to get the username and service account JSON file for this registry.

Provide the following additional information apart from the common fields:

Google Container Registry (GCR)

JSON key file authentication method can be used to authenticate with username and service account JSON file. Please follow link to get the username and service account JSON file for this registry.

Quay

Provide the following additional information apart from the common fields:

Other

Provide below information if you select the registry type as Other.

Registry Credential Access

You can create a Pod that uses a Secret to pull an image from a private container registry. You can use any private container registry of your choice, for e.g., Docker Hub.

Super-admin users can decide if they want to auto-inject registry credentials or use a secret to pull an image for deployment to environments on specific clusters.

1. To manage the access of registry credentials, click Manage.

There are two options to manage the access of registry credentials:

1. You can choose one of the two options for defining credentials:

• Use Registry Credentials

• Specify Image Pull Secret

Use Registry Credentials

If you select Use Registry Credentials, the clusters will be auto-injected with the registry credentials of your registry type. As an example, If you select Docker as Registry Type, then the clusters will be auto-injected with the username and password/token which you use on the Docker Hub account.

Click Save.

Specify Image Pull Secret

You can create a Secret by providing credentials on the command line.

Create this Secret and name it regcred (let's say):

kubectl create -n <namespace> secret docker-registry regcred --docker-server=<your-registry-server> --docker-username=<your-name> --docker-password=<your-pword> --docker-email=<your-email>

where,

• namespace is your sub-cluster, e.g., devtron-demo

• your-registry-server is your Private Docker Registry FQDN. Use https://index.docker.io/v1/ for Docker Hub.

• your-name is your Docker username

• your-pword is your Docker password

• your-email is your Docker email

You have successfully set your Docker credentials in the cluster as a Secret called regcred.

Enter the Secret name in the field and click Save.

You can add your existing Kubernetes clusters and environments on the Clusters and Environments section. You must have a super admin access to add a cluster.

Add Cluster:

To add cluster, go to the Clusters & Environments section of Global Configurations. Click Add cluster.

Add Clusters Using Server URL & Bearer Token

To add a Kubernetes cluster on Devtron using server url and the bearer token, provide the information in the following fields:

Get Cluster Credentials

Prerequisites: kubectl must be installed on the bastion.

Note: We recommend to use a self-hosted URL instead of cloud hosted URL. Refer the benefits of self-hosted URL.

You can get the Server URL & Bearer Token by running the following command depending on the cluster provider:

curl -O https://raw.githubusercontent.com/devtron-labs/utilities/main/kubeconfig-exporter/kubernetes_export_sa.sh && bash kubernetes_export_sa.sh cd-user  devtroncd

curl -O https://raw.githubusercontent.com/devtron-labs/utilities/main/kubeconfig-exporter/kubernetes_export_sa.sh && sed -i 's/kubectl/microk8s kubectl/g' \
kubernetes_export_sa.sh && bash kubernetes_export_sa.sh cd-user \
devtroncd

Benefits of Self-hosted URL

• Disaster Recovery:

  • It is not possible to edit the server URL of a cloud specific provider. If you're using an EKS URL (e.g. *****.eu-west-1.elb.amazonaws.com), it will be a tedious task to add a new cluster and migrate all the services one by one.

  • But in case of using a self-hosted URL (e.g. clear.example.com), you can just point to the new cluster's server URL in DNS manager and update the new cluster token and sync all the deployments.

• Easy Cluster Migrations:

  • In case of managed Kubernetes clusters (like EKS, AKS, GKE etc) which is a cloud provider specific, migrating your cluster from one provider to another will result in waste of time and effort.

  • On the other hand, migration for a self-hosted URL is easy as the URL is of single hosted domain independent of the cloud provider.

Add Clusters Using Kubeconfig

To add clusters using kubeconfig, follow these steps:

1. First, navigate to the global configurations menu, and then go to "clusters and environment" section.

2. Click on the Add cluster button. In the options provided, choose the From kubeconfig option.

3. Next, either paste the kubeconfig file or browse for it and select the appropriate file.

4. Afterward, click on the Get cluster button. This action will display the cluster details alongside the kubeconfig.

1. Select the desired cluster and click on Save to successfully add the cluster to Devtron.

Note: Please ensure that the kubeconfig file you use has admin permissions. It is crucial for Devtron to have the necessary administrative privileges; otherwise, it may encounter failures or disruptions during deployments and other operations. Admin permission is essential to ensure the smooth functioning of Devtron and to prevent any potential issues that may arise due to insufficient privileges.

Configure Prometheus (Enable Applications Metrics)

If you want to see application metrics against the applications deployed in the cluster, Prometheus must be deployed in the cluster. Prometheus is a powerful tool to provide graphical insight into your application behavior.

Note: Make sure that you install Monitoring (Grafana) from the Devtron Stack Manager to configure prometheus. If you do not install Monitoring (Grafana), then the option to configure prometheus will not be available.

Enable the application metrics to configure prometheus and provide the information in the following fields:

Now, click Save Cluster to save your cluster on Devtron.

Installing Devtron Agent

Your Kubernetes cluster gets mapped with Devtron when you save the cluster configurations. Now, the Devtron agent must be installed on the added cluster so that you can deploy your applications on that cluster.

When the Devtron agent starts installing, click Details to check the installation status.

A new window pops up displaying all the details about the Devtron agent.

Add Environment

Once you have added your cluster in the Clusters & Environments, you can add the environment by clicking Add environment.

A new environment window pops up.

Click Save and your environment will be created.

Update Environment

• You can also update an environment by clicking the environment.

• You can change Production and Non-Production options only.

• You cannot change the Environment Name and Namespace Name.

• Make sure to click Update to update your environment.

Git Accounts allow you to connect your code source with Devtron. You will be able to use these git accounts to build the code using the CI pipeline.

Add Git Account

To add git account, go to the Git accounts section of Global Configurations. Click Add git account.

Provide the information in the following fields to add your git account:

Update Git Account

To update the git account:

1. Click the git account which you want to update.

2. Update the required changes.

3. Click Update to save the changes.

Updates can only be made within one Authentication type or one protocol type, i.e. HTTPS (Anonymous or User Auth) & SSH. You can update from Anonymous to User Auth & vice versa, but not from Anonymous or User Auth to SSH and vice versa.

Note:

• You can enable or disable a git account. Enabled git accounts will be available on the App Configuration > Git repository.

A global configuration allows you to easily share common configuration between multiple repositories without copy/pasting it to these repositories.

Before you start creating an application, we recommend to provide basic information in different sections of Global Configurations available in Devtron.

Host URL

GitOps

Projects

Clusters & Environments

Git Accounts

Container/OCI Registry

Chart Repositories

Custom Charts

Authorization

Notifications

External Links

Catalog Framework

Scoped Variables

Pull Image Digest

Tags Policy

Lock Deployment Configuration

Image Promotion Policy

Filter Condition

Build Infra

You can also refer our YouTube video provided here.

You can add more chart repositories to Devtron. Once added, they will be available in the All Charts section of the Chart Store.

Note: After the successful installation of Devtron, click Refetch Charts to sync and download all the default charts listed on the dashboard.

Add Chart Repository

To add chart repository, go to the Chart Repositories section of Global Configurations. Click Add repository.

Note: Only public chart repositories can be connected as of now via Devtron.

Provide below information in the following fields:

Update Chart Repository

You can also update your saved chart repository settings.

1. Click the chart repository which you want to update.

2. Modify the required changes and click Update to save you changes.

Note:

• You can perform a dry run to validate the below chart repo configurations by clicking Validate.

• You can enable or disable your chart repository. If you enable it, then you will be able to see the enabled chart in All Charts section of the Chart Store.

In this section, we describe the steps in detail on how you can install Devtron with CI/CD integration.

Prerequisites

Install Helm, if you have not installed it already.

helm repo add aws-ebs-csi-driver \
https://kubernetes-sigs.github.io/aws-ebs-csi-driver \
helm repo update \
helm upgrade --install aws-ebs-csi-driver \
--namespace kube-system aws-ebs-csi-driver/aws-ebs-csi-driver

Command

Run the following command to install the latest version of Devtron along with the CI/CD module:

helm repo add devtron https://helm.devtron.ai 

helm repo update devtron

helm install devtron devtron/devtron-operator \
--create-namespace --namespace devtroncd \
--set installer.modules={cicd}

Install Multi-Architecture Nodes (ARM and AMD)

To install Devtron on clusters with the multi-architecture nodes (ARM and AMD), append the Devtron installation command with --set installer.arch=multi-arch.

Configure Blob Storage during Installation

Configuring Blob Storage in your Devtron environment allows you to store build logs and cache. In case, if you do not configure the Blob Storage, then:

• You will not be able to access the build and deployment logs after an hour.

• Build time for commit hash takes longer as cache is not available.

• Artifact reports cannot be generated in pre/post build and deployment stages.

Choose one of the options to configure blob storage:

helm repo add devtron https://helm.devtron.ai 

helm repo update devtron

helm install devtron devtron/devtron-operator \
--create-namespace --namespace devtroncd \
--set installer.modules={cicd} \
--set minio.enabled=true

helm repo add devtron https://helm.devtron.ai

helm repo update devtron

helm install devtron devtron/devtron-operator \
--create-namespace --namespace devtroncd \
--set installer.modules={cicd} \
--set configs.BLOB_STORAGE_PROVIDER=S3 \
--set configs.DEFAULT_CACHE_BUCKET=demo-s3-bucket \
--set configs.DEFAULT_CACHE_BUCKET_REGION=us-east-1 \
--set configs.DEFAULT_BUILD_LOGS_BUCKET=demo-s3-bucket \
--set configs.DEFAULT_CD_LOGS_BUCKET_REGION=us-east-1

helm repo add devtron https://helm.devtron.ai

helm repo update devtron

helm install devtron devtron/devtron-operator \
--create-namespace --namespace devtroncd \
--set installer.modules={cicd} \
--set configs.BLOB_STORAGE_PROVIDER=S3 \
--set configs.DEFAULT_CACHE_BUCKET=demo-s3-bucket \
--set configs.DEFAULT_CACHE_BUCKET_REGION=us-east-1 \
--set configs.DEFAULT_BUILD_LOGS_BUCKET=demo-s3-bucket \
--set configs.DEFAULT_CD_LOGS_BUCKET_REGION=us-east-1 \
--set secrets.BLOB_STORAGE_S3_ACCESS_KEY=<access-key> \
--set secrets.BLOB_STORAGE_S3_SECRET_KEY=<secret-key>

helm repo add devtron https://helm.devtron.ai

helm repo update devtron

helm install devtron devtron/devtron-operator \
--create-namespace --namespace devtroncd \
--set installer.modules={cicd} \
--set configs.BLOB_STORAGE_PROVIDER=S3 \
--set configs.DEFAULT_CACHE_BUCKET=demo-s3-bucket \
--set configs.DEFAULT_CACHE_BUCKET_REGION=us-east-1 \
--set configs.DEFAULT_BUILD_LOGS_BUCKET=demo-s3-bucket \
--set configs.DEFAULT_CD_LOGS_BUCKET_REGION=us-east-1 \
--set secrets.BLOB_STORAGE_S3_ACCESS_KEY=<access-key> \
--set secrets.BLOB_STORAGE_S3_SECRET_KEY=<secret-key> \
--set configs.BLOB_STORAGE_S3_ENDPOINT=<endpoint>

helm repo add devtron https://helm.devtron.ai

helm repo update devtron

helm install devtron devtron/devtron-operator \
--create-namespace --namespace devtroncd \
--set installer.modules={cicd} \
--set secrets.AZURE_ACCOUNT_KEY=xxxxxxxxxx \
--set configs.BLOB_STORAGE_PROVIDER=AZURE \
--set configs.AZURE_ACCOUNT_NAME=test-account \
--set configs.AZURE_BLOB_CONTAINER_CI_LOG=ci-log-container \
--set configs.AZURE_BLOB_CONTAINER_CI_CACHE=ci-cache-container

helm repo add devtron https://helm.devtron.ai

helm repo update devtron

helm install devtron devtron/devtron-operator \
--create-namespace --namespace devtroncd \
--set installer.modules={cicd} \
--set configs.BLOB_STORAGE_PROVIDER=GCP \
--set secrets.BLOB_STORAGE_GCP_CREDENTIALS_JSON=eyJ0eXBlIjogInNlcnZpY2VfYWNjb3VudCIsInByb2plY3RfaWQiOiAiPHlvdXItcHJvamVjdC1pZD4iLCJwcml2YXRlX2tleV9pZCI6ICI8eW91ci1wcml2YXRlLWtleS1pZD4iLCJwcml2YXRlX2tleSI6ICI8eW91ci1wcml2YXRlLWtleT4iLCJjbGllbnRfZW1haWwiOiAiPHlvdXItY2xpZW50LWVtYWlsPiIsImNsaWVudF9pZCI6ICI8eW91ci1jbGllbnQtaWQ+IiwiYXV0aF91cmkiOiAiaHR0cHM6Ly9hY2NvdW50cy5nb29nbGUuY29tL28vb2F1dGgyL2F1dGgiLCJ0b2tlbl91cmkiOiAiaHR0cHM6Ly9vYXV0aDIuZ29vZ2xlYXBpcy5jb20vdG9rZW4iLCJhdXRoX3Byb3ZpZGVyX3g1MDlfY2VydF91cmwiOiAiaHR0cHM6Ly93d3cuZ29vZ2xlYXBpcy5jb20vb2F1dGgyL3YxL2NlcnRzIiwiY2xpZW50X3g1MDlfY2VydF91cmwiOiAiPHlvdXItY2xpZW50LWNlcnQtdXJsPiJ9Cg== \
--set configs.DEFAULT_CACHE_BUCKET=cache-bucket \
--set configs.DEFAULT_BUILD_LOGS_BUCKET=log-bucket

Check Status of Devtron Installation

Run the following command to check the status of the installation:

kubectl -n devtroncd get installers installer-devtron \
-o jsonpath='{.status.sync.status}'

The command executes with one of the following output messages, indicating the status of the installation:

Check the Installer Logs

Run the following command to check the installer logs:

kubectl logs -f -l app=inception -n devtroncd

Devtron Dashboard

Run the following command to get the Devtron dashboard URL:

kubectl get svc -n devtroncd devtron-service \
-o jsonpath='{.status.loadBalancer.ingress}'

You will get an output similar to the example shown below:

[map[hostname:aaff16e9760594a92afa0140dbfd99f7-305259315.us-east-1.elb.amazonaws.com]]

Use the hostname aaff16e9760594a92afa0140dbfd99f7-305259315.us-east-1.elb.amazonaws.com (Loadbalancer URL) to access the Devtron dashboard.

Devtron Admin Credentials

When you install Devtron for the first time, it creates a default admin user and password (with unrestricted access to Devtron). You can use that credentials to log in as an administrator.

After the initial login, we recommend you set up any SSO service like Google, GitHub, etc., and then add other users (including yourself). Subsequently, all the users can use the same SSO (let's say, GitHub) to log in to Devtron's dashboard.

The sections below will help you understand the process of getting the administrator password.

For Devtron version v0.6.0 and higher

Username: admin Password: Run the following command to get the admin password:

kubectl -n devtroncd get secret devtron-secret \
-o jsonpath='{.data.ADMIN_PASSWORD}' | base64 -d

kubectl -n devtroncd get secret devtron-secret \
-o jsonpath='{.data.ACD_PASSWORD}' | base64 -d

• If you want to uninstall Devtron or clean Devtron helm installer, refer our uninstall Devtron.

• Related to installaltion, please also refer FAQ section also.

Introduction

In Devtron, you can either use Helm or GitOps (Argo CD) to deploy your applications and charts. GitOps is a branch of DevOps that focuses on using Git repositories to manage infrastructure and application code deployments.

If you use the GitOps approach, Devtron will store Kubernetes configuration files and the desired state of your applications in Git repositories.

Steps to Configure GitOps

1. Go to Global Configurations → GitOps
2. Select any one of the supported Git providers to configure GitOps.

1. Fill all the mandatory fields. Refer supported Git providers to know more about the respective fields.
2. In the Directory Management in Git section, you get the following options:

   • Use default git repository structure:

     This option lets Devtron automatically create a GitOps repository within your organization. The repository name will match your application name, and it cannot be changed. Since Devtron needs admin access to create the repository, ensure the Git credentials you provided in Step 3 have administrator rights.

   • Allow changing git repository for application:

     Select this option if you wish to use your own GitOps repo. This is ideal if there are any confidentiality/security concerns that prevent you from giving us admin access. Therefore, the onus is on you to create a GitOps repo with your Git provider, and then add it to the specific application on Devtron. Make sure the Git credentials you provided in Step 3 have at least read/write access. Choosing this option will unlock a GitOps Configuration page under the App Configuration tab.
3. Click Save/Update. A green tick will appear on the active Git provider.

Feature Flag

Alternatively, you may use the feature flag FEATURE_USER_DEFINED_GITOPS_REPO_ENABLE to enable or disable custom GitOps repo.

How to Use Feature Flag

1. Go to Devtron's Resource Browser.

2. Select the cluster where Devtron is running, i.e., default_cluster.

3. Go to the Config & Storage dropdown on the left.

4. Click ConfigMap.

5. Use the namespace filter (located on the right-hand side) to select devtroncd namespace. Therefore, it will show only the ConfigMaps related to Devtron, and filter out the rest.

6. Find the ConfigMap meant for the dashboard of your Devtron instance, i.e., dashboard-cm (with an optional suffix).

7. Click Edit Live Manifest.

8. Add the feature flag (with the intended boolean value) within the data dictionary

9. Click Apply Changes.

Supported Git Providers

Below are the Git providers supported in Devtron for storing configuration files.

• GitHub

• GitLab

• Azure

• Bitbucket

GitHub

Fill the following mandatory fields:

GitLab

Fill the following mandatory fields:

Azure

Fill the following mandatory fields:

Bitbucket

Here, you get 2 options:

• Bitbucket Cloud - Select this if you wish to store GitOps configuration in a web-based Git repository hosting service offered by Bitbucket.

• Bitbucket Data Center - Select this if you wish to store GitOps configuration in a git repository hosted on a self-managed Bitbucket Data Center (on-prem).

Bitbucket Cloud

Fill the following mandatory fields:

Bitbucket Data Center

Fill the following mandatory fields:

Miscellaneous

Creating Organization in GitHub

1. Create a new account on GitHub (if you do not have one).

2. On the upper-right corner of your GitHub page, click your profile photo, then click Settings.

3. On the Access section, click Organizations.

4. On the Organizations section, click New organization.

5. Pick a plan for your organization. You have the option to select create free organization also.

6. On the Set up your organization page,

   • Enter the organization account name, contact email.

   • Select the option your organization belongs to.

   • Verify your account and click Next.

   • Your GitHub organization name will be created.

7. Go to your profile and click Your organizations to view all the organizations you created.

For more information about the plans available for your team, see GitHub's products. You can also refer GitHub organization official doc page for more detail.

Note:

• repo - Full control of private repositories (able to access commit status, deployment status, and public repositories).

• admin:org - Full control of organizations and teams (Read and write access).

• delete_repo - Grants delete repo access on private repositories.

Creating Group in GitLab

1. Create a new account on GitLab (if you do not have one).

2. You can create a group by going to the 'Groups' tab on the GitLab dashboard and click New group.

3. Select Create group.

4. Enter the group name (required) and select the optional descriptions if required, and click Create group.

5. Your group will be created and your group name will be assigned with a new Group ID (e.g. 61512475).

Note:

• api - Grants complete read/write access to the scoped project API.

• write_repository - Allows read/write access (pull, push) to the repository.

Creating Project in Azure DevOps

1. Go to Azure DevOps and navigate to Projects.

2. Select your organization and click New project.

3. On the Create new project page,

   • Enter the project name and description of the project.

   • Select the visibility option (private or public), initial source control type, and work item process.

   • Click Create.

   • Azure DevOps displays the project welcome page with the project name.

You can also refer Azure DevOps - Project Creation official page for more details.

Note:

• code - Grants the ability to read source code and metadata about commits, change sets, branches, and other version control artifacts. More information on scopes in Azure DevOps.

Creating Workspace in Bitbucket

1. Create a new individual account on Bitbucket (if you do not have one).

2. Select your profile and settings avatar on the upper-right corner of the top navigation bar.

3. Select All workspaces from the dropdown menu.

4. Select the Create workspace on the upper-right corner of the Workspaces page.

5. On the Create a Workspace page:

• Enter a Workspace name.

• Enter a Workspace ID. Your ID cannot have any spaces or special characters, but numbers and capital letters are fine. This ID becomes part of the URL for the workspace and anywhere else where there is a label that identifies the team (APIs, permission groups, OAuth, etc.).

• Click Create.

1. Your Workspace name and Workspace ID will be created.

You can also refer official Bitbucket Workspace page for more details.

Note:

• repo - Full control of repositories (Read, Write, Admin, Delete) access.

Devtron is a tool integration platform for Kubernetes.

Devtron deeply integrates with products across the lifecycle of microservices i.e., CI/CD, security, cost, debugging, and observability via an intuitive web interface. Devtron helps you to deploy, observe, manage & debug the existing Helm apps in all your clusters.

Devtron's Key Features:

No Code Software Delivery Workflow for Kubernetes

• Workflow which understands the domain of Kubernetes, testing, CD, SecOps so that you don't have to write scripts

• Reusable and composable components so that workflows are easy to construct and reason through

Multi-cloud Deployment

• Deploy to multiple Kubernetes clusters on multiple cloud/on-prem from one Devtron setup

• Works for all cloud providers and on-premise Kubernetes clusters

Easy DevSecOps Integration

• Multi-level security policy at global, cluster, environment, and application-level for efficient hierarchical policy management

• Behavior-driven security policy

• Define policies and exceptions for Kubernetes resources

• Define policies for events for faster resolution

Application Debugging Dashboard

• One place for all historical Kubernetes events

• Access all manifests securely, such as secret obfuscation

• Application metrics for CPU, RAM, HTTP status code, and latency with a comparison between new and old

• Advanced logging with grep and JSON search

• Intelligent correlation between events, logs for faster triangulation of issue

• Auto issue identification

Enterprise-Grade Security and Compliances

• Fine-grained access control; control who can edit the configuration and who can deploy.

• Audit log to know who did what and when

• History of all CI and CD events

• Kubernetes events impacting application

• Relevant cloud events and their impact on applications

• Advanced workflow policies like blackout window, branch environment relationship to secure build and deployment pipelines

Implements GitOps

• GitOps exposed through API and UI so that you don't have to interact with git CLI

• GitOps backed by Postgres for easy analysis

• Enforce finer access control than Git

Operational Insights

• Deployment metrics to measure the success of the agile process. It captures MTTR, change failure rate, deployment frequency, and deployment size out of the box.

• Audit log to understand the failure causes

• Monitor changes across deployments and reverts easily

Compatibility Notes

• Devtron uses a modified version of Argo Rollout.

• Application metrics only work for K8s version 1.16+

Contributing Guidelines

Check out our contributing guidelines. Directions for opening issues, coding standards, and notes on our development processes are all included.

Community

Get updates on Devtron's development and chat with the project maintainers, contributors, and community members.

• Join the Discord Community

• Follow @DevtronL on Twitter

• Raise feature requests, suggest enhancements, report bugs at GitHub issues

• Read the Devtron blog

Vulnerability Reporting

We, at Devtron, take security and our users' trust very seriously. If you believe you have found a security issue in Devtron, please responsibly disclose it by contacting us at security@devtron.ai.

You can install and try Devtron on a high-end machine or a Cloud VM. If you install it on a laptop/PC, it may start to respond slowly, so it is recommended to uninstall Devtron from your system before shutting it down.

Prerequisites

1. 2 vCPUs

2. 4GB+ of free memory

3. 20GB+ free disk space

Before you get started, you must set up a cluster in your server and finish the following actions:

• Create a cluster using Minikube or Kind or K3s.

• Install Helm3.

• Install kubectl.

Tutorial

For Minikube, Microk8s, K3s, Kind

helm repo add devtron https://helm.devtron.ai

helm repo update devtron

helm install devtron devtron/devtron-operator \
--create-namespace --namespace devtroncd \
--set components.devtron.service.type=NodePort --set installer.arch=multi-arch

kubectl apply -f https://raw.githubusercontent.com/rancher/local-path-provisioner/master/deploy/local-path-storage.yaml

helm repo add devtron https://helm.devtron.ai

helm repo update devtron

helm install devtron devtron/devtron-operator \
--create-namespace --namespace devtroncd \
--set components.devtron.service.type=NodePort

Access Devtron Dashboard

To access Devtron dashboard when using Minikube as cluster, run the following command:

minikube service devtron-service --namespace devtroncd

To access Devtron dashboard when using Kind/k3s as cluster, run the following command to port forward the devtron service to port 8000:

kubectl -n devtroncd port-forward service/devtron-service 8000:80

Dashboard: http://127.0.0.1:8000.

Get Admin Credentials

When you install Devtron for the first time, it creates a default admin user and password (with unrestricted access to Devtron). You can use that credentials to log in as an administrator.

After the initial login, we recommend you set up any SSO service like Google, GitHub, etc., and then add other users (including yourself). Subsequently, all the users can use the same SSO (let's say, GitHub) to log in to Devtron's dashboard.

The section below will help you understand the process of getting the administrator credentials.

For Devtron version v0.6.0 and higher

Username: admin Password: Run the following command to get the admin password:

kubectl -n devtroncd get secret devtron-secret \
-o jsonpath='{.data.ADMIN_PASSWORD}' | base64 -d

kubectl -n devtroncd get secret devtron-secret \
-o jsonpath='{.data.ACD_PASSWORD}' | base64 -d

For Cloud VM (AWS EC2, Azure VM, GCP VM)

It is recommended to use Cloud VM with 2vCPU+, 4GB+ free memory, 20GB+ storage, Compute Optimized VM type & Ubuntu Flavoured OS.

Create Microk8s Cluster

sudo snap install microk8s --classic --channel=1.22
sudo usermod -a -G microk8s $USER
sudo chown -f -R $USER ~/.kube
newgrp microk8s
microk8s enable dns storage helm3
echo "alias kubectl='microk8s kubectl '" >> .bashrc
echo "alias helm='microk8s helm3 '" >> .bashrc
source .bashrc

Install Devtron

helm repo add devtron https://helm.devtron.ai

helm repo update devtron

helm install devtron devtron/devtron-operator \
--create-namespace --namespace devtroncd \
--set components.devtron.service.type=NodePort 

Get devtron-service Port Number

kubectl get svc -n devtroncd devtron-service -o jsonpath='{.spec.ports[0].nodePort}'

Make sure that the port on which the devtron-service runs remain open in the VM's security group or network security group.

Sample Configuration

Values You Would Require at SSO Provider

Devtron provides a sample configuration out of the box. There are some values that you need to either get from your SSO provider or give to your SSO provider.

Values to Fetch

• clientID

• clientSecret

Values to Provide

• redirectURI (provided in SSO Login Services by Devtron)

Reference

Projects are the logical grouping of your applications so that you can manage and control the access level of users.

Add Project:

1. To add a project name, go to the Projects section of Global Configurations.

2. Click Add Project.

3. Provide a project name in the field and click Save.

Devtron includes predefined helm charts that cover the majority of use cases. For any use case not addressed by the default helm charts, you can upload your own helm chart and use it as a custom chart in Devtron.

• Who can upload a custom chart - Super admins

• Who can use the custom chart - All users

A super admin can upload multiple versions of a custom helm chart.

Prerequisites

1. A valid helm chart, which contains Chart.yaml file with name and version fields.

2. Image descriptor template file .image_descriptor_template.json.

3. Custom chart packaged in the *.tgz format.

1. How to create a helm chart

You can use the following command to create the Helm chart:

Please see the following example:

2. Create the image descriptor template file .image_descriptor_template.json

It's a GO template file that should produce a valid JSON file upon rendering. This file is passed as the last argument in helm install -f myvalues.yaml -f override.yaml command.

Place the .image_descriptor_template.json file in the root directory of your chart.

You can use the following variables in the helm template (all the placeholders are optional):

The values from the CD deployment pipeline are injected at the placeholder specified in the .image_descriptor_template.json template file.

For example:

To create a template file to allow Devtron to only render the repository name and the tag from the CI/CD pipeline that you created, edit the .image_descriptor_template.json file as:

3. Package the custom chart in the *.tgz format

Before you begin, ensure that your helm chart includes both Chart.yaml (with name and version fields) and .image_descriptor_template.json files.

The helm chart to be uploaded must be packaged as a versioned archive file in the format <helm-chart-name>-vx.x.x.tgz.

The above command will create a my-custom-chart-0.1.0.tgz file.

Uploading a custom chart

A custom chart can only be uploaded by a super admin.

• On the Devtron dashboard, select Global Configurations > Custom charts.

• Select Import Chart.

• Select tar.gz file... and upload the packaged custom chart in the *.tgz format.

The chart is being uploaded and validated. You may also Cancel upload if required.

Validation

The uploaded archive will be validated against:

• Supported archive template should be in *.tgz format.

• Content of values.yaml should be there in app-values.yaml file.

• release-values.yaml file is required.

• Chart.yaml must include the name and the version number.

• ..image_descriptor_template.json file should be present and the field format must match the format listed in the image builder template section.

The following are the validation results:

View the custom charts

All users can view the custom charts.

To view a list of available custom charts, go to Global Configurations > Custom charts page.

• The charts can be searched with their name, version, or description.

Use the custom chart in an application

Info:

Authorization section describes how to authenticate and authorize access to resources, also managing role-based access levels in Devtron.

Access can be granted to a user via:

Sample Configuration

Values You Would Require at SSO Provider

Devtron provides a sample configuration out of the box. There are some values that you need to either get from your SSO provider or give to your SSO provider.

Values to Fetch

• clientID

• clientSecret

Values to Provide

• redirectURI (provided in SSO Login Services by Devtron)

Reference

Once Devtron is installed, it has a built-in admin user with super admin privileges with unrestricted access to all Devtron resources. We recommended to use a user with super admin privileges for initial and global configurations only and then switch to local users or configure SSO integration.

To add/edit SSO configuration, go to the SSO Login Services section of Global Configurations.

Supported SSO Providers

Below are the SSO providers which are available in Devtron. Select one of the SSO providers (e.g., GitHub) to configure SSO:

Dex implements connectors that target specific identity providers for each connector configuration. You must have a created account for the corresponding identity provider and registered an app for client key and secret.

Refer the following documents for more detail.

• https://dexidp.io/docs/connectors/

• https://dexidp.io/docs/connectors/google/

1. Create new SSO Configuration

• Go to the Global Configurations → SSO Login Services and click any SSO Provider of your choice.

• In the URL field, enter the valid Devtron application URL where it is hosted.

• For providing redirectURI or callbackURI registered with the SSO provider, you can either select Configuration or Sample Script.

• Provide the client ID and client Secret of your SSO provider (e.g. If you select Google as SSO provider, then you must enter $GOOGLE_CLIENT_ID and $GOOGLE_CLIENT_SECRET in the client ID and client Secret respectively.)

• Select Save to create and activate SSO Login Service.

Note:

• Only single SSO login configuration can be active at one time. Whenever you create or update any SSO configuration, it will be activated and used by Devtron and previous configurations will be deleted.

• Except for the domain substring, URL and redirectURI remains same.

2. Update SSO Configuration

You can change SSO configuration anytime by updating the configuration and click Update. Note: In case of configuration change, all users will be logged out of Devtron and will have to login again.

3. Configuration Payload

• type : Any platform name such as (Google, GitLab, GitHub etc.)

• name : Identity provider platform name

• config : User can put connector details for this key. Platforms may not have same structure but common configurations are clientID, clientSecret, redirectURI.

• hostedDomains : Domains authorized for SSO login.

Next Steps

Sample Configuration

Values You Would Require at SSO Provider

Devtron provides a sample configuration out of the box. There are some values that you need to either get from your SSO provider or give to your SSO provider.

Values to Fetch

• clientID

• clientSecret

Values to Provide

• redirectURI (provided in SSO Login Services by Devtron)

Reference