This page helps you to install Devtron without any integrations. Integrations can be added later using Devtron Stack Manager.

Before you begin

Install Helm if you haven't done that already!

helm repo add devtron https://helm.devtron.ai

helm install devtron devtron/devtron-operator --create-namespace --namespace devtroncd

Devtron dashboard

Use the following command to get the dashboard URL:

kubectl get svc -n devtroncd devtron-service -o jsonpath='{.status.loadBalancer.ingress}'

You will get the result something as shown below:

[test2@server ~]$ kubectl get svc -n devtroncd devtron-service -o jsonpath='{.status.loadBalancer.ingress}'
[map[hostname:aaff16e9760594a92afa0140dbfd99f7-305259315.us-east-1.elb.amazonaws.com]]

The hostname aaff16e9760594a92afa0140dbfd99f7-305259315.us-east-1.elb.amazonaws.com as mentioned above is the Loadbalancer URL where you can access the Devtron dashboard.

You can also do a CNAME entry corresponding to your domain/subdomain to point to this Loadbalancer URL to access it at a custom domain.

Devtron Admin credentials

For admin login, use the username as admin, and run the following command to get the admin password:

kubectl -n devtroncd get secret devtron-secret \
-o jsonpath='{.data.ACD_PASSWORD}' | base64 -d

Cleaning Helm installer

Please make sure that you do not have anything inside namespaces devtroncd, devtron-cd devtron-ci, and devtron-demo as the below steps will clean everything inside these namespaces

helm uninstall devtron --namespace devtroncd

kubectl delete -n devtroncd \
-f https://raw.githubusercontent.com/devtron-labs/charts/main/charts/devtron/crds/crd-devtron.yaml

kubectl delete ns devtroncd

Upgrade

To use the CI/CD capabilities with Devtron, users can Install the CI/CD integration.

After Devtron is installed, Devtron is accessible through service devtron-service. If you want to access devtron through ingress, edit devtron-service and change the loadbalancer to ClusterIP. You can do this using kubectl patch command like :

kubectl patch -n devtroncd svc devtron-service -p '{"spec": {"ports": [{"port": 80,"targetPort": "devtron","protocol": "TCP","name": "devtron"}],"type": "ClusterIP","selector": {"app": "devtron"}}}'

After that create ingress by applying the ingress yaml file. You can use this yaml file to create ingress to access devtron:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  labels:
    app: devtron
    release: devtron
  name: devtron-ingress
spec:
  ingressClassName: nginx
  rules:
  - http:
      paths:
      - backend:
          service:
            name: devtron-service
            port:
              number: 80
        path: /orchestrator
        pathType: ImplementationSpecific 
      - backend:
          service:
            name: devtron-service
            port:
              number: 80
        path: /dashboard
        pathType: ImplementationSpecific
      - backend:
          service:
            name: devtron-service
            port:
              number: 80
        path: /grafana
        pathType: ImplementationSpecific  

You can access devtron from any host after applying this yaml. For k8s versions <1.19, apply this yaml:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  labels:
    app: devtron
    release: devtron
  name: devtron-ingress
spec:
  rules:
  - http:
      paths:
      - backend:
          serviceName: devtron-service
          servicePort: 80
        path: /orchestrator
      - backend:
          serviceName: devtron-service
          servicePort: 80
        path: /dashboard
      - backend:
          service:
            name: devtron-service
            port:
              number: 80
        path: /grafana
        pathType: ImplementationSpecific  

Optionally you also can access devtron through a specific host like :

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  labels:
    app: devtron
    release: devtron
  name: devtron-ingress
spec:
  ingressClassName: nginx
  rules:
  - http:
      paths:
      - backend:
          service:
            name: devtron-service
            port:
              number: 80
        host: devtron.example.com
        path: /orchestrator
        pathType: ImplementationSpecific 
      - backend:
          service:
            name: devtron-service
            port:
              number: 80
       Host: devtron.example.com
        path: /dashboard
        pathType: ImplementationSpecific
      - backend:
          service:
            name: devtron-service
            port:
              number: 80
        path: /grafana
        pathType: ImplementationSpecific  

Before you begin

Install Helm.

Installing Devtron using Helm

1. Add Devtron repository

2. Install Devtron

helm repo add devtron https://helm.devtron.ai

helm install devtron devtron/devtron-operator \
--create-namespace --namespace devtroncd \
--set installer.modules={cicd}

helm repo add devtron https://helm.devtron.ai

helm install devtron devtron/devtron-operator --create-namespace --namespace devtroncd \
--set installer.modules={cicd} \
--set configs.BLOB_STORAGE_PROVIDER=S3 \
--set configs.DEFAULT_CACHE_BUCKET=demo-s3-bucket \
--set configs.DEFAULT_CACHE_BUCKET_REGION=us-east-1 \
--set configs.DEFAULT_BUILD_LOGS_BUCKET=demo-s3-bucket \
--set configs.DEFAULT_CD_LOGS_BUCKET_REGION=us-east-1

helm repo add devtron https://helm.devtron.ai

helm install devtron devtron/devtron-operator --create-namespace --namespace devtroncd \
--set installer.modules={cicd} \
--set secrets.AZURE_ACCOUNT_KEY=xxxxxxxxxx \
--set configs.BLOB_STORAGE_PROVIDER=AZURE \
--set configs.AZURE_ACCOUNT_NAME=test-account \
--set configs.AZURE_BLOB_CONTAINER_CI_LOG=ci-log-container \
--set configs.AZURE_BLOB_CONTAINER_CI_CACHE=ci-cache-container

Append the command with --set installer.release="vX.X.X" to install a particular version of Devtron. Where vx.x.x is the release tag.

For those countries/users where Github is blocked, you can use Gitee as the installation source:

helm repo add devtron https://helm.devtron.ai

helm install devtron devtron/devtron-operator --create-namespace --namespace devtroncd \
--set installer.source=gitee

If you are planning to use Devtron for production deployments, please refer to our recommended overrides for Devtron Installation.

Check Devtron installation status

The install commands start Devtron-operator, which takes about 20 minutes to spin up all of the Devtron microservices one by one. You can use the following command to check the status of the installation:

kubectl -n devtroncd get installers installer-devtron -o jsonpath='{.status.sync.status}'

The command executes with one of the following output messages, indicating the status of the installation:

Check the installer logs

To check the installer logs, run the following command:

kubectl logs -f -l app=inception -n devtroncd

Devtron dashboard

Use the following command to get the dashboard URL:

kubectl get svc -n devtroncd devtron-service \
-o jsonpath='{.status.loadBalancer.ingress}'

You will get an output similar to the one shown below:

[test2@server ~]$ kubectl get svc -n devtroncd devtron-service \
-o jsonpath='{.status.loadBalancer.ingress}'
[map[hostname:aaff16e9760594a92afa0140dbfd99f7-305259315.us-east-1.elb.amazonaws.com]]

The hostname aaff16e9760594a92afa0140dbfd99f7-305259315.us-east-1.elb.amazonaws.com as mentioned above is the Loadbalancer URL where you can access the Devtron dashboard.

If you don't see any results or receive a message that says "service doesn't exist," it means Devtron is still installing; please check back in 5 minutes.

Note: You can also do a CNAME entry corresponding to your domain/subdomain to point to this Loadbalancer URL to access it at a custom domain.

Devtron Admin credentials

For admin login, use the username:admin, and run the following command to get the admin password:

kubectl -n devtroncd get secret devtron-secret -o jsonpath='{.data.ACD_PASSWORD}' | base64 -d

Cleaning Devtron Helm installer

Please make sure that you do not have anything inside namespaces devtroncd, devtron-cd, devtron-ci, and devtron-demo as the below steps will clean everything inside these namespaces:

helm uninstall devtron --namespace devtroncd

kubectl delete -n devtroncd -f https://raw.githubusercontent.com/devtron-labs/charts/main/charts/devtron/crds/crd-devtron.yaml

kubectl delete -n argo -f https://raw.githubusercontent.com/devtron-labs/devtron/main/manifests/yamls/workflow.yaml

kubectl delete ns devtroncd devtron-cd devtron-ci devtron-demo

FAQs

kubectl -n devtroncd get installers installer-devtron -o jsonpath='{.status.sync.status}'

pod=$(kubectl -n devtroncd get po -l app=inception -o jsonpath='{.items[0].metadata.name}')&& kubectl -n devtroncd logs -f $pod

cd devtron-installation-script/
kubectl delete -n devtroncd -f yamls/
kubectl -n devtroncd patch installer installer-devtron --type json -p '[{"op": "remove", "path": "/status"}]'

Still facing issues, please reach out to us on Discord.

Devtron is installed over a Kubernetes cluster and can be installed standalone or along with CI/CD integration:

• Devtron with CI/CD: Devtron installation with the CI/CD integration is used to perform CI/CD, security scanning, GitOps, debugging, and observability.

• Devtron: The Devtron installation includes functionalities to deploy, observe, manage, and debug existing Helm applications in multiple clusters and deeply integrate with multiple tools using extensions.

Recommended resources

The minimum requirements for Devtron and Devtron with CI/CD integration in production and non-production environments include:

• Non-production

• Production (assumption based on 5 clusters)

Refer to the Override Configurations section for more information.

Before you begin

Create a Kubernetes cluster (preferably K8s 1.16 or higher) if you haven't done that already!

Refer to the Creating a Production grade EKS cluster using EKSCTL article to set up a cluster in the production environment.

Installing Devtron

• Install Devtron with CI/CD integration

• Install Devtron

• Upgrade Devtron to latest version

Git Accounts allow you to connect your code source with Devtron. You will be able to use these git accounts to build the code using the CI pipeline.

Git Account Configuration

Global Configuration helps you to add a Git provider. Click on Add git account button at the top of the Git Account Section. To add a new git provider, add the details as mentioned below.

1. Name

2. Git Host

3. URL

4. Authentication type

1. Name

Provide a Name to your Git provider. This name will be displayed in the the Git Provider drop-down inside the Git Material configuration section.

2. Git Host

It is the git provider on which corresponding application git repository is hosted. By default you will get Bitbucket and GitHub but you can add many as you want clicking on [+ Add Git Host].

3. URL

4. Authentication type

Here provide the type of authentication required by your version controller. Devtron supports three types of authentications. You can choose the one that suits you the best.

• Anonymous

If authentication type is set as Anonymous then you do not need to provide any username, password/authentication token or SSH key. Just click on Save to save the git account provider details.

If authentication type is set as Anonymous, only public git repository will be accessible.

• User Auth

If you select User Auth then you have to provide the Username and either of Password or Auth Token for the authentication of your version controller account. Click on Save to save the git account provider details.

• SSH Key

If you choose SSH Key then you have to provide the Private SSH Key corresponding to the public key added in your version controller account. Click on Save to save the git account provider details.

Update Git Account

You can update your saved git account settings at anytime. To update the git account:

1. Click on the git account which you want to update.

2. Make the required changes

3. Click on Update to save the changes.

Updates can only be made within one Authentication type or one protocol type, i.e. HTTPS(Anonymous or User Auth) & SSH. You can update from Anonymous to User Auth & vice versa, but not from Anonymous/User Auth to SSH or reverse.

Note:

Disabled git accounts will be unavailable for use in future applications. Applications already using a disabled git account will not be affected.

API tokens are like ordinary OAuth access tokens. They can be used instead of username and password for programmatic access to API. API token allows users to generate API tokens with the desired access. Only super admin users can generate tokens and see generated tokens.

Generate API Token

To generate API tokens, go to global configurations -> Authorizations -> API tokens and click on Generate New Token.

1. Enter a name for the token

2. Add Description.

3. Select an Expiration date for the token(7 days, 30 days, 60 days, 90 days, custom and no expiration) To select a custom expiration date, select Custom from the drop-down. This will pop-up a calender from where you can select your custom expiration date for the API token.

4. Assign permissions to the token. To generate a token with super admin permission, select super admin permission. Or select specific permission if you want to generate a token with a specific role over a particular Devtron app or Helm app or chart group.

5. Now click on Generate Token.

A pop-up window will appear over the screen from where you can copy the API token.

Use API Token

Once devtron api token has been generated, you can use this token to hit devtron apis using any api testing tool like Jmeter, postman, citrus. Using postman here.

Open postman. Enter the request URL with POST method and under HEADERS, enter the API token as shown in the image below.

Now, under body, provide the api payload as shown below and click on Send.

As soon as you click on send, the create application api will be triggered and a new devtron app will be created as you mentioned in the payload.

Update API Token

To set a new expiration date or to make changes in permissions assigned to the token, we need to update the API token. To update the API token, click over the token name or click on the edit icon.

Delete API Token

To delete an API token, click on the delete icon. Any applications or scripts using this token will no longer be able to access the Devtron API.

This feature allows you to add more chart repositories to Devtron. Once added they will be available in the Discover section of the Chart Store.

Learn more about

Note : After the successfull installation of Devtron, click on Refresh Charts to sync & download all the default charts listed on the dashboard.

Add Chart Repository

Select the Chart Repository section of global configuration and click on Add Repository button at the top of the Chart Repository Section. To add new chart, you need to provide three inputs as below:

1. Name

2. URL

3. Authentication type

1. Name

Provide a Name to your Chart Repository. This name is added as prefix to the name of the chart in the listing on the helm chart section of application.

2. URL

3. Authentication type

Here you have to provide the type of Authentication required by your version controller. We support three types of authentications, You can choose the one that suits you the best.

• Anonymous

If you select Anonymous then you do not have to provide any username, password, or authentication token. Just click on Save to save your chart repository details.

• Password/Auth token

If you select Password/Auth token, then you have to provide the Access Token for the authentication of your version controller account inside the Access token box. Click on Save to save your chart repository details.

• User Auth

If you choose User Auth then you have to provide the Username and Password of your version controller account. Click on Save to save your chart repository details.

Update Chart Repository

You can update your saved chart repository settings at any point in time. Just click on the chart repository which you want to update. Make the required changes and click on Update to save you changes.

Note: You can enable and disable your chart repository setting. If you enable it, then you will be able to see that enabled chart in Discover section of Chart Store.

This documentaion consist of authorizations available in Devtron

Parts of the documentaion

Hurray! Your Devtron stack is completely setup. Let's get started by deploying a simple application on it.

Find out the steps here

This is a sample Nodejs application which we are going to deploy using Devtron. For a detailed step-wise procedure, please have a look at the link below -

Getting Started with Deploying application through devtron

1. Job

A Job creates one or more Pods and will continue to retry execution of the Pods until a specified number of them successfully terminate. As pods successfully complete, the Job tracks the successful completions. When a specified number of successful completions is reached, the task (ie, Job) is complete. Deleting a Job will clean up the Pods it created. Suspeding a Job will delete its active Pods until the Job is resumed again.

Example:

kind: Job
jobConfigs:
    activeDeadlineSeconds: 120
    backoffLimit: 6
    completions: 1
    parallelism: 1
    suspend: false
    ttlSecondsAfterFinished: 100

2. Cronjob

A Cronjob creates Jobs on a repeating schedule , One Cronjob object is like one line of a crontab (cron table) file. It runs a job periodically on a given schedule, written in Cron format. Cronjobs are meant for performing regular scheduled actions such as backups, report generation, and so on. Each of those tasks should be configured to recur indefinitely (for example: once a day / week / month); you can define the point in time within that interval when the job should start.

Example:

kind: CronJob
cronjobConfigs:
    concurrencyPolicy: Allow
    failedJobsHistoryLimit: 1
    restartPolicy: OnFailure
    schedule: 32 8 * * *
    startingDeadlineSeconds: 100
    successfulJobsHistoryLimit: 3
    suspend: false

A deployment configuration is a manifest for the application. It defines the runtime behavior of the application.

Devtron includes deployment template for both default as well as custom charts created by a super admin.

To configure a deployment chart for your application:

• Go to Applications and create a new application.

• Go to App Configuration page and configure your application.

• On the Deployment Template page, select the drop-down under Chart type.

You can select a chart in one of the following ways:

Select chart by Devtron

1. Rollout Deployment

2. Job and Cronjob

3. Knative

Select chart from custom charts

Custom charts are added by a super admin from the custom charts section.

Users can select the available custom charts from the drop-down list.

Upload custom chart

A custom chart can be uploaded by a super admin.

Application Metrics

Enable show application metrics toggle to view the application metrics on the App Details page.

IMPORTANT: Enabling Application metrics introduces a sidecar container to your main container which may require some additional configuration adjustments, we recommend you to do load test after enabling it in a non-prod environment before enabling it in production environment.

Select Save to save your configurations.

Users can run the test case using the Devtron dashboard or by including the test cases in the devtron.ci.yaml file in the source git repository. For reference, check:

The test cases given in the script will run before the test cases given in the devtron.ci.yaml

Secrets and configmaps both are used to store environment variables but there is one major difference between them: Configmap stores key-values in normal text format while secrets store them in base64 encrypted form. Devtron hides the data of secrets for the normal users and it is only visible to the users having edit permission.

Secret objects let you store and manage sensitive information, such as passwords, authentication tokens, and ssh keys. Embedding this information in secrets is safer and more flexible than putting it verbatim in a Pod definition or in a container image.

Configure Secret

Click on Add Secret to add a new secret.

Volume Mount Path

Specify the volume mount folder path in Volume Mount Path, a path where the data volume needs to be mounted. This volume will be accessible to the containers running in a pod.

Sub Path

For multiple files mount at the same location you need to check sub path bool field, it will use the file name (key) as sub path. Sub Path feature is not applicable in case of external configmap except AWS Secret Manager, AWS System Manager and Hashi Corp Vault, for these cases Name (Secret key) as sub path will be picked up automatically.

File Permission

File permission will be provide at the configmap level not on the each key of the configmap. it will take 3 digit standard permission for the file.

Click on Save Secret to save the secret.

You can see the Secret is added.

Update Secrets

You can update your secrets anytime later, but you cannot change the name of your secrets. If you want to change your name of secrets then you have to create a new secret.

To update secrets, click on the secret you wish to update.

Click on Update Secret to update your secret.

Delete Secret

You can delete your secret. Click on your secret and click on the delete sign to delete your secret.

Data Types

There are five Data types that you can use to save your secret.

• Kubernetes Secret: The secret that you create using Devtron.

• Kubernetes External Secret: The secret data of your application is fetched by Devtron externally. Then the Kubernetes External Secret is converted to Kubernetes Secret.

• AWS Secret Manager: The secret data of your application is fetched from AWS Secret Manager and then converted to Kubernetes Secret from AWS Secret.

• AWS System Manager: The secret data for your application is fetched from AWS System Secret Manager and all the secrets stored in AWS System Manager are converted to Kubernetes Secret.

• Hashi Corp Vault: The secret data for your application is fetched from Hashi Corp Vault and the secrets stored in Hashi Corp Vault are converted to Kubernetes Secret.

Note: The conversion of secrets from various data types to Kubernetes Secrets is done within Devtron and irrespective of the data type, after conversion, the Pods access secrets normally.

External Secrets

In some cases, it may be that you already have secrets for your application on some other sources and you want to use that on devtron. External secrets are fetched by devtron externally and then converted to kubernetes secrets.

Kubernetes External Secret

The secret that is already created and stored in the environment and being used by devtron externally is referred here as Kubernetes External Secret. For this option, devtron will not create any secret by itself but they can be used within the pods. Before adding secret from kubernetes external secret, please make sure that secret with the same name is present in the environment. To add secret from kubernetes external secret, follow the steps mentioned below:

1. Navigate to Secrets of the application.

2. Click on Add Secret to add a new secret.

3. Select Kubernetes External Secret from dropdown of Data type.

4. Provide a name to your secret. Devtron will search secret in the environment with the same name that you mention here.

AWS Secret Manager

Before adding any external secrets on devtron, kubernetes-external-secrets must be installed on the target cluster. Kubernetes External Secrets allows you to use external secret management systems (e.g., AWS Secrets Manager, Hashicorp Vault, etc) to securely add secrets in Kubernetes.

Installing kubernetes-external-secrets Using Chart

To install the chart with the release named my-release:

$ helm install my-release external-secrets/kubernetes-external-secrets

To install the chart with AWS IAM Roles for Service Accounts:

$ helm install my-release external-secrets/kubernetes-external-secrets --set securityContext.fsGroup=65534 --set serviceAccount.annotations."eks\.amazonaws\.com/role-arn"='arn:aws:iam::111111111111:role/ROLENAME'

Adding Secrets From AWS Secret Manager

To add secrets from AWS secret manager, navigate to Secrets of the application and follow the steps mentioned below :

1. Click on Add Secret to add a new secret.

1. Select AWS Secret Manager from dropdown of Data type.

2. Provide a name to your secret.

3. Select how you want to use the secret. You may leave it selected as environment variable and also you may leave Role ARN empty.

4. In Data section, you will have to provide data in key-value format.

All the required field to pass your data to fetch secrets on devtron are described below :

Adding Secrets in AWS Secret Manager

To add secrets in AWS secret manager, do the following steps :

1. Go to AWS secret manager console.

2. Click on Store a new secret.

3. Add and save your secret.

You will see all your environments associated with an application under the Environment Overrides section.

You can customize your Deployment template, ConfigMap, Secrets in Environment Overrides section to add separate customizations for different environments such as dev, test, integration, prod, etc.

Deployment template - Functionality

If you want to deploy an application in a non-production environment and then in production environment once testing is done in non-prod environment, then you do not need to create a new application for prod environment. Your existing pipeline(non-prod env) will work for both the environments with little customization in your deployment template under Environment overrides.

Example customization:

In a Non-production environment, you may have specified 100m CPU resources in the deployment template but in the Production environment you may want to have 500m CPU resources as the traffic on Pods will be higher than traffic on non-prod env.

Configuring the Deployment template inside Environment Overrides for a specific environment will not affect the other environments because Environment Overrides will configure deployment templates on environment basis. And at the time of deployment, it will always pick the overridden deployment template if any.

If there are no overrides specified for an environment in the Environment Overrides section, the deployment template will be the one you specified in the deployment template section of the app creation.

(Note: This example is meant only for a representational purpose. You can choose to add any customizations you want in your deployment templates in the Environment Overrides tab)

Any changes in the configuration will not be added to the template, instead, it will make a copy of the template and lets you customize it for each particular environment. And now this overridden template will be used only for the specified Environment.

This will save you the trouble to manually create deployment files separately for each environment. Instead, all you have to do is to change the required variables in the deployment template.

How to add Environment Overrides

In the Environment Overrides section, click on Allow Override and make changes to your Deployment template and click on Save to save your changes of the Deployment template.

ConfigMaps & Secrets

The same goes for ConfigMap and Secrets. You can also create an environment-specific configmap and Secrets inside the Environment override section.

Click on Update ConfigMap to update Configmaps.

Click on Update Secrets to update Secrets.

Introduction

In this application, you will learn about how to create a Expressjs Application that connects to mongoDb.

Follow the below-mentioned steps, to deploy the application on Devtron using mongoDb Helm Chart.

1. Deploy mongoDb Helm Chart

To deploy mongoDb Helm Chart, you can refer to our documentation on Deploy mongoDb Helm Chart

2. Fork the Git Repository

For this example, we are using the following GitHub Repo, you can clone this repository and make following changes in the files.

*Dockerfile

This is the Dockerfile. This exposes our expressjs application to port number 8080

FROM node:7
WORKDIR /app
COPY package.json /app
RUN npm install
COPY . /app
CMD node server.js
EXPOSE 8080

*db.js File

This file will be used to connect to our database. This will include the service-name of the mongoDb Helm Chart, that you have deployed in Step1.

The syntax is as follows:

<service-name>:27017/<database-name>

This maps our service name to mongoDb's port number 27017.

module.exports = {
    DB: 'mondo-dev-mongodb-replicaset-client:27017/sale'
}

3. Create Application on Devtron

To learn how to create an application on Devtron, refer to our documentation on Creating Application

*Git Repository

In this example, we are using the url of the forked Git Repository.

*Docker configuration

Give, the path of the Dockerfile.

*Configure Deployment Template

Enable Ingress, and give the path on which you want to host the application.

*Set up the CI/CD Pipelines

Set up the CI/CD pipelines. You can set them to trigger automatically or manually.

*Trigger Pipelines

Trigger the CI Pipeline, build should be Successful, then trigger the CD Pipeline, deployment pipeline will be initiated, after some time the status should be Healthy

4. Final Step

Check the Expressjs app connected to mongodb database, running successfully by hitting your application url.

The syntax is: http://<hostname>/<path>/

path will be the one that you have given in Step 3 while configuring the Deployment Template.

The output of our application would be as follows:

You can see that we are getting the JSON response. We have successfully connected our expressjs application to the mongoDb database.

We always try to make your experience of using Devtron as smooth as possible but still if you face any issues, follow the troubleshooting guide given below or join our discord channel if you couldn't find the solution for the issue you are facing.

1. How to resolve unauthorized errors, while trying to save global configurations like hostname, GitOps etc. after successful devtron installation

This occurs most of the time because any one or multiple jobs get failed during installation. To resolve this, you'll need to first check which jobs have failed. Follow these steps:

• Run the following command and check which are the jobs with 0/1 completions:

kubectl get jobs -n devtroncd

• Note down or remember the names of jobs with 0/1 completions and check if their pods are in running state still or not by running the command: kubectl get pods -n devtroncd

• If they are in running condition, please wait for the jobs to be completed as it may be due to internet issue and if not in running condition, then delete those incomplete jobs using: kubectl delete jobs -n devtroncd

• Now download migrator.yaml file from our github repository using the command: wget https://raw.githubusercontent.com/devtron-labs/devtron/main/manifests/yamls/migrator.yaml

• Now edit the file you downloaded in step 3 and remove the postgresql-migrator secret resource creation and then apply the yaml file using the command: kubectl apply -f migrator.yaml -n devtroncd

• It will re-create the failed jobs and you’ll see their pods created again. Just wait for a few minutes until the jobs gets completed then you are good to go. You should be able to save your global configurations now.

2. Not able to see deployment metrics on production environment or Not able to enable application-metrics or Not able to deploy the app after creating a configmap or secret with data-volume option enabled

Update the rollout crds to latest version, run the following command:

kubectl apply -f https://raw.githubusercontent.com/devtron-labs/devtron/main/manifests/yamls/rollout.yaml -n devtroncd

3. SSO Login not working even after entering correct SSO Credentials

Delete devtron pod once to reload the configurations using:

kubectl delete pod -n devtroncd -l app=devtron

4. Logs are not Visible on UI while running the build and not even able to abort the same

Check if the pods are being created when you start a new build, run the command and look if a new pod is created when you started the build:

kubectl get pods -n devtron-ci

If yes, delete kubewatch and devtron pod so that kubewatch can restart and start sharing the logs again:

kubectl delete pod -n devtroncd -l app=devtron; kubectl delete pod -n devtroncd -l app=kubewatch

Wait for 5 minutes and then trigger a new build again, if still not resolved then run the following commands one by one

kubectl delete pod -n devtroncd devtron-nats-0
kubectl delete pod -n devtroncd devtron-stan-0
kubectl delete pod -n devtroncd -l app=devtron
kubectl delete pod -n devtroncd -l app=kubewatch

Again wait for 5 minutes and your issue should be resolved

5. Grafana dashboards not visible in App Details page even after adding prometheus endpoint or Graphs showing error panel with id 2 not found

If the graphs are not visible check if prometheus is configured properly. Then go to Global Configurations > Clusters & Environments > Click on any environment for the cluster where you added prometheus endpoint and simply click Update. If the charts are still not visible, try visiting the url: /grafana?orgId=2 If you see Not Found on this page, then follow all the given steps or if the page is accessible and you are getting panel with id 2 not found then follow from step 6:

1. Get grafana password using kubectl -n devtroncd get secret devtron-secret -o jsonpath='{.data.GRAFANA_PASSWORD}' | base64 -d

2. kubectl run --rm -it --image quay.io/devtron/k8s-utils:tutum-curl curl Run this command and it will create a pod for using curl

3. Copy the following and change grafana-password with your password of grafana and change the value of prometheusUrl with your prometheus endpoint

cat << EOF
grafanaUrl = "http://admin:grafana-password@devtron-grafana.devtroncd/grafana"
prometheusUrl = "http://prometheus.example.com"

ORG_ID=$( curl -d '{"name":"devtron-metrics-view"}' -H "Content-Type: application/json" -X POST "${grafanaUrl}/api/orgs" )

echo $ORG_ID

curl -X POST "${grafanaUrl}/api/user/using/2";

curl -X PUT -H "Content-Type: application/json" -d '{"homeDashboardId":0,"theme":"light","timezone":"browser"}' "${grafanaUrl}/api/org/preferences";

curl "${grafanaUrl}/api/datasources" -H 'content-type: application/json' -H 'x-grafana-org-id: 2' --data '{"name":"Prometheus-devtron-demo","type":"prometheus","access":"proxy","isDefault":true}'

curl "${grafanaUrl}/api/datasources/2" -X PUT \
    -H 'content-type: application/json' \
    -H 'x-grafana-org-id: 2' \
    --data '{"id": 2 ,
    "orgId": 2,
    "name":"Prometheus-devtron-demo","type":"prometheus","access":"proxy",
    "url":${prometheusUrl},
    "basicAuth":false,"jsonData":{},"version":1}'
EOF

and run in the pod that we created above in step 2. 4. Now visit /grafana?orgId=2 again and you'll see grafana login page. Login using username admin and password from step 1 and check if prometheus url is updated in datasources. If not, update it in the default datasource. 5. Now from devtron UI, update any of the environment again and it's datasource will be created automatically. 6. In Grafana UI you need to be logged in and Go to Dashboards > Manage then click Import and Import the given dashboards one by one.

https://grafana.com/api/dashboards/13322/revisions/4/download
https://grafana.com/api/dashboards/13320/revisions/4/download
https://grafana.com/api/dashboards/13325/revisions/4/download
https://grafana.com/api/dashboards/13321/revisions/6/download

After that, your issue should be resolved and you should be able to see all the graphs on UI.

6. Unable to Login Devtron dashboard even if the password is correct

If you are not able to login into Devtron dashboard even after giving the correct password, it is possible that the argocd token of previous session has been stored in the cookies and is not able to override the new token that is generated for the new session. If you are facing this issue, follow the steps below -

If using Firefox -

1. Goto login page of Devtron and open inspect.

2. Navigate to storage tab in inspect.

3. Click on url where Devtron has been installed under Cookies tab and you could see an argocd token with its value, something similar to below image.

1. Now right click on token, and click on Delete All Session Cookies option.

If using Chrome -

1. Goto login page of Devtron and open inspect.

2. Navigate to Application tab, and under Storage tab click on Cookies.

3. Click on url under Cookie and you would be able tto see an argocd token with its value, as shown in the image below.

1. Now right click on token and click on delete option.

If using Safari -

1. Goto Safari preferences >> Advanced options and check the show develop menu as shown in the image below.

1. Now goto login page of Devtron and press option+command+I. It will open inspect element.

2. Then navigate to Storage, click on Cookies and you would be able to see an argocd token with its value as shown in the image below.

1. Now right click on token and select delete option.

After clearing Cookies, try again to login, you should be able to login now.

7. No charts found in Charts Discover Section

In the Devtron's Discover Chart section, if you are not able to see any charts available, goto Global Configuration >> Chart Repositories and click on Refresh Chart at the top-right as shown in the image below. After clicking the button, it might take 4-5mins to show all the charts in Discover section depending upon the chart repositories added.

8. Not able to update cluster

In Global Configurations >> Cluters & Environments, if you try to update a cluster which has been already added in Devtron, you might get an error as {"message":"Failed to update datasource. Reload new version and try again"}. If you are facing such issue, please follow the following steps -

1. Edit the changes you want to make in respective cluster

2. Click on save after making changes and you may get error message stated above.

3. Go to cluster where devtron has been installed and execute - kubectl -ndevtroncd delete po -l app=devtron

4. Now refresh the page and you should be able to save it.

[Note: If you already have created some environments in that cluster, it needs to be updated again]

9. Postgresql is in crashloop with error - Failed to pull image

There may be some other pods also in crashloop as they are not able to connect to database. To resolve this issue, you can either update devtron to latest version or run the following commands to fix instantly on the same version you are using:

kubectl patch -n devtroncd statefulset postgresql-postgresql -p '{"spec":{"template":{"spec":{"initContainers":[{"name":"init-chmod-data","image":"quay.io/devtron/minideb:latest"}],"containers":[{"name":"postgresql-postgresql","image":"quay.io/devtron/postgres:11.3.0-debian-9-r28"}]}}}}'

Then delete postgresql pod so that it can fetch the updated images:

kubectl delete pod -n devtroncd postgresql-postgresql-0

You can also delete other pods which are in crashloop after postgresql is up and running so that they can restart and connect to postgresql and Devtron will be up and running again in a few moments.

10. Unable to fetch the latest commit and not able to trigger auto build.

To solve this, bounce the git-sensor-0 pod.

kubectl delete pod -n devtroncd git-sensor-0

11. If you have restricted devtron-service to be accessible on certain IPs only and SSO login isn’t working

Whitelist the NAT-gateway IPs of the cluster (There can be multiple NAT-gateways if your cluster is multi-AZ)

12. If CPU metrics are not showing but memory metrics are visible in graphs.

Do the following:-

1. Go to Grafana and Login with the credentials.

2. Edit the CPU graphs and remove image!=”” from the query.

3. Save the dashboard.

CPU metrics should start showing up in a while.

13. If user not able to upload a file more than specific size.

Please use below annotation in ingress

nginx.ingress.kubernetes.io/proxy-body-size: 100m

Note:- Where m is is MiB.

14. If AWS Load balancer controller is unable to provision ALB and getting message in alb controller as unauthorized, attach these IAM policy to the nodegroup IAM Role.

IAM policy

15. When app metrics is not coming on grafana and devtron dashboard, set the value of the following parameter as false in kube prometheus stack values.

serviceMonitorSelectorNilUsesHelmValues: false

16. Unable to deploy metrics-server using chart on devtron

To solve

Disable certificate validation by passing --kubelet-insecure-tls argument to metrics server chart.

17. Unable to delete a database from postgres

Description of issue

ERROR: database <db-name> is being accessed by other users

DETAIL: There is 1 other session using the database.

You have to terminate the connections to the database first, for that you can use the command.

SELECT pg_terminate_backend(pg_stat_activity.pid) FROM pg_stat_activity WHERE pg_stat_activity.datname = 'TARGET_DB';

Then run the command to delete database - drop databases <db-name>

18. Unable to login with the auth password (argocd server)

Debug

Run the command for Admin Credentials and use it for login in dashboard

kubectl -n devtroncd get secret devtron-secret -o jsonpath='{.data.ACD_PASSWORD}' | base64 -d

If you are getting an error message of “invalid username or password”, follow the solution to solve it.

Solution

Run kubectl get secret -n devtroncd and then edit the argocd-secret, remove both the admin.password lines.

Run kubectl delete po your-argocd-server-pod -n devtroncd, it will create a new pod after deletion and reset your admin password. Re-run the command for admin credentials again to get the new password.

19. After installing Devtron using helm, getting the admin password does not work.(if using windows)

Debug

'base64' is not recognized as an internal or external command, operable program or batch file.

Solution

The first way to debug is either install base64 encode and decode into your windows machine and use the appropriate cmd to get the admin password.

The other way is to get the password in the encoded form using the cmd

kubectl -n devtroncd get secret devtron-secret -o jsonpath='{.data.ACD_PASSWORD}', further decode it into plaintext using an online encoder decoder.

20. Getting UPGRADE FAILED: cannot patch "postgresql-postgresql" while upgrading Devtron to newer versions

Debug:

1. Make sure to annotate and label all the Devtron resources.

2. Description of error

Error: UPGRADE FAILED: cannot patch "postgresql-postgresql" with kind StatefulSet: StatefulSet.apps "postgresql-postgresql" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'template', 'updateStrategy' and 'minReadySeconds' are forbidden

Solution: Verify if annotations & lables are set to all k8s resources in devtroncd namespace and add --set components.postgres.persistence.volumeSize=20Gi parameter in Devtron upgrade command.

helm upgrade devtron devtron/devtron-operator --namespace devtroncd \
-f https://raw.githubusercontent.com/devtron-labs/devtron/main/charts/devtron/devtron-bom.yaml \
--set installer.modules={cicd} --reuse-values \
--set components.postgres.persistence.volumeSize=20Gi

Configuration

Configure Secrets

For helm installation this section referes to secrets.env section of values.yaml. For kubectl based installation it refers to kind: secret in install/devtron-operator-configs.yaml.

Configure the following properties:

Configure ConfigMaps

For helm installation this section refers to configs section of values.yaml. For kubectl based installation it refers to kind: ConfigMap in install/devtron-operator-configs.yaml.

Configure the following properties:

Storage for Logs and Cache

AWS SPECIFIC

While installing Devtron and using the AWS-S3 bucket for storing the logs and caches, the below parameters are to be used in the ConfigMap.

NOTE: For using the S3 bucket it is important to add the S3 permission policy to the IAM role attached to the nodes of the cluster.

AZURE SPECIFIC

While installing Devtron using Azure Blob Storage for storing logs and caches, the below parameters will be used in the ConfigMap.

NOTE: For using the storage containers it is mandatory to enable versioning on the storage account. Refer this guide to enable the same.

To convert string to base64 use the following command:

echo -n "string" | base64 -d

Note:

1. Ensure that the cluster has read and write access to the S3 buckets/Azure Blob storage container mentioned in DEFAULT_CACHE_BUCKET, DEFAULT_BUILD_LOGS_BUCKET or AZURE_BLOB_CONTAINER_CI_LOG, or AZURE_BLOB_CONTAINER_CI_CACHE.

2. Ensure that the cluster has read access to AWS secrets backends (SSM & secrets manager).

The following tables contain parameters and their details for Secrets and ConfigMaps that are configured during the installation of Devtron. While installing Devtron using kubectl the following parameters can be tweaked in devtron-operator-configs.yaml file. If the installation is proceeded using helm3, the values can be tweaked in values.yaml file.

We can use the --set flag to override the default values when installing with Helm. For example, to update POSTGRESQL_PASSWORD and BLOB_STORAGE_PROVIDER, use the install command as:

helm install devtron devtron/devtron-operator --create-namespace --namespace devtroncd \
--set secrets.POSTGRESQL_PASSWORD=change-me \
--set configs.BLOB_STORAGE_PROVIDER=S3 \

Secrets

ConfigMaps

Dashboard Configurations

RECOMMEND_SECURITY_SCANNING=false
FORCE_SECURITY_SCANNING=false
HIDE_DISCORD=false

Overview

Once installed Devtron has one built-in admin user with super-admin privileges that has complete access to the system. It is recommended to use admin user only for initial and global configuration and then switch to local users or configure SSO integration.

Only users with super-admin privileges have access to create SSO configuration. Devtron uses dex for authenticating a user against the identity provider.

To add/edit SSO configuration please go to the left main panel -> Select Global Configurations -> Select SSO Login Services

Supported SSO Providers

LDAP GitHub OpenID Connect Google Microsoft OpenShift

Dex implements connectors that target specific identity providers, for each connector configuration user must have created account for the corresponding identity provider and registered an app for client key and secret. For examples see

• https://dexidp.io/docs/connectors/

• https://dexidp.io/docs/connectors/google/

1. Create new SSO Configuration

Login as a user with super-admin privileges and go to Global Configurations -> SSO Login Services and click on any Identity Provider and fill the configuration.

Add valid devtron application URL where it is hosted.

Fill correct redirect URL or callback URL from which you have registered with the identity provider in the previous step along with the client id and client secret shared by the identity provider.

Only single SSO login configuration can be active at one time. Whenever you create or update any SSO config, it will be activated and used by the system and previous configurations will be deleted.

Except for the domain substring, URL and redirectURI should be the same as in the screenshots.

Select Save to create and activate SSO login.

2. Update SSO Configuration

SSO configuration can be changed by the user at any later point in time by updating the configuration and clicking on the Save button at the bottom right. In case of configuration change all users will be logged out of the system and will have to login again.

3. Configuration Payload

• type : oidc or any platform name such as (google, gitlab, github etc)

• name : identity provider platform name

• id : identity provider platform unique id in string. (refer to dexidp.io)

• config : user can put connector details into this key. platforms may not have same structure but commons are clientID, clientSecret, redirectURI.

• hostedDomains : domains authorized for SSO login.

Introduction

Let's assume that you are creating an application and want to use mongodb to store data of your application. You can deploy mongodb using stable/mongodb-replicaset Helm chart and connect it to your application.

This guide will introduce you to how to deploy the mongoDB's Helm chart.

1. Discover the Chart from the Chart Store

Visit the Chart Store page by clicking on Charts present on left panel and find stable/mongodb-replicaset Helm Chart. You also can search mongodb chart using the search bar.

2. Configure the Chart

After selecting the stable/mongodb Helm chart, click on Deploy.

Enter the following details before deploying the mongoDB chart:

Configure values.yaml

You can configure the values.yaml according to your project's requirements. To learn about different parameters used in the chart, you can check Documentation of mongodb Helm chart

Click on Deploy Chart once you have finished configuring the chart.

3. Check the Status of Deployment

After clicking on Deploy Chart, you will be redirected to App Details page that shows the deployment status of the chart. The Status of the chart should be Healthy. It might take few seconds after initiating the deployment.

In case the status of the deployment is Degraded or takes a long time to get deployed, click on Status or check the logs of the pods to debug the issue.

4. Extract the Service name

Copy the service name, it will be used to connect your application to mongoDB.

This feature helps you to update Deployment Template, ConfigMaps & Secrets for multiple apps in one go! You can filter the apps on the basis of environments, global flag, and app names(we provide support for both substrings included and excluded in the app name).

Overview

Need to make some common changes across multiple devtron applications? Bulk Edit allows you to do that. Eg. You can change the value for MaxReplicas in Deployment Templates of multiple Devtron applications or you can add key-value pairs in multiple ConfigMaps & Secrets.

Support

Bulk edit is currently supported for:

• Deployment Template

• ConfigMaps

• Secrets

Steps:

1. Click on the Bulk Edit option in the main navigation. This is where you can write and execute scripts to perform bulk updates in Devtron objects.

1. To help you get started, a script template is provided under the See Samples section.

1. Copy and Paste the Sample Script in the code editor and make desired changes. Refer Payload Configuration in the Readme to understand the parameters.

Example

Example below will select all applications having abc and xyz present in their name and out of those will exclude applications having abcd and xyza in their name. Since global flag is false and envId 23 is provided, it will make changes in envId 23 and not in global deployment template for this application.

If you want to update globally then please set global: true. If you have provided an envId but deployment template, configMap or secret is not overridden for that particular environment then it will not apply the changes. Also, of all the provided names of configMaps/secrets, for every app & environment override only the names that are present in them will be considered.

Sample Script

This is the piece of code which works as the input and has to be pasted in the code editor for achieving bulk updation task.

apiVersion: batch/v1beta1
kind: Application
spec:
  includes:
    names:
    - "%abc%"
    - "%xyz%"
  excludes:
    names:
    - "%abcd%"
    - "%xyza%"
  envIds:
  - 23
  global: false
  deploymentTemplate:
    spec:
      patchJson: '[{ "op": "add", "path": "/MaxSurge", "value": 1 },{"op": "replace","path":"/GracePeriod","value": "30"}]'
  configMap:
    spec:
      names:
      - "configmap1"
      - "configmap2"
      - "configmap3"
      patchJson: '[{ "op": "add", "path": "/{key}", "value": "{value}" },{"op": "replace","path":"/{key}","value": "{value}"}]'
  secret:
    spec:
      names:
      - "secret1"
      - "secret2"
      patchJson: '[{ "op": "add", "path": "/{key}", "value": "{value}" },{"op": "replace","path":"/{key}","value": "{value}"}]'

Payload Configuration

The following tables list the configurable parameters of the Payload component in the Script and their description along with example. Also, if you do not need to apply updates on all the tasks, i.e. Deployment Template, ConfigMaps & Secrets, leave the Spec object empty for that respective task.

1. Once you have modified the script, you can click on the Show Impacted Objects button to see the names of all applications that will be modified when the script is Run.

1. Click on the Run button to execute the script. Status/Output of the script execution will be shown in the Output section of the bottom drawer.

Why command bar?

The command bar is designed to enable you to navigate across the devtron dashboard without having to click around the screen.

Top-level categories (eg. app, chart, security, global-config) are auto-filled depending upon your location on the Devtron dashboard.

You can clear the top-level category to navigate within other category locations.

Shortcuts

How to use the command bar (Eg. Navigate to Workflow editor of an App.)

Steps to use

1. Open command bar by clicking the 🔍 search icon on left navbar or pressing Cmd/Ctrl + /

2. Start typing the app name you're looking for.

3. Navigate using ↓ ↑ between the matching results and press → to view nested options.

   Note: Pressing Enter on a highlighted option will navigate to the selected page location.

4. In this case, app / dashboard / configure / workflow-editor will navigate to the Workflow editor in dashboard application.

Similarly, you can use the command bar to navigate around the Devtron dashboard without a click.

We would love to know your experience with the command bar. Jump in to the Devtron Discord Community

This documentation consists of the Global Configurations available in Devtron.

Parts of the Documentation

GitOps

Git Accounts

Container Registries

Chart Repositories

Cluster And Environments

Projects

SSO Login Service

Authorization

Manage Notification

Devtron integrations extend the functionality of your Devtron stack.

Discover and install integrations

The current release of Devtron supports the Build and Deploy (CI/CD) integration. More integrations will be available soon; to request one, please submit a ticket

Integrations can be installed by super admins; However other user roles can browse and request super admins to install the required integrations.

Integrations are updated along with Devtron updates.

Select Devtron Stack Manager from the left navigation bar. Under INTEGRATIONS, select Discover.

Although the integrations are installed separately, they cannot be upgraded separately. Integrations update happens automatically with Devtron upgrade.

Build and Deploy (CI/CD) integration

Devtron CI/CD integration enables software development teams to automate the build and deployment process, allowing them to focus on meeting the business requirements, maintaining code quality, and ensuring security.

Features

• Leverages Kubernetes auto-scaling and centralized caching to give you unlimited cost-efficient CI workers.

• Supports pre-CI and post-CI integrations for code quality monitoring.

• Seamlessly integrates with Clair for image vulnerability scanning.

• Supports different deployment strategies: Blue/Green, Rolling, Canary, and Recreate.

• Implements GitOps to manage the state of Kubernetes applications.

• Integrates with ArgoCD for continuous deployment.

• Check logs, events, and manifests or exec inside containers for debugging.

• Provides deployment metrics like; deployment frequency, lead time, change failure rate, and mean-time recovery.

• Seamless integration with Grafana for continuous application metrics like CPU and memory usage, status code, throughput, and latency on the dashboard.

Installation

1. On the Devtron Stack Manager > Discover page, select the Build and Deploy (CI/CD) integration.

2. On the Discover integrations/Build and Deploy (CI/CD) page, select Install.

The installation status may be one of the following:

A list of installed integrations can be viewed on the Devtron Stack Manager > Installed page.

To update an installed integration, please update Devtron.

Why Devtron takes GitOps Configuration?

Devtron uses GitOps and stores configurations in git; Git Credentials can be entered at Global Configuration > GitOps which is used by Devtron for configuration management and storing desired state of the application configuration. In case GitOps is not configured, Devtron cannot deploy any application or charts.

Areas impacted by GitOps are:

• Deployment Template, click here to learn more.

• Charts, click here to learn more.

Add Git Configuration

Select the GitOps section of global configuration. At the top of the section, four Git providers are available.

• GitHub

• GitLab

• Azure

• BitBucket Cloud

Select one of the Git providers. To add git account, You need to provide the following inputs as given below:

1. Git Host / Azure DevOps Organisation Url / BitBucket Host

2. GitHub Organization Name / Gitlab Group id / Azure DevOps Project Name / BitBucket Workspace ID

3. BitBucket Project Key (only for BitBucket Cloud)

4. Git access credential

1. Git Host:

This field is filled by default, Showing the URL of the selected git providers. For example- https://github.com for GitHub, https://gitlab.com for GitLab, https://dev.azure.com/ for Azure & https://bitbucket.org for BitBucket. Please replace them(not available for GitHub & BitBucket) if they are not the url you want to use.

2. GitHub Organization Name / GitLab Group ID / Azure DevOps Project Name / BitBucket Workspace ID:

In the case of GitHub provide Github Organization Name*. Learn more about Github organization Name. In the case of Gitlab provide Gitlab group Id*. Learn more about Gitlab group Id. Similarly in the case of Azure provide Azure DevOps Project Name*. Learn more about Azure DevOps Project Name. For Bitbucket Cloud, provide Bitbucket Workspace Id*. Learn more about Bitbucket Workspace Id.

3. BitBucket Project Key:

This field is non-mandatory and is only to be filled when you have chosen Bitbucket Cloud as your git provider. If not provided, the oldest project in the workspace will be used. Learn more about Bitbucket Project Key.

4. Git access credential

Provide Git Username and Personal Access Token of your git account.

(a) Username Username for your git account.

(b) Personal Access Token A personal access token (PAT) is used as an alternate password to authenticate into your git accounts.

For GitHub Creating a GitHub Personal Access Token:

• repo - Full control of private repositories(Access commit status , Access deployment status and Access public repositories).

• admin:org - Full control of organizations and teams(Read and write access).

• delete_repo - Grants delete repo access on private repositories.

For GitLab Creating a GitLab Personal Access Token:

• api - Grants complete read/write access to the scoped project API.

• write_repository - Allows read-write access (pull, push) to the repository.

For Azure DevOps Creating a Azure DevOps Access Token:

• code - Grants the ability to read source code and metadata about commits, change sets, branches, and other version control artifacts. More Information on scopes in Azure devops.

For BitBucket Cloud Creating a Bitbucket Cloud Personal Access Token (App passwords)

• repo - Full control of repositories (Read, Write, Admin, Delete access).

Click on Save to save your gitOps configuration details.

Note: A Green tick will appear on the active gitOps provider.

Devtron 🚀

Devtron is a tool integration platform for Kubernetes.

Devtron deeply integrates with products across the lifecycle of microservices,i.e., CI, CD, security, cost, debugging, and observability via an intuitive web interface.

To quickly get started, refer to the Devtron Installation Guide ⎈ ⎈

Why Devtron?

To improve the use of Kubernetes, we employ several tools. Using these tools at the same time, however, is cumbersome and complex. This is because these tools do not communicate with one another to manage different aspects of the application lifecycle, such as CI, CD, security, cost, observability, and stabilization.

Devtron is a one-stop solution for the complexity of the tools mentioned above!

Devtron is an open-source modular product that provides a 'seamless' and 'implementation agnostic uniform interface', that can be integrated with both open-source and commercial tools across the entire lifecycle. All this is achieved while focusing on a slick user experience, including a self-serve model.

You can efficiently handle security, stability, cost, and more in a unified experience.

Devtron Features:

Zero code software delivery workflow for Kubernetes

• Workflow which understands the domain of Kubernetes, testing, CD, SecOps so that you don't have to write scripts

• Reusable and composable components so that workflows are easy to construct and reason through

Multi cloud deployment

• Deploy to multiple Kubernetes clusters on multiple cloud/on-prem from one Devtron setup.

• Works for all cloud providers and on-premise Kubernetes clusters.

Easy DevSecOps integration

• Multi-level security policy at global, cluster, environment, and application-level for efficient hierarchical policy management

• Behavior-driven security policy

• Define policies and exceptions for Kubernetes resources

• Define policies for events for faster resolution

Application debugging dashboard

• One place for all historical Kubernetes events

• Access all manifests securely, such as secret obfuscation

• Application metrics for CPU, RAM, HTTP status code, and latency with a comparison between new and old

• Advanced logging with grep and JSON search

• Intelligent correlation between events, logs for faster triangulation of issue

• Auto issue identification

Enterprise-Grade security and compliances

• Fine-grained access control; control who can edit the configuration and who can deploy.

• Audit log to know who did what and when

• History of all CI and CD events

• Kubernetes events impacting application

• Relevant cloud events and their impact on applications

• Advanced workflow policies like blackout window, branch environment relationship to secure build and deployment pipelines

GitOps aware

• GitOps exposed through API and UI so that you don't have to interact with git CLI

• GitOps backed by Postgres for easy analysis

• Enforce finer access control than Git

Operational insights

• Deployment metrics to measure the success of the agile process. It captures MTTR, change failure rate, deployment frequency, and deployment size out of the box.

• Audit log to understand the failure causes

• Monitor changes across deployments and reverts easily

Compatibility notes

• It uses a modified version of argo rollout.

• Application metrics only work for k8s 1.16+

What's next

• Install Devtron

Contribute

Check out our contributing guidelines. Directions for opening issues, coding standards, and notes on our development processes are all included.

Community

Get updates on Devtron's development and chat with the project maintainers, contributors, and community members.

• Join the Discord Community

• Follow @DevtronL on Twitter

• Raise feature requests, suggest enhancements, report bugs at GitHub issues

• Read the Devtron blog

Vulnerability Reporting

We at Devtron take security and our users' trust very seriously. If you believe you have found a security issue in Devtron, please responsibly disclose it by contacting us at security@devtron.ai.

License

Devtron is available under the Apache License, Version 2.0.

Projects are nothing but a logical grouping of your applications so that you can manage and control the access level of users. We will discuss User Access in the next step.

Add Project:

Click on the Projects inside the Global configuration tab. Click on Add projects and give a name to your project and press the Save button to save your project

This is used to assign user to a particular group and user inherits all the permissions granted to this group. The Permission groups section contains a drop-down of all existing groups on which you have access. This is optional field and more than one groups can be selected for a user.

The advantage of the groups is to define a set of privileges like create, edit, or delete for the given set of resources that can be shared among the users within the group. Users can be added to an existing group to utilize the privileges that it grants. Any access change to group is reflected immediately in user access.

You can select the group which you are creating in the Group permissions section inside Add users.

1. Add new Group

Go to Global configurations -> Authorization -> Permission group and click on Add Group to create a new group.

Enter the Group Name and Description.

2. Create permission Group

Once you have given the group name and group description.

Assign the permissions of groups in the Devtron Apps, Helm Apps or Group Chart permissions section. Manage the project, environment, application and role access the same as we discuss in the user permissions section.

You can add multiple rows for the Devtron Apps and Helm Apps Permissions section.

Once you have finished assigning the appropriate permissions for the permission group, click on Save.

3. Edit Group Permissions

You can edit the permission groups by clicking on the downward arrow.

Then you can edit the permission group here.

Once you are done editing the permission group, click on Save.

If you want to delete the groups with particular permission group, click on Delete.

4. Manage Chart Group Permissions

The chart group permissions for the permission groups will be managed in the same way as for the users. For reference, check Manage chart group permissions for users.

Container registries are used to store images built by the CI Pipeline. Here you can configure the container registry you want to use for storing images.

When configuring an application, you can choose which registry and repository it should use in the App Configuration > section.

Add Container Registry configuration:

Go to the Container Registry section of Global Configuration. Click on Add container registry.

You will see below the input fields to configure the container registry.

• Name

• Registry type

  • ecr

    • AWS region

    • Access key ID

    • Secret access key

  • docker hub

    • Username

    • Password

  • Others

    • Username

    • password

• Registry URL

• Set as default

Name

Provide a name to your registry, this name will be shown to you in Docker Build Config as a drop-down.

Registry type

Here you can select the type of the Registry. We are supporting three types- docker hub, ecr and others. You can select any one of them from the drop-down. By default, this value is ecr. If you select ecr then you have to provide some information like- AWS region, Access Key, and Secret Key. If you select docker hub then you have to provide Username and Password. And if you select others then you have to provide the Username and Password.

Registry URL

Select any type of Registry from the drop-down, you have to provide the URL of your registry. Create your registry and provide the URL of that registry in the URL box.

Registry Type: ECR

To add an Amazon Elastic Container Registry (ECR), select the ECR Registry type. Amazon ECR is an AWS-managed container image registry service. The ECR provides resource-based permissions to the private repositories using AWS Identity and Access Management (IAM). ECR allows both Key-based and Role-based authentications.

To set this ECR as the default registry hub for your images, select [x] Set as default registry.

Select Save.

Registry Type- Docker Hub

You have to provide the below information if you select the registry type as Docker Hub.

• Username

Give the username of the docker hub account you used for creating your registry in.

• Password

Registry Type Others:

You have to provide the below information if you select the registry type as others.

• Username

Give the username of your account, where you have created your registry in.

• Password

Give the password corresponding to the username of your registry.

Set as default:

If you enable the Set as default option, then this registry name will be set as default in the Container Registry section inside the Docker build config page. This is optional. You can keep it disabled.

Advance Registry Url connection options:

• If you enable the Allow Only Secure Connection option, then this registry allows only secure connections.

• If you enable the Allow Secure Connection With CA Certificate option, then you have to upload/provide private CA certificate (ca.crt).

• If the container registry is insecure (for eg : SSL certificate is expired), then you enable the Allow Insecure Connection option.

Now click on Save to save the configuration of the Container registry.

Note:

You can use any registry which can be authenticated using docker login -u <username> -p <password> <registry-url>. However these registries might provide a more secured way for authentication, which we will support later. Some popular registries which can be used using username and password mechanism:

Integrating With External Container Registry

If you want to use a private registry for container registry other than ecr, this will be used to push image and then create a secret in same environment to pull the image to deploy. To create secret, go to charts section and search for chart ‘dt-secrets’ and configure the chart. Provide an App Name and select the Project and Environment in which you want to deploy this chart and then configure the values.yaml as shown in example. The given example is for DockerHub but you can configure similarly for any container registry that you want to use.

Devtron includes predefined helm charts that cover the majority of use cases. For any use case not addressed by the default helm charts, you can upload your own helm chart and use it as a custom chart in Devtron.

• Who can upload a custom chart - Super admins

• Who can use the custom chart - All users

A super admin can upload multiple versions of a custom helm chart.

Prerequisites

1. A valid helm chart, which contains Chart.yaml file with name and version fields.

2. Image descriptor template file - .image_descriptor_template.json.

3. Custom chart packaged in the *.tgz format.

1. How to create a helm chart

2. Create the image descriptor template file - .image_descriptor_template.json

It's a GO template file that should produce a valid JSON file upon rendering. This file is passed as the last argument in helm install -f myvalues.yaml -f override.yaml command.

Place the .image_descriptor_template.json file in the root directory of your chart.

You can use the following variables in the helm template (all the placeholders are optional):

The values from the CD deployment pipeline are injected at the placeholder specified in the .image_descriptor_template.json template file.

For example:

To create a template file to allow Devtron to only render the repository name and the tag from the CI/CD pipeline that you created, edit the .image_descriptor_template.json file as:

3. Package the custom chart in the *.tgz format

Before you begin, ensure that your helm chart includes both Chart.yaml (with name and version fields) and .image_descriptor_template.json files.

The helm chart to be uploaded must be packaged as a versioned archive file in the format - <helm-chart-name>-vx.x.x.tgz.

The above command will create a my-custom-chart-0.1.0.tgz file.

Uploading a custom chart

A custom chart can only be uploaded by a super admin.

• On the Devtron dashboard, select Global Configurations > Custom charts.

• Select Import Chart.

• Choose Select tar.gz file... and upload the packaged custom chart in the *.tgz format.

The chart is being uploaded and validated. You may also Cancel upload if required.

Validation

The uploaded archive will be validated against:

• Supported archive template should be in *.tgz format.

• Chart.yaml must include the name and the version number.

• image_descriptor_template.json file should be present and the field format must match the format listed in the image builder template section.

The following are the validation results:

View the custom charts

All users can view the custom charts.

To view a list of available custom charts, go to Global Configurations > Custom charts page.

• The charts can be searched with their name, version, or description.

Use the custom chart in an application

Info:

Devtron can be upgraded in one of the following ways:

Upgrade Devtron using Helm

Versions Upgrade

• 0.4.x to 0.4.x

• 0.3.x to 0.4.x

• 0.3.x to 0.3.x

• 0.2.x to 0.3.x

Upgrade Devtron from the UI

• Update Devtron from Devtron UI

The Global configuration provides a feature of Cluster & Environment in which you can add your Kubernetes clusters and environment.

Select the Cluster & Environment section of global configuration and click on Add Cluster to add your cluster.

Add Cluster:

To add a cluster on devtron, you must have superadmin access.

Navigate to the Global Configurations → Clusters and Environments on devtron and click on Add Cluster. Provide the below informations to add your kubernetes cluster:

1. Name

2. Kubernetes Cluster Info

   • Server URL

   • Bearer token

3. Prometheus Info

   • Prometheus endpoint

     • Basic

       • Username

       • Password

     • Anonymous

   • TLS Key

   • TLS Certificate

1. Name

Give a name to your cluster inside the name box.

2. Kubernetes Cluster Info

Provide your kubernetes cluster’s credentials.

• Server URL

Provide the endpoint/URL of your kubernetes cluster.It is recommended to use a self-hosted URL instead of cloud hosted. Self-hosted URL will provide the following benefits.

(a) Disaster Recovery - It is not possible to edit the server-url of a cluster. So if you're using an eks url, For eg- *****.eu-west-1.elb.amazonaws.com it will be a tedious task to add a new cluster and migrate all the services one by one. While using a self-hosted url For eg- clear.example.com you can just point to the new cluster's server url in DNS manager and update the new cluster token and sync all the deployments.

(b) Easy cluster migrations - Cluster url is given in the name of the cloud provider used, so migrating your cluster from one provider to another will result in waste of time and effort. On the other hand, if using a self-hosted url migrations will be easy as the url is of single hosted domain independent of the cloud provider.

• Bearer token

Provide your kubernetes cluster’s Bearer token for authentication purposes so that Devtron is able to communicate with your kubernetes cluster and can deploy your application in your kubernetes cluster.

Get the server url and generate the bearer token by running the following command:

curl -O https://raw.githubusercontent.com/devtron-labs/utilities/main/kubeconfig-exporter/kubernetes_export_sa.sh && bash kubernetes_export_sa.sh cd-user devtroncd https://raw.githubusercontent.com/devtron-labs/utilities/main/kubeconfig-exporter/clusterrole.yaml

Please ensure that kubectl and jq are installed on the bastion on which you’re running the command.

3. Prometheus Info

Prometheus is a powerful solution to provide graphical insight into your application behavior. If you want to see your application matrix against your applications deployed in kubernetes, install Prometheus in your kubernetes cluster. The below inputs are required to configure your prometheus into Devtron’s tool.

• Prometheus endpoint

Provide the URL of your prometheus. Prometheus supports two types of authentication Basic and Anonymous. Select the authentication type for your Prometheus setup.

• Basic

If you select the basic type of authentication then you have to provide the Username and Password of prometheus for authentication.

• Anonymous

If you select Anonymous then you do not have to provide any username and password for authentication.

• TLS Key & TLS Certificate

TLS key and TLS certificate both options are optional, these options are used when you use a custom URL, in that case, you can pass your TLS key and TLS certificate.

K8s Version

on saving or update a cluster there is a call to fetch k8s version, it will store corresponding to cluster on db. used in listing api's and app detail page for grafana url.

Check the below screenshots to know how it looks like If you select the Basic authentication type

If you select the Anonymous authentication type

Now click on Save Cluster to save your cluster information.

Note:

Your kubernetes cluster gets mapped with the Devtron when you save your kubernetes cluster Configuration. Now the agents of devtron will be installed on your cluster so that the components of devtron can communicate to your cluster. When the agent starts installing on your cluster, you can check the status of the agents in the Cluster & Environment tab also.

Click on Details to check what got installed inside the agents. A new window will be popped up displaying all the details about these agents.

Add Environment

Once you have added your cluster in Cluster & Environment, you can add the environment also. Click on Add Environment, a window will be opened. Give a name to your environment in the Environment Name box and provide a namespace corresponding to your environment in the Namespace input box. Now choose if your environment is for Production purposes or for Non-production purposes. Production and Non-production options are only for tagging purposes. Click on Save and your environment will be created.

You can update an already created environment, Select and click on the environment which you want to update. You can only change Production and Non-production options here.

Note

You can not change the Environment name and Namespace name.

Click on Update to update your environment.

Devtron can be updated from the Devtron Stack Manager > About Devtron section.

• Select Update to Devtron

The update process may show one of the following statuses, with details available for tracking, troubleshooting, and additional information:

Updating Devtron also updates the installed integrations.

his feature helps you manage the notifications for your build and deployment pipelines. You can receive the notifications on Slack or via e-mail.

Click on Global Configurations -> Notifications

Notification Configuration:

Click on Configurations and you will see Devtron support two types of configurations SES Configurations or Slack Configurations.

Manage SES Configurations

You can manage the SES configuration to receive e-mails by entering the valid credentials. Make sure your e-mail is verified by SES.

Click on Add and configure SES.

Click on Save to save your SES configuration or e-mail ID

Manage Slack Configurations