In certain cases, you may want to override default configurations provided by Devtron. For example, for deployments or statefulsets you may want to change the memory or CPU requests or limit or add node affinity or taint tolerance. Say, for ingress, you may want to add annotations or host. Samples are available inside the manifests/updates directory.

To modify a particular object, it looks in namespace devtroncd for the corresponding configmap as mentioned in the mapping below:

Let's take an example to understand how to override specific values. Say, you want to override annotations and host in the ingress, i.e., you want to change devtronIngress, copy the file devtron-ingress-override.yaml. This file contains a configmap to modify devtronIngress as mentioned above. Please note the structure of this configmap, data should have the key override with a multiline string as a value.

apiVersion, kind, metadata.name in the multiline string is used to match the object which needs to be modified. In this particular case it will look for apiVersion: extensions/v1beta1, kind: Ingress and metadata.name: devtron-ingress and will apply changes mentioned inside update: as per the example inside the metadata: it will add annotations owner: app1 and inside spec.rules.http.host it will add http://change-me.

In case you want to change multiple objects, for eg in argocd you want to change the config of argocd-dex-server as well as argocd-redis then follow the example in devtron-argocd-override.yaml.

Once we have made these changes in our local system we need to apply them to a Kubernetes cluster on which Devtron is installed currently using the below command:

kubectl apply -f file-name -n devtroncd

Run the following command to make these changes take effect:

kubectl patch -n devtroncd installer installer-devtron --type='json' -p='[{"op": "add", "path": "/spec/reSync", "value": true }]'

Our changes would have been propagated to Devtron after 20-30 minutes.

Recommended Resources for Production use

To use Devtron for production deployments, use our recommended production overrides located in manifests/updates/production. This configuration should be enough for handling up to 200 microservices.

The overall resources required for the recommended production overrides are:

The production overrides can be applied as pre-devtron installation as well as post-devtron installation in the respective namespace.

Pre-Devtron Installation

If you want to install a new Devtron instance for production-ready deployments, this is the best option for you.

Create the namespace and apply the overrides files as stated above:

kubectl create ns devtroncd

After files are applied, you are ready to install your Devtron instance with production-ready resources.

Post-Devtron Installation

If you have an existing Devtron instance and want to migrate it for production-ready deployments, this is the right option for you.

In the existing namespace, apply the production overrides as we do it above.

kubectl apply -f prod-configs -n devtroncd

This page helps you to install Devtron without any integrations. Integrations can be added later using Devtron Stack Manager.

Before you begin

Install Helm if you haven't done that already!

helm repo add devtron https://helm.devtron.ai

helm install devtron devtron/devtron-operator --create-namespace --namespace devtroncd

Devtron dashboard

Use the following command to get the dashboard URL:

kubectl get svc -n devtroncd devtron-service -o jsonpath='{.status.loadBalancer.ingress}'

You will get the result something as shown below:

[test2@server ~]$ kubectl get svc -n devtroncd devtron-service -o jsonpath='{.status.loadBalancer.ingress}'
[map[hostname:aaff16e9760594a92afa0140dbfd99f7-305259315.us-east-1.elb.amazonaws.com]]

The hostname aaff16e9760594a92afa0140dbfd99f7-305259315.us-east-1.elb.amazonaws.com as mentioned above is the Loadbalancer URL where you can access the Devtron dashboard.

You can also do a CNAME entry corresponding to your domain/subdomain to point to this Loadbalancer URL to access it at a custom domain.

Devtron Admin credentials

For admin login, use the username as admin, and run the following command to get the admin password:

kubectl -n devtroncd get secret devtron-secret \
-o jsonpath='{.data.ACD_PASSWORD}' | base64 -d

Cleaning Helm installer

Please make sure that you do not have anything inside namespaces devtroncd, devtron-cd devtron-ci, and devtron-demo as the below steps will clean everything inside these namespaces

helm uninstall devtron --namespace devtroncd

kubectl delete -n devtroncd \
-f https://raw.githubusercontent.com/devtron-labs/charts/main/charts/devtron/crds/crd-devtron.yaml

kubectl delete ns devtroncd

Upgrade

To use the CI/CD capabilities with Devtron, users can Install the CI/CD integration.

Devtron 🚀

Devtron is a tool integration platform for Kubernetes.

Devtron deeply integrates with products across the lifecycle of microservices,i.e., CI, CD, security, cost, debugging, and observability via an intuitive web interface.

To quickly get started, refer to the Devtron Installation Guide ⎈ ⎈

Why Devtron?

To improve the use of Kubernetes, we employ several tools. Using these tools at the same time, however, is cumbersome and complex. This is because these tools do not communicate with one another to manage different aspects of the application lifecycle, such as CI, CD, security, cost, observability, and stabilization.

Devtron is a one-stop solution for the complexity of the tools mentioned above!

Devtron is an open-source modular product that provides a 'seamless' and 'implementation agnostic uniform interface', that can be integrated with both open-source and commercial tools across the entire lifecycle. All this is achieved while focusing on a slick user experience, including a self-serve model.

You can efficiently handle security, stability, cost, and more in a unified experience.

Devtron Features:

Zero code software delivery workflow for Kubernetes

• Workflow which understands the domain of Kubernetes, testing, CD, SecOps so that you don't have to write scripts

• Reusable and composable components so that workflows are easy to construct and reason through

Multi cloud deployment

• Deploy to multiple Kubernetes clusters on multiple cloud/on-prem from one Devtron setup.

• Works for all cloud providers and on-premise Kubernetes clusters.

Easy DevSecOps integration

• Multi-level security policy at global, cluster, environment, and application-level for efficient hierarchical policy management

• Behavior-driven security policy

• Define policies and exceptions for Kubernetes resources

• Define policies for events for faster resolution

Application debugging dashboard

• One place for all historical Kubernetes events

• Access all manifests securely, such as secret obfuscation

• Application metrics for CPU, RAM, HTTP status code, and latency with a comparison between new and old

• Advanced logging with grep and JSON search

• Intelligent correlation between events, logs for faster triangulation of issue

• Auto issue identification

Enterprise-Grade security and compliances

• Fine-grained access control; control who can edit the configuration and who can deploy.

• Audit log to know who did what and when

• History of all CI and CD events

• Kubernetes events impacting application

• Relevant cloud events and their impact on applications

• Advanced workflow policies like blackout window, branch environment relationship to secure build and deployment pipelines

GitOps aware

• GitOps exposed through API and UI so that you don't have to interact with git CLI

• GitOps backed by Postgres for easy analysis

• Enforce finer access control than Git

Operational insights

• Deployment metrics to measure the success of the agile process. It captures MTTR, change failure rate, deployment frequency, and deployment size out of the box.

• Audit log to understand the failure causes

• Monitor changes across deployments and reverts easily

Compatibility notes

• It uses a modified version of argo rollout.

• Application metrics only work for k8s 1.16+

What's next

• Install Devtron

Contribute

Check out our contributing guidelines. Directions for opening issues, coding standards, and notes on our development processes are all included.

Community

Get updates on Devtron's development and chat with the project maintainers, contributors, and community members.

• Join the Discord Community

• Follow @DevtronL on Twitter

• Raise feature requests, suggest enhancements, report bugs at GitHub issues

• Read the Devtron blog

Vulnerability Reporting

We at Devtron take security and our users' trust very seriously. If you believe you have found a security issue in Devtron, please responsibly disclose it by contacting us at security@devtron.ai.

License

Devtron is available under the Apache License, Version 2.0.

After Devtron is installed, Devtron is accessible through service devtron-service. If you want to access devtron through ingress, edit devtron-service and change the loadbalancer to ClusterIP. You can do this using kubectl patch command like :

After that create ingress by applying the ingress yaml file. You can use to create ingress to access devtron:

Optionally you also can access devtron through a specific host like :

Devtron is installed over a Kubernetes cluster and can be installed standalone or along with CI/CD integration:

• : Devtron installation with the CI/CD integration is used to perform CI/CD, security scanning, GitOps, debugging, and observability.

• : The Devtron installation includes functionalities to deploy, observe, manage, and debug existing Helm applications in multiple clusters and deeply integrate with multiple tools using extensions.

Recommended resources

The minimum requirements for Devtron and Devtron with CI/CD integration in production and non-production environments include:

• Non-production

• Production (assumption based on 5 clusters)

Refer to the section for more information.

Before you begin

Create a (preferably K8s 1.16 or higher) if you haven't done that already!

Refer to the article to set up a cluster in the production environment.

Installing Devtron

Devtron integrations extend the functionality of your Devtron stack.

Discover and install integrations

The current release of Devtron supports the Build and Deploy (CI/CD) integration. More integrations will be available soon; to request one, please

Integrations can be installed by super admins; However other user roles can browse and request super admins to install the required integrations.

Integrations are updated along with .

Select Devtron Stack Manager from the left navigation bar. Under INTEGRATIONS, select Discover.

Build and Deploy (CI/CD) integration

Devtron CI/CD integration enables software development teams to automate the build and deployment process, allowing them to focus on meeting the business requirements, maintaining code quality, and ensuring security.

Features

• Leverages Kubernetes auto-scaling and centralized caching to give you unlimited cost-efficient CI workers.

• Supports pre-CI and post-CI integrations for code quality monitoring.

• Seamlessly integrates with Clair for image vulnerability scanning.

• Supports different deployment strategies: Blue/Green, Rolling, Canary, and Recreate.

• Implements GitOps to manage the state of Kubernetes applications.

• Integrates with ArgoCD for continuous deployment.

• Check logs, events, and manifests or exec inside containers for debugging.

• Provides deployment metrics like; deployment frequency, lead time, change failure rate, and mean-time recovery.

• Seamless integration with Grafana for continuous application metrics like CPU and memory usage, status code, throughput, and latency on the dashboard.

Installation

1. On the Devtron Stack Manager > Discover page, select the Build and Deploy (CI/CD) integration.

2. On the Discover integrations/Build and Deploy (CI/CD) page, select Install.

The installation status may be one of the following:

A list of installed integrations can be viewed on the Devtron Stack Manager > Installed page.

Configuration

Configure Secrets

For helm installation this section referes to secrets.env section of values.yaml. For kubectl based installation it refers to kind: secret in install/devtron-operator-configs.yaml.

Configure the following properties:

Configure ConfigMaps

For helm installation this section refers to configs section of values.yaml. For kubectl based installation it refers to kind: ConfigMap in install/devtron-operator-configs.yaml.

Configure the following properties:

Storage for Logs and Cache

AWS SPECIFIC

While installing Devtron and using the AWS-S3 bucket for storing the logs and caches, the below parameters are to be used in the ConfigMap.

NOTE: For using the S3 bucket it is important to add the S3 permission policy to the IAM role attached to the nodes of the cluster.

AZURE SPECIFIC

While installing Devtron using Azure Blob Storage for storing logs and caches, the below parameters will be used in the ConfigMap.

To convert string to base64 use the following command:

Note:

1. Ensure that the cluster has read and write access to the S3 buckets/Azure Blob storage container mentioned in DEFAULT_CACHE_BUCKET, DEFAULT_BUILD_LOGS_BUCKET or AZURE_BLOB_CONTAINER_CI_LOG, or AZURE_BLOB_CONTAINER_CI_CACHE.

2. Ensure that the cluster has read access to AWS secrets backends (SSM & secrets manager).

We can use the --set flag to override the default values when installing with Helm. For example, to update POSTGRESQL_PASSWORD and BLOB_STORAGE_PROVIDER, use the install command as:

Secrets

ConfigMaps

Dashboard Configurations

This documentation consists of the Global Configurations available in Devtron.

Parts of the Documentation

Container registries are used to store images built by the CI Pipeline. Here you can configure the container registry you want to use for storing images.

When configuring an application, you can choose which registry and repository it should use in the App Configuration > Docker Build Config section.

Add Container Registry configuration:

Go to the Container Registry section of Global Configuration. Click on Add container registry.

You will see below the input fields to configure the container registry.

• Name

• Registry type

  • ecr

    • AWS region

    • Access key ID

    • Secret access key

  • docker hub

    • Username

    • Password

  • Others

    • Username

    • password

• Registry URL

• Set as default

Name

Provide a name to your registry, this name will be shown to you in Docker Build Config as a drop-down.

Registry type

Here you can select the type of the Registry. We are supporting three types- docker hub, ecr and others. You can select any one of them from the drop-down. By default, this value is ecr. If you select ecr then you have to provide some information like- AWS region, Access Key, and Secret Key. If you select docker hub then you have to provide Username and Password. And if you select others then you have to provide the Username and Password.

Registry URL

Select any type of Registry from the drop-down, you have to provide the URL of your registry. Create your registry and provide the URL of that registry in the URL box.

Registry Type: ECR

To add an Amazon Elastic Container Registry (ECR), select the ECR Registry type. Amazon ECR is an AWS-managed container image registry service. The ECR provides resource-based permissions to the private repositories using AWS Identity and Access Management (IAM). ECR allows both Key-based and Role-based authentications.

Before you begin, create an IAM user, and attach only ECR policy ( AmazonEC2ContainerRegistryFullAccess ) if using Key-based auth. Or attach the ECR policy ( AmazonEC2ContainerRegistryFullAccess) to the cluster worker nodes IAM role of your Kubernetes cluster if using Role-based access.

To set this ECR as the default registry hub for your images, select [x] Set as default registry.

Select Save.

To use the ECR container image, go to the Applications page and select your application, and then select App Configuration > Docker Build Config.

Registry Type- Docker Hub

You have to provide the below information if you select the registry type as Docker Hub.

• Username

Give the username of the docker hub account you used for creating your registry in.

• Password

Give the password/token corresponding to your docker hub account.

Registry Type Others:

You have to provide the below information if you select the registry type as others.

• Username

Give the username of your account, where you have created your registry in.

• Password

Give the password corresponding to the username of your registry.

Set as default:

If you enable the Set as default option, then this registry name will be set as default in the Container Registry section inside the Docker build config page. This is optional. You can keep it disabled.

Advance Registry Url connection options:

• If you enable the Allow Only Secure Connection option, then this registry allows only secure connections.

• If you enable the Allow Secure Connection With CA Certificate option, then you have to upload/provide private CA certificate (ca.crt).

• If the container registry is insecure (for eg : SSL certificate is expired), then you enable the Allow Insecure Connection option.

Now click on Save to save the configuration of the Container registry.

Note:

You can use any registry which can be authenticated using docker login -u <username> -p <password> <registry-url>. However these registries might provide a more secured way for authentication, which we will support later. Some popular registries which can be used using username and password mechanism:

• Google Container Registry (GCR) : JSON key file authentication method can be used to authenticate with username and password. Please follow link for getting username and password for this registry. Please remove all the white spaces from json key and wrap it in single quote while putting in password field.

• Google Artifact Registry (GAR) : JSON key file authentication method can be used to authenticate with username and password. Please follow link for getting username and password for this registry. Please remove all the white spaces from json key and wrap it in single quote while putting in password field.

• Azure Container Registry (ACR) : Service principal authentication method can be used to authenticate with username and password. Please follow link for getting username and password for this registry.

Integrating With External Container Registry

If you want to use a private registry for container registry other than ecr, this will be used to push image and then create a secret in same environment to pull the image to deploy. To create secret, go to charts section and search for chart ‘dt-secrets’ and configure the chart. Provide an App Name and select the Project and Environment in which you want to deploy this chart and then configure the values.yaml as shown in example. The given example is for DockerHub but you can configure similarly for any container registry that you want to use.

name: regcred
type: kubernetes.io/dockerconfigjson
labels:
 test: chart
secrets:
 data:
   - key: .dockerconfigjson
     value: '{"auths":{"https://index.docker.io/v1/":{"username":"<username>","password":"<password>}}}'

The name that you provide in values.yaml ie. regcred is name of the secret that will be used as imagePullSecrets to pull the image from docker hub to deploy. To know how imagePullSecrets will be used in the deployment-template, please follow the documentation.

The Global configuration provides a feature of Cluster & Environment in which you can add your Kubernetes clusters and environment.

Select the Cluster & Environment section of global configuration and click on Add Cluster to add your cluster.

Add Cluster:

To add a cluster on devtron, you must have superadmin access.

Navigate to the Global Configurations → Clusters and Environments on devtron and click on Add Cluster. Provide the below informations to add your kubernetes cluster:

1. Name

2. Kubernetes Cluster Info

   • Server URL

   • Bearer token

3. Prometheus Info

   • Prometheus endpoint

     • Basic

       • Username

       • Password

     • Anonymous

   • TLS Key

   • TLS Certificate

1. Name

Give a name to your cluster inside the name box.

2. Kubernetes Cluster Info

Provide your kubernetes cluster’s credentials.

• Server URL

Provide the endpoint/URL of your kubernetes cluster.It is recommended to use a self-hosted URL instead of cloud hosted. Self-hosted URL will provide the following benefits.

(a) Disaster Recovery - It is not possible to edit the server-url of a cluster. So if you're using an eks url, For eg- *****.eu-west-1.elb.amazonaws.com it will be a tedious task to add a new cluster and migrate all the services one by one. While using a self-hosted url For eg- clear.example.com you can just point to the new cluster's server url in DNS manager and update the new cluster token and sync all the deployments.

(b) Easy cluster migrations - Cluster url is given in the name of the cloud provider used, so migrating your cluster from one provider to another will result in waste of time and effort. On the other hand, if using a self-hosted url migrations will be easy as the url is of single hosted domain independent of the cloud provider.

• Bearer token

Provide your kubernetes cluster’s Bearer token for authentication purposes so that Devtron is able to communicate with your kubernetes cluster and can deploy your application in your kubernetes cluster.

Get the server url and generate the bearer token by running the following command:

curl -O https://raw.githubusercontent.com/devtron-labs/utilities/main/kubeconfig-exporter/kubernetes_export_sa.sh && bash kubernetes_export_sa.sh cd-user devtroncd https://raw.githubusercontent.com/devtron-labs/utilities/main/kubeconfig-exporter/clusterrole.yaml

Please ensure that kubectl and jq are installed on the bastion on which you’re running the command.

3. Prometheus Info

Prometheus is a powerful solution to provide graphical insight into your application behavior. If you want to see your application matrix against your applications deployed in kubernetes, install Prometheus in your kubernetes cluster. The below inputs are required to configure your prometheus into Devtron’s tool.

• Prometheus endpoint

Provide the URL of your prometheus. Prometheus supports two types of authentication Basic and Anonymous. Select the authentication type for your Prometheus setup.

• Basic

If you select the basic type of authentication then you have to provide the Username and Password of prometheus for authentication.

• Anonymous

If you select Anonymous then you do not have to provide any username and password for authentication.

• TLS Key & TLS Certificate

TLS key and TLS certificate both options are optional, these options are used when you use a custom URL, in that case, you can pass your TLS key and TLS certificate.

K8s Version

on saving or update a cluster there is a call to fetch k8s version, it will store corresponding to cluster on db. used in listing api's and app detail page for grafana url.

Check the below screenshots to know how it looks like If you select the Basic authentication type

If you select the Anonymous authentication type

Now click on Save Cluster to save your cluster information.

Note:

Your kubernetes cluster gets mapped with the Devtron when you save your kubernetes cluster Configuration. Now the agents of devtron will be installed on your cluster so that the components of devtron can communicate to your cluster. When the agent starts installing on your cluster, you can check the status of the agents in the Cluster & Environment tab also.

Click on Details to check what got installed inside the agents. A new window will be popped up displaying all the details about these agents.

Add Environment

Once you have added your cluster in Cluster & Environment, you can add the environment also. Click on Add Environment, a window will be opened. Give a name to your environment in the Environment Name box and provide a namespace corresponding to your environment in the Namespace input box. Now choose if your environment is for Production purposes or for Non-production purposes. Production and Non-production options are only for tagging purposes. Click on Save and your environment will be created.

You can update an already created environment, Select and click on the environment which you want to update. You can only change Production and Non-production options here.

Note

You can not change the Environment name and Namespace name.

Click on Update to update your environment.

Overview

Once installed Devtron has one built-in admin user with super-admin privileges that has complete access to the system. It is recommended to use admin user only for initial and global configuration and then switch to local users or configure SSO integration.

Only users with super-admin privileges have access to create SSO configuration. Devtron uses dex for authenticating a user against the identity provider.

To add/edit SSO configuration please go to the left main panel -> Select Global Configurations -> Select SSO Login Services

Supported SSO Providers

LDAP GitHub OpenID Connect Google Microsoft OpenShift

Dex implements connectors that target specific identity providers, for each connector configuration user must have created account for the corresponding identity provider and registered an app for client key and secret. For examples see

• https://dexidp.io/docs/connectors/

• https://dexidp.io/docs/connectors/google/

1. Create new SSO Configuration

Login as a user with super-admin privileges and go to Global Configurations -> SSO Login Services and click on any Identity Provider and fill the configuration.

Add valid devtron application URL where it is hosted.

Fill correct redirect URL or callback URL from which you have registered with the identity provider in the previous step along with the client id and client secret shared by the identity provider.

Only single SSO login configuration can be active at one time. Whenever you create or update any SSO config, it will be activated and used by the system and previous configurations will be deleted.

Except for the domain substring, URL and redirectURI should be the same as in the screenshots.

Select Save to create and activate SSO login.

2. Update SSO Configuration

SSO configuration can be changed by the user at any later point in time by updating the configuration and clicking on the Save button at the bottom right. In case of configuration change all users will be logged out of the system and will have to login again.

3. Configuration Payload

• type : oidc or any platform name such as (google, gitlab, github etc)

• name : identity provider platform name

• id : identity provider platform unique id in string. (refer to dexidp.io)

• config : user can put connector details into this key. platforms may not have same structure but commons are clientID, clientSecret, redirectURI.

• hostedDomains : domains authorized for SSO login.

Devtron includes predefined helm charts that cover the majority of use cases. For any use case not addressed by the default helm charts, you can upload your own helm chart and use it as a custom chart in Devtron.

• Who can upload a custom chart - Super admins

• Who can use the custom chart - All users

A super admin can upload multiple versions of a custom helm chart.

Prerequisites

1. A valid helm chart, which contains Chart.yaml file with name and version fields.

2. Image descriptor template file - .image_descriptor_template.json.

3. Custom chart packaged in the *.tgz format.

1. How to create a helm chart

Chart.yaml is the metadata file that gets created when you create a helm chart.

helm create my-custom-chart

2. Create the image descriptor template file - .image_descriptor_template.json

It's a GO template file that should produce a valid JSON file upon rendering. This file is passed as the last argument in helm install -f myvalues.yaml -f override.yaml command.

Place the .image_descriptor_template.json file in the root directory of your chart.

You can use the following variables in the helm template (all the placeholders are optional):

The values from the CD deployment pipeline are injected at the placeholder specified in the .image_descriptor_template.json template file.

{
    "server": {
        "deployment": {
            "image_tag": "{{.Tag}}"
            "image": "{{.Name}}"
        }
    },
    "pipelineName": "{{.PipelineName}}",
    "releaseVersion": "{{.ReleaseVersion}}",
    "deploymentType": "{{.DeploymentType}}", ?
    "app": "{{.App}}",
    "env": "{{.Env}}",
    "appMetrics": {
        {.AppMetrics
        }
    }
}

For example:

To create a template file to allow Devtron to only render the repository name and the tag from the CI/CD pipeline that you created, edit the .image_descriptor_template.json file as:

Copy

{
    "image": {
          "repository": "{{.Name}}",
          "tag": "{{.Tag}}"
    }
}

3. Package the custom chart in the *.tgz format

Before you begin, ensure that your helm chart includes both Chart.yaml (with name and version fields) and .image_descriptor_template.json files.

The helm chart to be uploaded must be packaged as a versioned archive file in the format - <helm-chart-name>-vx.x.x.tgz.

helm package my-custom-chart

The above command will create a my-custom-chart-0.1.0.tgz file.

Uploading a custom chart

A custom chart can only be uploaded by a super admin.

• On the Devtron dashboard, select Global Configurations > Custom charts.

• Select Import Chart.

• Choose Select tar.gz file... and upload the packaged custom chart in the *.tgz format.

The chart is being uploaded and validated. You may also Cancel upload if required.

Validation

The uploaded archive will be validated against:

• Supported archive template should be in *.tgz format.

• Chart.yaml must include the name and the version number.

• image_descriptor_template.json file should be present and the field format must match the format listed in the image builder template section.

The following are the validation results:

View the custom charts

All users can view the custom charts.

To view a list of available custom charts, go to Global Configurations > Custom charts page.

• The charts can be searched with their name, version, or description.

• New custom charts can be uploaded by selecting Upload chart.

Use the custom chart in an application

The custom charts can be used from the Deployment Template section.

Info:

The deployment strategy for a custom chart is fetched from the custom chart template and cannot be configured in the CD pipeline.

Git Accounts allow you to connect your code source with Devtron. You will be able to use these git accounts to build the code using the CI pipeline.

Git Account Configuration

Global Configuration helps you to add a Git provider. Click on Add git account button at the top of the Git Account Section. To add a new git provider, add the details as mentioned below.

1. Name

2. Git Host

3. URL

4. Authentication type

1. Name

Provide a Name to your Git provider. This name will be displayed in the the Git Provider drop-down inside the Git Material configuration section.

2. Git Host

It is the git provider on which corresponding application git repository is hosted. By default you will get Bitbucket and GitHub but you can add many as you want clicking on [+ Add Git Host].

3. URL

Provide the URL. For example- https://github.com for Github, https://gitlab.com for GitLab, etc.

4. Authentication type

Here provide the type of authentication required by your version controller. Devtron supports three types of authentications. You can choose the one that suits you the best.

• Anonymous

If authentication type is set as Anonymous then you do not need to provide any username, password/authentication token or SSH key. Just click on Save to save the git account provider details.

If authentication type is set as Anonymous, only public git repository will be accessible.

• User Auth

If you select User Auth then you have to provide the Username and either of Password or Auth Token for the authentication of your version controller account. Click on Save to save the git account provider details.

• SSH Key

If you choose SSH Key then you have to provide the Private SSH Key corresponding to the public key added in your version controller account. Click on Save to save the git account provider details.

Update Git Account

You can update your saved git account settings at anytime. To update the git account:

1. Click on the git account which you want to update.

2. Make the required changes

3. Click on Update to save the changes.

Updates can only be made within one Authentication type or one protocol type, i.e. HTTPS(Anonymous or User Auth) & SSH. You can update from Anonymous to User Auth & vice versa, but not from Anonymous/User Auth to SSH or reverse.

Note:

You can enable or disable a git account. Enabled git accounts will be available to be used in Application configuration > Git repository.

Disabled git accounts will be unavailable for use in future applications. Applications already using a disabled git account will not be affected.

Projects are nothing but a logical grouping of your applications so that you can manage and control the access level of users. We will discuss User Access in the next step.

Add Project:

Click on the Projects inside the Global configuration tab. Click on Add projects and give a name to your project and press the Save button to save your project

Why Devtron takes GitOps Configuration?

Devtron uses GitOps and stores configurations in git; Git Credentials can be entered at Global Configuration > GitOps which is used by Devtron for configuration management and storing desired state of the application configuration. In case GitOps is not configured, Devtron cannot deploy any application or charts.

Areas impacted by GitOps are:

• Deployment Template, click here to learn more.

• Charts, click here to learn more.

Add Git Configuration

Select the GitOps section of global configuration. At the top of the section, four Git providers are available.

• GitHub

• GitLab

• Azure

• BitBucket Cloud

Select one of the Git providers. To add git account, You need to provide the following inputs as given below:

1. Git Host / Azure DevOps Organisation Url / BitBucket Host

2. GitHub Organization Name / Gitlab Group id / Azure DevOps Project Name / BitBucket Workspace ID

3. BitBucket Project Key (only for BitBucket Cloud)

4. Git access credential

1. Git Host:

This field is filled by default, Showing the URL of the selected git providers. For example- https://github.com for GitHub, https://gitlab.com for GitLab, https://dev.azure.com/ for Azure & https://bitbucket.org for BitBucket. Please replace them(not available for GitHub & BitBucket) if they are not the url you want to use.

2. GitHub Organization Name / GitLab Group ID / Azure DevOps Project Name / BitBucket Workspace ID:

In the case of GitHub provide Github Organization Name*. Learn more about Github organization Name. In the case of Gitlab provide Gitlab group Id*. Learn more about Gitlab group Id. Similarly in the case of Azure provide Azure DevOps Project Name*. Learn more about Azure DevOps Project Name. For Bitbucket Cloud, provide Bitbucket Workspace Id*. Learn more about Bitbucket Workspace Id.

3. BitBucket Project Key:

This field is non-mandatory and is only to be filled when you have chosen Bitbucket Cloud as your git provider. If not provided, the oldest project in the workspace will be used. Learn more about Bitbucket Project Key.

4. Git access credential

Provide Git Username and Personal Access Token of your git account.

(a) Username Username for your git account.

(b) Personal Access Token A personal access token (PAT) is used as an alternate password to authenticate into your git accounts.

For GitHub Creating a GitHub Personal Access Token:

• repo - Full control of private repositories(Access commit status , Access deployment status and Access public repositories).

• admin:org - Full control of organizations and teams(Read and write access).

• delete_repo - Grants delete repo access on private repositories.

For GitLab Creating a GitLab Personal Access Token:

• api - Grants complete read/write access to the scoped project API.

• write_repository - Allows read-write access (pull, push) to the repository.

For Azure DevOps Creating a Azure DevOps Access Token:

• code - Grants the ability to read source code and metadata about commits, change sets, branches, and other version control artifacts. More Information on scopes in Azure devops.

For BitBucket Cloud Creating a Bitbucket Cloud Personal Access Token (App passwords)

• repo - Full control of repositories (Read, Write, Admin, Delete access).

Click on Save to save your gitOps configuration details.

Note: A Green tick will appear on the active gitOps provider.

This feature allows you to add more chart repositories to Devtron. Once added they will be available in the Discover section of the Chart Store.

Learn more about Discover chart link

Note : After the successfull installation of Devtron, click on Refresh Charts to sync & download all the default charts listed on the dashboard.

Add Chart Repository

Select the Chart Repository section of global configuration and click on Add Repository button at the top of the Chart Repository Section. To add new chart, you need to provide three inputs as below:

1. Name

2. URL

3. Authentication type

1. Name

Provide a Name to your Chart Repository. This name is added as prefix to the name of the chart in the listing on the helm chart section of application.

2. URL

Provide the URL. For example- github.com for Github, https://gitlab.com for GitLab, etc.

3. Authentication type

Here you have to provide the type of Authentication required by your version controller. We support three types of authentications, You can choose the one that suits you the best.

• Anonymous

If you select Anonymous then you do not have to provide any username, password, or authentication token. Just click on Save to save your chart repository details.

• Password/Auth token

If you select Password/Auth token, then you have to provide the Access Token for the authentication of your version controller account inside the Access token box. Click on Save to save your chart repository details.

• User Auth

If you choose User Auth then you have to provide the Username and Password of your version controller account. Click on Save to save your chart repository details.

Update Chart Repository

You can update your saved chart repository settings at any point in time. Just click on the chart repository which you want to update. Make the required changes and click on Update to save you changes.

Note: You can enable and disable your chart repository setting. If you enable it, then you will be able to see that enabled chart in Discover section of Chart Store.

Learn more about Discover chart link

API tokens are like ordinary OAuth access tokens. They can be used instead of username and password for programmatic access to API. API token allows users to generate API tokens with the desired access. Only super admin users can generate tokens and see generated tokens.

Generate API Token

To generate API tokens, go to global configurations -> Authorizations -> API tokens and click on Generate New Token.

1. Enter a name for the token

2. Add Description.

3. Select an Expiration date for the token(7 days, 30 days, 60 days, 90 days, custom and no expiration) To select a custom expiration date, select Custom from the drop-down. This will pop-up a calender from where you can select your custom expiration date for the API token.

4. Assign permissions to the token. To generate a token with super admin permission, select super admin permission. Or select specific permission if you want to generate a token with a specific role over a particular Devtron app or Helm app or chart group.

5. Now click on Generate Token.

A pop-up window will appear over the screen from where you can copy the API token.

Use API Token

Once devtron api token has been generated, you can use this token to hit devtron apis using any api testing tool like Jmeter, postman, citrus. Using postman here.

Open postman. Enter the request URL with POST method and under HEADERS, enter the API token as shown in the image below.

Now, under body, provide the api payload as shown below and click on Send.

As soon as you click on send, the create application api will be triggered and a new devtron app will be created as you mentioned in the payload.

Update API Token

To set a new expiration date or to make changes in permissions assigned to the token, we need to update the API token. To update the API token, click over the token name or click on the edit icon.

Delete API Token

To delete an API token, click on the delete icon. Any applications or scripts using this token will no longer be able to access the Devtron API.

This is used to assign user to a particular group and user inherits all the permissions granted to this group. The Permission groups section contains a drop-down of all existing groups on which you have access. This is optional field and more than one groups can be selected for a user.

The advantage of the groups is to define a set of privileges like create, edit, or delete for the given set of resources that can be shared among the users within the group. Users can be added to an existing group to utilize the privileges that it grants. Any access change to group is reflected immediately in user access.

You can select the group which you are creating in the Group permissions section inside Add users.

1. Add new Group

Go to Global configurations -> Authorization -> Permission group and click on Add Group to create a new group.

Enter the Group Name and Description.

2. Create permission Group

Once you have given the group name and group description.

Assign the permissions of groups in the Devtron Apps, Helm Apps or Group Chart permissions section. Manage the project, environment, application and role access the same as we discuss in the user permissions section.

You can add multiple rows for the Devtron Apps and Helm Apps Permissions section.

Once you have finished assigning the appropriate permissions for the permission group, click on Save.

3. Edit Group Permissions

You can edit the permission groups by clicking on the downward arrow.

Then you can edit the permission group here.

Once you are done editing the permission group, click on Save.

If you want to delete the groups with particular permission group, click on Delete.

4. Manage Chart Group Permissions

The chart group permissions for the permission groups will be managed in the same way as for the users. For reference, check Manage chart group permissions for users.

Like any enterprise product, Devtron supports fine grained access control to the resources based on

1. Type of action allowed on the Devtron resources (Create Vs View)

2. Sensitivity of the data (Editing image Vs Editing memory)

Access can be added to the User either directly or via Groups.

Role-based Access Levels

Devtron supports 5 levels of access:

1. View: User with view only access has the least privilege. This user can only view combination of environments, applications and helm charts on which access has been granted to the user. This user cannot view sensitive data like secrets used in applications or charts.

2. Build and Deploy: In addition to view privilege mentioned in above, user with build and deploy permission can build and deploy the image of permitted applications and helm charts to permitted environments.

3. Admin: User with admin access can create, edit, delete and view permitted applications in permitted projects.

4. Manager: User with manager access can do everything that an admin type user can do, in addition they can also give and revoke access of users for the applications and environments of which they are manager.

5. Super Admin: User with super admin privilege has unrestricted access to all Devtron resources. Super admin can create, modify, delete and view any Devtron resource without any restriction; its like Superman without the weakness of Kryptonite. Super Admin can also add and delete user access across any Devtron resource, add delete git repository credentials, container registry credentials, cluster and environment.

User Roles And Permissions

1. Custom Applications

2. Helm Charts

3. User Access

4. Global Configurations

To control the access of User and Group-

Go to the left main panel -> Select Global Configurations -> Select User Access

Users

1. Add new user

Click on Add User, to add one or multiple users.

2. Create User Permissions

When you click on Add User, you will see 6 options to set permission for users which are as follow:

• Email addresses

• Assign super admin permissions

• Group Permissions

• Devtron Apps Permissions

  • Project

  • Environment

  • Applications

  • Roles

• Helm Apps Permissions

  • Project

  • Environment or cluster/namespace

  • Applications

  • Permission

• Chart group permissions

Email addresses:

In the Email address box, you have to provide the mail ID of the user to whom you want to give access to your applications.

IMP Please note that Email address should be same as that in the email field in the JWT token returned by OIDC provider.

Assign super admin permissions

Click on Save and your user will be saved with super admin permissions.

We suggest that super admin privileges should be given to only select few.

If you don’t want to assign super admin permissions then you have to provide the rest of the information.

Devtron Apps permissions

Access to devtron applications can be given to user by attaching permission directly to his/her email id through the Devtron Apps section. This section has 4 options to manage the permissions of your users.

• Project

Select a project from the drop-down to which you want to give permission to the users. You can select only one project at a time if you want to select more than one project then click Add row.

• Environment

In the Environment section, you can select one or more than one or all environments at a time. Click on the environment section, you will see a drop-down of your environments and select any environment on which you want to give permission to the user.

IMP If all environments option is selected then user gets access to all current environments and any new environment which gets associated with this application later.

• Applications

Similarly, you can select Applications from the drop-down corresponding to your selected Environments. In this section, you can also give permissions to one or more than one or to all applications at a time.

IMP If all applications option is selected then user gets access to all current applications and any new application which gets associated with this project later.

• Roles

  Inside the Role, you actually choose which type of permissions you want to give to the users.

You can add multiple rows, for Devtron app permission.

Once you have finished assigning the appropriate permissions for the listed users, Click on Save.

Helm Apps Permissions

Access to devtron applications can be given to user by attaching permission directly to his/her email id through the Devtron Apps section. This section has 4 options to manage the permissions of your users.

• Project

Select a project from the drop-down to which you want to give permission to the users. You can select only one project at a time if you want to select more than one project then click Add row.

• Environment or cluster/namespace

In the Environment section, you can select one or more than one or all environments at a time. Click on the environment section, you will see a drop-down of your environments and select any environment on which you want to give permission to the user.

IMP If all environments option is selected then user gets access to all current environments and any new environment which gets associated with this application later.

• Applications

Similarly, you can select Applications from the drop-down corresponding to your selected Environments. In this section, you can also give permissions to one or more than one or to all applications at a time.

IMP If all applications option is selected then user gets access to all current applications and any new application which gets associated with this project later.

• Permission

  Inside the Role, you actually choose which type of permissions you want to give to the users.

Chart Group Permissions

You can also manage the access of users to Chart Groups in your project.

NOTE: You can only give users the ability to create or edit, not both.

Click on the checkbox of Create, if you want the users to create, view, edit, or delete the chart groups.

To permit a user to only edit the chart groups, check Specific chart group from Edit drop-down. In the following field, select the chart group for which you want to grant the user edit permission.

Go to Edit drop-down, if you want to allow or deny users to edit the chart groups.

Select on Deny option from the drop-down, if you want to restrict the users to edit the chart groups.

Select the Specific Charts option from the drop-down and then select the chart groups for which you want to allow users to edit, from the other drop-down menu.

Click on Save, once you have configured all the required permissions for the users.

3. Edit User Permissions

You can edit the user permissions, by clicking on the downward arrow.

Then you can edit the user permissions here.

After you have done editing the user permissions, click on Save.

If you want to delete the user/users with particular permissions, click on Delete.

This documentaion consist of authorizations available in Devtron

Parts of the documentaion

his feature helps you manage the notifications for your build and deployment pipelines. You can receive the notifications on Slack or via e-mail.

Click on Global Configurations -> Notifications

Notification Configuration:

Click on Configurations and you will see Devtron support two types of configurations SES Configurations or Slack Configurations.

Manage SES Configurations

You can manage the SES configuration to receive e-mails by entering the valid credentials. Make sure your e-mail is verified by SES.

Click on Add and configure SES.

Click on Save to save your SES configuration or e-mail ID

Manage Slack Configurations

You can manage the Slack configurations to receive notifications on your preferred Slack channel.

Click on Add to add new Slack Channel.

Click on Save and your slack channel will be added.

Manage Notifications

Click on Add New to receive new notification.

Manage Slack Notifications

Send To

Select Pipelines

• Then, to fetch pipelines of an application, project and environment.

  • Choose a filter type(environment, project or application)

  • You will see a list of pipelines corresponding to your selected filter type, you can select any number of pipelines. For each pipeline, there are 3 types of events Trigger, Success, and Failure. Click on the checkboxes for the events, on which you want to receive notifications.

Click on Save when you are done with your Slack notification configuration.

Manage SES Notifications

Send To

• Click on the Send To box, select your e-mail address/addresses on which you want to send e-mail notifications. Make sure e-mail id are SES Verified.

Select Pipelines

• To fetch pipelines of an application, project and environment.

  • Choose a filter type(environment, project or application)

  • You will see a list of pipelines corresponding to your selected filter type, you can select any number of pipelines. For each pipeline, there are 3 types of events Trigger, Success, and Failure. Click on the checkboxes for the events, on which you want to receive notifications.

Click on Save once you have configured the e-mail notification.

Hurray! Your Devtron stack is completely setup. Let's get started by deploying a simple application on it.

Find out the steps here

This is a sample Nodejs application which we are going to deploy using Devtron. For a detailed step-wise procedure, please have a look at the link below -

Devtron can be upgraded in one of the following ways:

Upgrade Devtron using Helm

Versions Upgrade

Upgrade Devtron from the UI

Devtron can be updated from the Devtron Stack Manager > About Devtron section.

• Select Update to Devtron

The update process may show one of the following statuses, with details available for tracking, troubleshooting, and additional information:

Updating Devtron also updates the installed integrations.

If you want to check the current version of Devtron you are using, please use the following command.

Follow the below mentioned steps to upgrade the Devtron version using Helm

1. Check the devtron release name

2. Set release name in the variable

3. Fetch the latest Devtron helm chart

4. Annotate and Label Devtron resources

5. Upgrade Devtron

5.1 Upgrade Devtron to latest version

OR

5.2 Upgrade Devtron to a custom version. You can find the latest releases from Devtron on Github https://github.com/devtron-labs/devtron/releases

If you want to check the current version of Devtron you are using, please use the following command.

Follow the below mentioned steps to upgrade the Devtron version using Helm

1. Check the devtron release name

2. Set release name in the variable

3. Annotate and Label all the Devtron resources

4. Fetch the latest Devtron helm chart

5. Upgrade Devtron

5.1 Upgrade Devtron to latest version

OR

5.2 Upgrade Devtron to a custom version. You can find the latest releases from Devtron on Github https://github.com/devtron-labs/devtron/releases

If you want to check the current version of Devtron you are using, please use the following command.

kubectl -n devtroncd get installers installer-devtron -o jsonpath='{.status.sync.data}' | grep "^LTAG=" | cut -d"=" -f2-

Follow the below mentioned steps to upgrade the Devtron version using Helm

1. Fetch the latest Devtron helm chart

helm repo update

1. Input the target Devtron version that you want to upgrade to. You can find the latest releases from Devtron on Github https://github.com/devtron-labs/devtron/releases

DEVTRON_TARGET_VERSION=v0.3.x

1. Upgrade Devtron

helm upgrade devtron devtron/devtron-operator --namespace devtroncd --set installer.release=$DEVTRON_TARGET_VERSION

Follow the below mentioned steps to upgrade the Devtron version using Kubectl

1. Input the target Devtron version that you want to upgrade to. You can find the latest releases from Devtron on Github https://github.com/devtron-labs/devtron/releases

DEVTRON_TARGET_VERSION=v0.3.x

1. Patch Devtron Installer

kubectl patch -n devtroncd installer installer-devtron --type='json' -p='[{"op": "add", "path": "/spec/reSync", "value": true },{"op": "replace", "path": "/spec/url", "value": "https://raw.githubusercontent.com/devtron-labs/devtron/'$DEVTRON_TARGET_VERSION'/manifests/installation-script"}]'

Follow the required steps to update the Devtron version

STEP 1

Delete the respective resources i.e, nats-operator , nats-streaming and nats-server using the following commands.

kubectl delete -f https://raw.githubusercontent.com/devtron-labs/devtron/v0.2.37/manifests/yamls/nats-operator.yaml
kubectl -n devtroncd delete -f https://raw.githubusercontent.com/devtron-labs/devtron/v0.2.37/manifests/yamls/nats-streaming.yaml
kubectl -n devtroncd delete -f https://raw.githubusercontent.com/devtron-labs/devtron/v0.2.37/manifests/yamls/nats-server.yaml

STEP 2

Verify the deletion of resources using the following commands.

kubectl -n devtroncd get pods 
kubectl -n devtroncd get serviceaccount
kubectl -n devtroncd get clusterrole

STEP 3

Set reSync: true in the installer object, this will initiate upgrade of the entire Devtron stack, you can use the following command to do this.

kubectl patch -n devtroncd installer installer-devtron --type='json' -p='[{"op": "add", "path": "/spec/reSync", "value": true }]'

The CI pipeline includes Pre and Post-build steps to validate and introduce checkpoints in the build process.

The pre/post plugins allow you to execute some standard tasks, such as Code analysis, Load testing, Security scanning, and so on. You can build custom pre/post tasks or use one from the standard preset plugins provided by Devtron.

Before you begin

Create a CI build pipeline if you haven't done that already!

Configuring Pre/Post-build stages

Each Pre/Post-build stage is executed as a series of events called tasks and includes custom scripts. You could create one or more tasks that are dependent on one another for execution. In other words, the output variable of one task can be used as an input for the next task to build a CI runner.

The tasks will run following the execution order.

The tasks may be re-arranged by using drag-and-drop; however, the order of passing the variables must be followed.

Creating a task

1. Go to Applications and select your application from the Devtron Apps tabs.

2. From the App Configuration tab select Workflow Editor.

3. Select the build pipeline for editing the stages.

Devtron CI pipeline includes the following build stages:

• Pre-build stage: The tasks in this stage run before the image is built.

• Build stage: In this stage, the build is triggered from the source code that you provide.

• Post-build stage: The tasks in this stage are triggered once the build is complete.

You can create a task either by selecting one of the available preset plugins or by creating a custom script.

Preset plugins | Execute custom script

Preset plugins

Prerequisite: Set up Sonarqube, or get the API keys from an admin.

The example shows a Post-build stage with a task created using a preset plugin - Sonarqube.

1. On the Edit build pipeline screen, select the Post-build stage (or Pre-build).

2. Select + Add task.

3. Select Sonarqube from PRESET PLUGINS.

Input Variables

Select Update Pipeline.

Execute custom script

1. On the Edit build pipeline screen, select the Pre-build stage.

2. Select + Add task.

3. Select Execute custom script.

The task type of the custom script may be a Shell or a Container image.

Custom script - Shell

• Select the Task type as Shell.

Consider an example that creates a Shell task to stop the build if the database name is not "mysql". The script takes 2 input variables, one is a global variable (DOCKER_IAMGE), and the other is a custom variable (DB_NAME) with a value "mysql". The task triggers only if the database name matches "mysql". If the trigger condition fails, this Pre-build task will be skipped and the build process will start. The variable DB_NAME is declared as an output variable that will be available as an input variable for the next task. The task fails if DB_NAME is not equal to "mysql".

• Select Update Pipeline.

Here is a screenshot with the failure message from the task:

Custom script - Container image

• Select the Task type as Container image.

This example creates a Pre-build task from a container image. The output variable from the previous task is available as an input variable.