You can configure Devtron by using configuration files. Configuration files are YAML files which are user-friendly. The configuration allows you to quickly roll back a configuration change if necessary. It also aids cluster re-creation and restoration.

There are two ways you can perform configurations while setting up Devtron dashboard:

• Installation Configurations

• Override Configurations

You can also setup ingress while setting up Devtron dashboard. Refer here for ingress setup.

Introduction

If you wish to use Ingress as a means to access the Devtron services available in your cluster, you can configure it either during the installation or after the installation of Devtron.

Refer the section relevant to you:

• During Devtron Installation

• After Devtron Installation

If you have successfully configured Ingress, refer Post Ingress Setup.

Enabling Ingress during Devtron Installation

If you are installing Devtron, you can enable Ingress either via set flag or by using ingress-values.yaml to specify the desired Ingress settings.

Using set flag

You can use the --set flag to specify the desired Ingress settings.

Here, we have added 5 configurations you can perform depending on your requirements:

• Only Basic Configuration

• Configuration Including Labels

• Configuration Including Annotations

• Configuration Including TLS Settings

• Comprehensive Configuration

Only Basic Configuration

To enable Ingress and set basic parameters, use the following command:

helm install devtron devtron/devtron-operator -n devtroncd \
  --set components.devtron.ingress.enabled=true \
  --set components.devtron.ingress.className=nginx \
  --set components.devtron.ingress.host=devtron.example.com

Configuration Including Labels

To add labels to the Ingress resource, use the following command:

helm install devtron devtron/devtron-operator -n devtroncd \
  --set components.devtron.ingress.enabled=true \
  --set components.devtron.ingress.className=nginx \
  --set components.devtron.ingress.host=devtron.example.com \
  --set components.devtron.ingress.labels.env=production

Configuration Including Annotations

To add annotations to the Ingress resource, use the following command:

helm install devtron devtron/devtron-operator -n devtroncd \
  --set components.devtron.ingress.enabled=true \
  --set components.devtron.ingress.className=nginx \
  --set components.devtron.ingress.host=devtron.example.com \
  --set components.devtron.ingress.annotations."kubernetes\.io/ingress\.class"=nginx \
  --set components.devtron.ingress.annotations."nginx\.ingress\.kubernetes\.io\/app-root"="/dashboard"

Configuration Including TLS Settings

To configure TLS settings, including secretName and hosts, use the following command:

helm install devtron devtron/devtron-operator -n devtroncd \
  --set components.devtron.ingress.enabled=true \
  --set components.devtron.ingress.className=nginx \
  --set components.devtron.ingress.host=devtron.example.com \
  --set components.devtron.ingress.tls[0].secretName=devtron-tls \
  --set components.devtron.ingress.tls[0].hosts[0]=devtron.example.com

Comprehensive Configuration

To include all the above settings in a single command, use:

helm install devtron devtron/devtron-operator -n devtroncd \
  --set components.devtron.ingress.enabled=true \
  --set components.devtron.ingress.className=nginx \
  --set components.devtron.ingress.host=devtron.example.com \
  --set components.devtron.ingress.annotations."kubernetes\.io/ingress\.class"=nginx \
  --set components.devtron.ingress.annotations."nginx\.ingress\.kubernetes\.io\/app-root"="/dashboard" \
  --set components.devtron.ingress.labels.env=production \
  --set components.devtron.ingress.pathType=ImplementationSpecific \
  --set components.devtron.ingress.tls[0].secretName=devtron-tls \
  --set components.devtron.ingress.tls[0].hosts[0]=devtron.example.com

Using ingress-values.yaml

As an alternative to the set flag method, you can enable Ingress using ingress-values.yaml instead.

Create an ingress-values.yaml file. You may refer the below format for an advanced ingress configuration which includes labels, annotations, secrets, and many more.

components:
  devtron:
    ingress:
      enabled: true
      className: nginx
      labels: {}
        # env: production
      annotations: {}
        # nginx.ingress.kubernetes.io/app-root: /dashboard
      pathType: ImplementationSpecific
      host: devtron.example.com
      tls: []
    #    - secretName: devtron-info-tls
    #      hosts:
    #        - devtron.example.com

Once you have the ingress-values.yaml file ready, run the following command:

helm install devtron devtron/devtron-operator -n devtroncd  --reuse-values  -f ingress-values.yaml

Configuring Ingress after Devtron Installation

After Devtron is installed, Devtron is accessible through devtron-service. If you wish to access Devtron through ingress, you'll need to modify this service to use a ClusterIP instead of a LoadBalancer.

You can do this using the kubectl patch command:

kubectl patch -n devtroncd svc devtron-service -p '{"spec": {"ports": [{"port": 80,"targetPort": "devtron","protocol": "TCP","name": "devtron"}],"type": "ClusterIP","selector": {"app": "devtron"}}}'

Next, create ingress to access Devtron by applying the devtron-ingress.yaml file. The file is also available on this link. You can access Devtron from any host after applying this yaml.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations: 
    nginx.ingress.kubernetes.io/app-root: /dashboard
  labels:
    app: devtron
    release: devtron
  name: devtron-ingress
  namespace: devtroncd
spec:
  ingressClassName: nginx
  rules:
  - http:
      paths:
      - backend:
          service:
            name: devtron-service
            port:
              number: 80
        path: /orchestrator
        pathType: ImplementationSpecific 
      - backend:
          service:
            name: devtron-service
            port:
              number: 80
        path: /dashboard
        pathType: ImplementationSpecific
      - backend:
          service:
            name: devtron-service
            port:
              number: 80
        path: /grafana
        pathType: ImplementationSpecific  

For k8s versions < 1.19, apply this yaml:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations: 
    nginx.ingress.kubernetes.io/app-root: /dashboard
  labels:
    app: devtron
    release: devtron
  name: devtron-ingress
  namespace: devtroncd
spec:
  rules:
  - http:
      paths:
      - backend:
          serviceName: devtron-service
          servicePort: 80
        path: /orchestrator
      - backend:
          serviceName: devtron-service
          servicePort: 80
        path: /dashboard
        pathType: ImplementationSpecific  

Optionally, you also can access Devtron through a specific host by running the following YAML file:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations: 
    nginx.ingress.kubernetes.io/app-root: /dashboard
  labels:
    app: devtron
    release: devtron
  name: devtron-ingress
  namespace: devtroncd
spec:
  ingressClassName: nginx
  rules:
    - host: devtron.example.com
      http:
        paths:
          - backend:
              service:
                name: devtron-service
                port:
                  number: 80
            path: /orchestrator
            pathType: ImplementationSpecific
          - backend:
              service:
                name: devtron-service
                port:
                  number: 80
            path: /dashboard
            pathType: ImplementationSpecific
          - backend:
              service:
                name: devtron-service
                port:
                  number: 80
            path: /grafana
            pathType: ImplementationSpecific

Enable HTTPS For Devtron

Once Ingress setup for Devtron is done and you want to run Devtron over https, you need to add different annotations for different ingress controllers and load balancers.

1. Nginx Ingress Controller

In case of nginx ingress controller, add the following annotations under service.annotations under nginx ingress controller to run devtron over https.

(i) Amazon Web Services (AWS)

If you are using AWS cloud, add the following annotations under service.annotations under nginx ingress controller.

  annotations:
    service.beta.kubernetes.io/aws-load-balancer-backend-protocol: "http"
    service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "443"
    nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
    service.beta.kubernetes.io/aws-load-balancer-ssl-cert: "<acm-arn-here>"

(ii) Digital Ocean

If you are using Digital Ocean cloud, add the following annotations under service.annotations under nginx ingress controller.

annotations:
  service.beta.kubernetes.io/do-loadbalancer-protocol: "http"
  service.beta.kubernetes.io/do-loadbalancer-tls-ports: "443"
  service.beta.kubernetes.io/do-loadbalancer-certificate-id: "<your-certificate-id>"
  service.beta.kubernetes.io/do-loadbalancer-redirect-http-to-https: "true"

2. AWS Application Load Balancer (AWS ALB)

In case of AWS application load balancer, add following annotations under ingress.annotations to run devtron over https.

  annotations:
    kubernetes.io/ingress.class: alb
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS": 443}]'
    alb.ingress.kubernetes.io/certificate-arn: "<acm-arn-here>"

3. Azure Application Gateway

In case of AWS application load balancer, the following annotations need to be added under ingress.annotations to run devtron over https.

 annotations:
  kubernetes.io/ingress.class: "azure/application-gateway"
  appgw.ingress.kubernetes.io/backend-protocol: "http"
  appgw.ingress.kubernetes.io/ssl-redirect: "true"
  appgw.ingress.kubernetes.io/appgw-ssl-certificate: "<name-of-appgw-installed-certificate>"

For an Ingress resource to be observed by AGIC (Application Gateway Ingress Controller) must be annotated with kubernetes.io/ingress.class: azure/application-gateway. Only then AGIC will work with the Ingress resource in question.

Note: Make sure NOT to use port 80 with HTTPS and port 443 with HTTP on the Pods.

In certain cases, you may want to override default configurations provided by Devtron. For example, for deployments or statefulsets you may want to change the memory or CPU requests or limit or add node affinity or taint tolerance. Say, for ingress, you may want to add annotations or host. Samples are available inside the manifests/updates directory.

To modify a particular object, it looks in namespace devtroncd for the corresponding configmap as mentioned in the mapping below:

Let's take an example to understand how to override specific values. Say, you want to override annotations and host in the ingress, i.e., you want to change devtronIngress, copy the file devtron-ingress-override.yaml. This file contains a configmap to modify devtronIngress as mentioned above. Please note the structure of this configmap, data should have the key override with a multiline string as a value.

apiVersion, kind, metadata.name in the multiline string is used to match the object which needs to be modified. In this particular case it will look for apiVersion: extensions/v1beta1, kind: Ingress and metadata.name: devtron-ingress and will apply changes mentioned inside update: as per the example inside the metadata: it will add annotations owner: app1 and inside spec.rules.http.host it will add http://change-me.

In case you want to change multiple objects, for eg in argocd you want to change the config of argocd-dex-server as well as argocd-redis then follow the example in devtron-argocd-override.yaml.

Once we have made these changes in our local system we need to apply them to a Kubernetes cluster on which Devtron is installed currently using the below command:

kubectl apply -f file-name -n devtroncd

Run the following command to make these changes take effect:

kubectl patch -n devtroncd installer installer-devtron --type='json' -p='[{"op": "add", "path": "/spec/reSync", "value": true }]'

Our changes would have been propagated to Devtron after 20-30 minutes.

Recommended Resources for Production use

To use Devtron for production deployments, use our recommended production overrides located in manifests/updates/production. This configuration should be enough for handling up to 200 microservices.

The overall resources required for the recommended production overrides are:

The production overrides can be applied as pre-devtron installation as well as post-devtron installation in the respective namespace.

Pre-Devtron Installation

If you want to install a new Devtron instance for production-ready deployments, this is the best option for you.

Create the namespace and apply the overrides files as stated above:

kubectl create ns devtroncd

After files are applied, you are ready to install your Devtron instance with production-ready resources.

Post-Devtron Installation

If you have an existing Devtron instance and want to migrate it for production-ready deployments, this is the right option for you.

In the existing namespace, apply the production overrides as we do it above.

kubectl apply -f prod-configs -n devtroncd

Configure Secrets

For Helm installation this section refers to secrets section of values.yaml.

Configure the following properties:

Configure ConfigMaps

For Helm installation this section refers to configs section of values.yaml.

Configure the following properties:

Configure Resources

Devtron provides ways to control how much memory or CPU can be allocated to each Devtron microservice. You can adjust the resources that are allocated to these microservices based on your requirements. The resource configurations are available in following sizes:

Small: To configure the small resources (e.g. to manage less than 10 apps on Devtron ) based on the requirements, append the Devtron installation command with -f https://raw.githubusercontent.com/devtron-labs/devtron/main/charts/devtron/resources-small.yaml.

Configure Overrides

For Helm installation this section refers to customOverrides section of values.yaml. In this section you can override values of devtron-cm which you want to keep persistent. For example:

You can configure the following properties:

Storage for Logs and Cache

AWS SPECIFIC

While installing Devtron and using the AWS-S3 bucket for storing the logs and caches, the below parameters are to be used in the ConfigMap.

NOTE: For using the S3 bucket it is important to add the S3 permission policy to the IAM role attached to the nodes of the cluster.

The below parameters are to be used in the Secrets :

AZURE SPECIFIC

While installing Devtron using Azure Blob Storage for storing logs and caches, the below parameters will be used in the ConfigMap.

GOOGLE CLOUD STORAGE SPECIFIC

While installing Devtron using Google Cloud Storage for storing logs and caches, the below parameters will be used in the ConfigMap.

To convert string to base64 use the following command:

echo -n "string" | base64

Note:

1. Ensure that the cluster has read and write access to the S3 buckets/Azure Blob storage container mentioned in DEFAULT_CACHE_BUCKET, DEFAULT_BUILD_LOGS_BUCKET or AZURE_BLOB_CONTAINER_CI_LOG, or AZURE_BLOB_CONTAINER_CI_CACHE.

2. Ensure that the cluster has read access to AWS secrets backends (SSM & secrets manager).

The following tables contain parameters and their details for Secrets and ConfigMaps that are configured during the installation of Devtron. If the installation is done using Helm, the values can be tweaked in values.yaml file.

We can use the --set flag to override the default values when installing with Helm. For example, to update POSTGRESQL_PASSWORD and BLOB_STORAGE_PROVIDER, use the install command as:

helm install devtron devtron/devtron-operator --create-namespace --namespace devtroncd \
--set secrets.POSTGRESQL_PASSWORD=change-me \
--set configs.BLOB_STORAGE_PROVIDER=S3

Configuration of Blob Storage

Blob Storage allows users to store large amounts of unstructured data. Unstructured data is a data that does not adhere to a particular data model or definition, such as text or binary data. Configuring blob storage in your Devtron environment allows you to store build logs and cache.

In case, if you do not configure the Blob Storage, then:

• You will not be able to access the build and deployment logs after an hour.

• Build time for commit hash takes longer as cache is not available.

• Artifact reports cannot be generated in pre/post build and deployment stages.

You can configure Blob Storage with one of the following Blob Storage providers given below:

Note: You can also use the respective following command to switch to another Blob Storage provider. As an example, If you are using MinIO Storage and want to switch to Azure Blob Storage, use the command provided on the Azure Blob Storage tab to switch.

helm repo update

helm upgrade devtron devtron/devtron-operator --namespace devtroncd \
--reuse-values \
--set installer.modules={cicd} \
--set minio.enabled=true

helm repo update
helm upgrade devtron devtron/devtron-operator --namespace devtroncd \
--reuse-values \
--set installer.modules={cicd} \
--set configs.BLOB_STORAGE_PROVIDER=S3 \
--set configs.DEFAULT_CACHE_BUCKET=demo-s3-bucket \
--set configs.DEFAULT_CACHE_BUCKET_REGION=us-east-1 \
--set configs.DEFAULT_BUILD_LOGS_BUCKET=demo-s3-bucket \
--set configs.DEFAULT_CD_LOGS_BUCKET_REGION=us-east-1

helm repo update

helm upgrade devtron devtron/devtron-operator --namespace devtroncd \
--reuse-values \
--set installer.modules={cicd} \
--set configs.BLOB_STORAGE_PROVIDER=S3 \
--set configs.DEFAULT_CACHE_BUCKET=demo-s3-bucket \
--set configs.DEFAULT_CACHE_BUCKET_REGION=us-east-1 \
--set configs.DEFAULT_BUILD_LOGS_BUCKET=demo-s3-bucket \
--set configs.DEFAULT_CD_LOGS_BUCKET_REGION=us-east-1 \
--set secrets.BLOB_STORAGE_S3_ACCESS_KEY=<access-key> \
--set secrets.BLOB_STORAGE_S3_SECRET_KEY=<secret-key>

helm repo update

helm upgrade devtron devtron/devtron-operator --namespace devtroncd \
--reuse-values \
--set installer.modules={cicd} \
--set configs.BLOB_STORAGE_PROVIDER=S3 \
--set configs.DEFAULT_CACHE_BUCKET=demo-s3-bucket \
--set configs.DEFAULT_CACHE_BUCKET_REGION=us-east-1 \
--set configs.DEFAULT_BUILD_LOGS_BUCKET=demo-s3-bucket \
--set configs.DEFAULT_CD_LOGS_BUCKET_REGION=us-east-1 \
--set secrets.BLOB_STORAGE_S3_ACCESS_KEY=<access-key> \
--set secrets.BLOB_STORAGE_S3_SECRET_KEY=<secret-key> \
--set configs.BLOB_STORAGE_S3_ENDPOINT=<endpoint>

helm repo update
helm upgrade devtron devtron/devtron-operator --namespace devtroncd \
--reuse-values \
--set installer.modules={cicd} \
--set secrets.AZURE_ACCOUNT_KEY=xxxxxxxxxx \
--set configs.BLOB_STORAGE_PROVIDER=AZURE \
--set configs.AZURE_ACCOUNT_NAME=test-account \
--set configs.AZURE_BLOB_CONTAINER_CI_LOG=ci-log-container \
--set configs.AZURE_BLOB_CONTAINER_CI_CACHE=ci-cache-container

helm repo update

helm upgrade devtron devtron/devtron-operator --namespace devtroncd \
--reuse-values \
--set installer.modules={cicd} \
--set configs.BLOB_STORAGE_PROVIDER=GCP \
--set secrets.BLOB_STORAGE_GCP_CREDENTIALS_JSON=eyJ0eXBlIjogInNlcnZpY2VfYWNjb3VudCIsInByb2plY3RfaWQiOiAiPHlvdXItcHJvamVjdC1pZD4iLCJwcml2YXRlX2tleV9pZCI6ICI8eW91ci1wcml2YXRlLWtleS1pZD4iLCJwcml2YXRlX2tleSI6ICI8eW91ci1wcml2YXRlLWtleT4iLCJjbGllbnRfZW1haWwiOiAiPHlvdXItY2xpZW50LWVtYWlsPiIsImNsaWVudF9pZCI6ICI8eW91ci1jbGllbnQtaWQ+IiwiYXV0aF91cmkiOiAiaHR0cHM6Ly9hY2NvdW50cy5nb29nbGUuY29tL28vb2F1dGgyL2F1dGgiLCJ0b2tlbl91cmkiOiAiaHR0cHM6Ly9vYXV0aDIuZ29vZ2xlYXBpcy5jb20vdG9rZW4iLCJhdXRoX3Byb3ZpZGVyX3g1MDlfY2VydF91cmwiOiAiaHR0cHM6Ly93d3cuZ29vZ2xlYXBpcy5jb20vb2F1dGgyL3YxL2NlcnRzIiwiY2xpZW50X3g1MDlfY2VydF91cmwiOiAiPHlvdXItY2xpZW50LWNlcnQtdXJsPiJ9Cg== \
--set configs.DEFAULT_CACHE_BUCKET=cache-bucket \
--set configs.DEFAULT_BUILD_LOGS_BUCKET=log-bucket

Secrets

ConfigMaps

Dashboard Configurations

RECOMMEND_SECURITY_SCANNING=false
FORCE_SECURITY_SCANNING=false
HIDE_DISCORD=false