Skip to main content

Infrastructure Overview

This document describes briefly the NEW ODJ infrastructure which has its basis on the new Access-Management. While being in the implementation phase this document may be subject to change.

STACKIT-Readiness​

Goal of the new infrastructure setup was to be STACKIT-ready, which means there is no GCP lock-in effect since all managed components which are used are also available within STACKIT. Excluded are here "legacy" components which have been in use historically and will be replaced at one point with new developments.

Design Principles​

The infrastructure is deployed and managed to be high available as well the deployments have been build while having The Twelve-Factor App in mind. Further we took the chance to standardise the deployment for all domain-specific applications using a standard Helm-Chart.

Infra foundation​

The main deployment of the ODJ uses due to the non-standard character a basic-mode product within the ODJ. The project id within GPC is sit-one-digital-lvzt-run-bats. All basic-components are managed using Terraform with a standardized pipeline that enforces linting and correctness.

Terraform takes care of deploying infrastructure components such as:

  • Networking including Router and Firewall Setup
  • Private GKE-Cluster
  • PostgreSQL including Users

and supporting infrastructure software components are deployed with terraform using helm:

  • Ingress-nginx
  • Reflector
  • cert-manager

The TCP Load Balancer is deployed via the NGINX Service using a fixed IP annotation to ensure we have a fixed IP. SSL-certificates are managed using Cert-Manager and is terminated at ingress level.

A brief overview of the full deployment can be seen in the following drawing:

Infrastructure

The main terraform can be found within the Infrastructure Repository.

Domain-specific Application Deployments​

All domain-specific applications are deployed to the central kubernetes cluster using a standardized pipeline with a central Helm-Chart. The Helm-Chart can be found within the Component Deployment Reference Repository.

Shared components​

Shared components within the kubernetes cluster are mainly managed with the Infrastructure Terraform. Focus is here to use standard tools with official and stable official helm charts.

Cert-Manager​

Cert-Manager takes care of handling everything around certificates within the cluster. The default setup uses the letsencrypt ClusterIssuer and can be used with the following annotations within the Ingress:

...

  annotations:
    kubernetes.io/tls-acme: "true"
    cert-manager.io/cluster-issuer: letsencrypt-prod

...

Forward-Proxy Rules for console.odj.cloud​

Since not all components of the ODJ are deployed yet to the single kubernetes cluster. The single touchpoint is managed by special ingress rules managed through a helm chart. The helm chart can be found within the infrastructure terraform.

NGINX Forwarding Ingress-Rules

The Ingress-Rules are also setup that way that they validate every request. If a request cant be validated, NGINX redirects the user to the authentication endpoint which triggers the login flow. This is configured using standard annotations and functionality of the Ingress-nginx.

...

  annotations:
    nginx.ingress.kubernetes.io/auth-signin: https://$host/authenticate?redirect_uri=$escaped_request_uri
    nginx.ingress.kubernetes.io/auth-url: https://live.api.schwarz/odj/oauth2/v1/validate

...

Reflector​

Reflector is an operator and takes care of synchronising secrets such as for certificates from a central location in odj-common to the target namespaces. This solves for example the challenge to have a single certificate of console.odj.cloud issued from letsencrypt for all domain specific deployments across different namespaces.