Setting up Opendesk on Kubernetes

Since i had little prior experience with K8S, this document serves as a sort of self-documentation of setting up opendesk, a sovereign office suite. While it might be nice to use ArgoCD for such a setup in production, I have no experience with that *yet*.

This is not a full tutorial. Just a reference for your own install and it may have some things out of order!

Ensure you have enough CPU cores and RAM or opendesk will fail to install

First of all, you want to set up DNS:

I am just going to link to the opendesk guys here:
https://gitlab.opencode.de/bmi/opendesk/deployment/opendesk/-/blob/develop/docs/getting-started.md#dns

When setting up opendesk on a second level domain, I ran into issues with ruby limits and K8S’s ndots settings: https://gitlab.opencode.de/bmi/opendesk/deployment/opendesk/-/issues/252
If you still run into this issue you may want to use https://github.com/maxlaverse/ndots-admission-controller. This is more performant anyways.

Kubernetes using KubeAdm

KUBERNETES_VERSION=v1.34
CRIO_VERSION=v1.34

apt-get update
apt-get install -y software-properties-common curl unzip

curl -fsSL https://pkgs.k8s.io/core:/stable:/$KUBERNETES_VERSION/deb/Release.key |
    gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg

echo "deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/$KUBERNETES_VERSION/deb/ /" |
    tee /etc/apt/sources.list.d/kubernetes.list

curl -fsSL https://download.opensuse.org/repositories/isv:/cri-o:/stable:/$CRIO_VERSION/deb/Release.key |
    gpg --dearmor -o /etc/apt/keyrings/cri-o-apt-keyring.gpg

echo "deb [signed-by=/etc/apt/keyrings/cri-o-apt-keyring.gpg] https://download.opensuse.org/repositories/isv:/cri-o:/stable:/$CRIO_VERSION/deb/ /" |
    tee /etc/apt/sources.list.d/cri-o.list

apt-get update
apt-get install -y cri-o kubelet kubeadm kubectl

systemctl start crio.service

swapoff -a
modprobe br_netfilter
echo 'br_netfilter' > /etc/modules-load.d/br_netfilter.conf
sysctl -w net.ipv4.ip_forward=1

kubeadm init --pod-network-cidr=10.244.0.0/16
kubectl get pods --all-namespaces

# Setup kubectl
mkdir -p $HOME/.kube
cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
chown $(id -u):$(id -g) $HOME/.kube/config

sudo sysctl -w fs.inotify.max_user_watches=2099999999
sudo sysctl -w fs.inotify.max_user_instances=2099999999
sudo sysctl -w fs.inotify.max_queued_events=2099999999

Since i’m running a single node cluster for now, I need to tell k8s that it can use the current machine:

kubectl taint nodes --all node-role.kubernetes.io/control-plane-
kubectl label nodes --all node.kubernetes.io/exclude-from-external-load-balancers-

Cri-o since kube 1.34 does some stupid name deduplication that breaks longhorn. Disable it by creating a /etc/crio/crio.conf.d/20-shortname.conf

#/etc/crio/crio.conf.d/20-shortname.conf
[crio.image]
short_name_mode = "disabled"

service crio restart

Since the normal kubectl CLI get quite limiting, I like to use the k9s TUI for easier cluster management:

wget https://github.com/derailed/k9s/releases/download/v0.50.16/k9s_linux_amd64.deb
dpkg -i k9s_linux_amd64.deb

Kubernetes needs a Container Network Interface (CNI) for networking. I chose to use Flannel just because it is a popular choice:

kubectl apply -f https://github.com/flannel-io/flannel/releases/latest/download/kube-flannel.yml

You also want to install Helm, Helmdiff, Helmfile to manage opendesk:

curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash
helm plugin install https://github.com/databus23/helm-diff
wget https://github.com/helmfile/helmfile/releases/download/v1.1.3/helmfile_1.1.3_linux_amd64.tar.gz
tar -zxvf helmfile_1.1.3_linux_amd64.tar.gz
mv helmfile /usr/local/bin

We install ingress-nginx, which i believe is going out of support but is the only currently supported ingress controller for opendesk

# This may be outdated. Take care!
helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm repo update
helm install quickstart ingress-nginx/ingress-nginx --set controller.config.annotations-risk-level=Critical   --set controller.config.strict-validate-path-type=false --set controller.allowSnippetAnnotations=true
 --set controller.admissionWebhooks.allowSnippetAnnotations=true 
 # -f values.yaml

# You may or may not need this values.yaml:
controller:
#  service:
#    type: "NodePort"
  hostPort:
    enabled: true
  service:
    type: "ClusterIP"
  config:
    annotations-risk-level: "Critical"
    strict-validate-path-type: "false"
  allowSnippetAnnotations: true
  admissionWebhooks:
    allowSnippetAnnotations: true

Storage

The local path provisioner does not work for opendesk as it needs the sticky bit. Use Longhorn!

helm repo add longhorn https://charts.longhorn.io
helm repo update
helm install longhorn longhorn/longhorn --namespace longhorn-system --create-namespace --version 1.10.0
USER=luc; PASSWORD=<snip>; echo "${USER}:$(openssl passwd -stdin -apr1 <<< ${PASSWORD})" >> auth
kubectl -n longhorn-system create secret generic basic-auth --from-file=auth
nano longhorn-ingress.yml # See below
kubectl -n longhorn-system apply -f longhorn-ingress.yml
kubectl -n longhorn-system get ingress

# longhorn-ingress.yml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: longhorn-ingress
  namespace: longhorn-system
  annotations:
    # type of authentication
    nginx.ingress.kubernetes.io/auth-type: basic
    # prevent the controller from redirecting (308) to HTTPS
    nginx.ingress.kubernetes.io/ssl-redirect: 'false'
    # name of the secret that contains the user/password definitions
    nginx.ingress.kubernetes.io/auth-secret: basic-auth
    # message to display with an appropriate context why the authentication is required
    nginx.ingress.kubernetes.io/auth-realm: 'Authentication Required '
    # custom max body size for file uploading like backing image uploading
    nginx.ingress.kubernetes.io/proxy-body-size: 10000m
spec:
  ingressClassName: nginx
  rules:
  - http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: longhorn-frontend
            port:
              number: 80

Now go to the longhorn web UI to check! Under settings, set the default replica count to 1 and minimum number of backingimage copies to 1 aswell if running single-node. You might have to do this after the opendesk install in the UI aswell since it doesnt seem to follow the defaults.

Certificates

Use cert-manager with Nginx

kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.19.1/cert-manager.yaml
kubectl create --edit -f https://raw.githubusercontent.com/cert-manager/website/master/content/docs/tutorials/acme/example/production-issuer.yaml # Change Issuer to ClusterIssuer and remove the namespace

Loadbalancer

I chose metalLB, though this has caused some issues. Can’t flannel help here?

kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.15.2/config/manifests/metallb-native.yaml
kubectl apply -f pool.yml # you need the file below
kubectl apply -f adv.yml # See below

# pool.yml
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  name: first-pool
  namespace: metallb-system
spec:
  addresses:
  - 45.136.141.166-45.136.141.168
  - 45.136.141.154-45.136.141.156 # I don't think you need this many IPs

#adv.yml
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
  name: example
  namespace: metallb-system

Install Opendesk

# Update with the newest version
wget https://gitlab.opencode.de/bmi/opendesk/deployment/opendesk/-/archive/v1.10.0/opendesk-v1.10.0.zip
unzip opendesk-v1.10.0.zip
kubectl create namespace opendesk
kubectl config set-context --current --namespace opendesk

Now, configure and start!

First of all, have you setup all required DNS settings?

Then, configure the domain:

root@Kubernetes:~/opendesk-v1.10.0# cat helmfile/environments/dev/global.yaml.gotmpl
global:
  domain: "rabevcqhguoovcu.xyz"

export MASTER_PASSWORD="<snip>"

root@Kubernetes:~/opendesk-v1.10.0# helmfile apply -e dev -n opendesk

https://gitlab.opencode.de/bmi/opendesk/deployment/opendesk/-/blob/develop/docs/getting-started.md

Bash history

Since it may help in debugging, my bash history can be found here: https://gist.github.com/Wqrld/8b9a1b6569f215d2807764f1e717aa53

Warning: It is a mess and i had no clue what i was doing. Don’t read too much into it.