Since i had little prior experience with K8S, this document serves as a sort of self-documentation of setting up opendesk, a sovereign office suite. While it might be nice to use ArgoCD for such a setup in production, I have no experience with that *yet*.
This is not a full tutorial. Just a reference for your own install and it may have some things out of order!
Ensure you have enough CPU cores and RAM or opendesk will fail to install
First of all, you want to set up DNS:
I am just going to link to the opendesk guys here:
https://gitlab.opencode.de/bmi/opendesk/deployment/opendesk/-/blob/develop/docs/getting-started.md#dns
When setting up opendesk on a second level domain, I ran into issues with ruby limits and K8S’s ndots settings: https://gitlab.opencode.de/bmi/opendesk/deployment/opendesk/-/issues/252
If you still run into this issue you may want to use https://github.com/maxlaverse/ndots-admission-controller. This is more performant anyways.
Kubernetes using KubeAdm
I chose kubeadm as it seems to be a bit more production-oriented than minikube. Interestingly, i found the cri-o site to be the best source for information regarding K8S installation. Cri-o is a container runtime for Kubernetes similar to containerd. I will be using ubuntu 24.04 as the base.
KUBERNETES_VERSION=v1.34
CRIO_VERSION=v1.34
apt-get update
apt-get install -y software-properties-common curl unzip
curl -fsSL https://pkgs.k8s.io/core:/stable:/$KUBERNETES_VERSION/deb/Release.key |
gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
echo "deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/$KUBERNETES_VERSION/deb/ /" |
tee /etc/apt/sources.list.d/kubernetes.list
curl -fsSL https://download.opensuse.org/repositories/isv:/cri-o:/stable:/$CRIO_VERSION/deb/Release.key |
gpg --dearmor -o /etc/apt/keyrings/cri-o-apt-keyring.gpg
echo "deb [signed-by=/etc/apt/keyrings/cri-o-apt-keyring.gpg] https://download.opensuse.org/repositories/isv:/cri-o:/stable:/$CRIO_VERSION/deb/ /" |
tee /etc/apt/sources.list.d/cri-o.list
apt-get update
apt-get install -y cri-o kubelet kubeadm kubectl
systemctl start crio.service
swapoff -a
modprobe br_netfilter
echo 'br_netfilter' > /etc/modules-load.d/br_netfilter.conf
sysctl -w net.ipv4.ip_forward=1
kubeadm init --pod-network-cidr=10.244.0.0/16
kubectl get pods --all-namespaces# Setup kubectl
mkdir -p $HOME/.kube
cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
chown $(id -u):$(id -g) $HOME/.kube/configsudo sysctl -w fs.inotify.max_user_watches=2099999999
sudo sysctl -w fs.inotify.max_user_instances=2099999999
sudo sysctl -w fs.inotify.max_queued_events=2099999999You might get very close to the pod limit which gives very cryptic errors (110), so up it:
#nano /var/lib/kubelet/config.yaml
maxPods: 1024sudo systemctl restart kubeletSince i’m running a single node cluster for now, I need to tell k8s that it can use the current machine:
kubectl taint nodes --all node-role.kubernetes.io/control-plane-
kubectl label nodes --all node.kubernetes.io/exclude-from-external-load-balancers-Cri-o since kube 1.34 does some stupid name deduplication that breaks longhorn. Disable it by creating a /etc/crio/crio.conf.d/20-shortname.conf
#/etc/crio/crio.conf.d/20-shortname.conf
[crio.image]
short_name_mode = "disabled"service crio restart
Since the normal kubectl CLI get quite limiting, I like to use the k9s TUI for easier cluster management:
wget https://github.com/derailed/k9s/releases/download/v0.50.16/k9s_linux_amd64.deb
dpkg -i k9s_linux_amd64.debKubernetes needs a Container Network Interface (CNI) for networking. I chose to use Flannel just because it is a popular choice:
kubectl apply -f https://github.com/flannel-io/flannel/releases/latest/download/kube-flannel.ymlYou also want to install Helm, Helmdiff, Helmfile to manage opendesk:
curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash
helm plugin install https://github.com/databus23/helm-diff
wget https://github.com/helmfile/helmfile/releases/download/v1.1.3/helmfile_1.1.3_linux_amd64.tar.gz
tar -zxvf helmfile_1.1.3_linux_amd64.tar.gz
mv helmfile /usr/local/binWe install ingress-nginx, which i believe is going out of support but is the only currently supported ingress controller for opendesk
# This may be outdated. Take care!
helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm repo update
helm install quickstart ingress-nginx/ingress-nginx --set controller.config.annotations-risk-level=Critical --set controller.config.strict-validate-path-type=false --set controller.allowSnippetAnnotations=true
--set controller.admissionWebhooks.allowSnippetAnnotations=true
# -f values.yaml# You may or may not need this values.yaml:
controller:
# service:
# type: "NodePort"
hostPort:
enabled: true
service:
type: "ClusterIP"
config:
annotations-risk-level: "Critical"
strict-validate-path-type: "false"
allowSnippetAnnotations: true
admissionWebhooks:
allowSnippetAnnotations: trueStorage
The local path provisioner does not work for opendesk as it needs the sticky bit. Use Longhorn!
helm repo add longhorn https://charts.longhorn.io
helm repo update
helm install longhorn longhorn/longhorn --namespace longhorn-system --create-namespace --version 1.10.0
USER=luc; PASSWORD=<snip>; echo "${USER}:$(openssl passwd -stdin -apr1 <<< ${PASSWORD})" >> auth
kubectl -n longhorn-system create secret generic basic-auth --from-file=auth
nano longhorn-ingress.yml # See below
kubectl -n longhorn-system apply -f longhorn-ingress.yml
kubectl -n longhorn-system get ingress# longhorn-ingress.yml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: longhorn-ingress
namespace: longhorn-system
annotations:
# type of authentication
nginx.ingress.kubernetes.io/auth-type: basic
# prevent the controller from redirecting (308) to HTTPS
nginx.ingress.kubernetes.io/ssl-redirect: 'false'
# name of the secret that contains the user/password definitions
nginx.ingress.kubernetes.io/auth-secret: basic-auth
# message to display with an appropriate context why the authentication is required
nginx.ingress.kubernetes.io/auth-realm: 'Authentication Required '
# custom max body size for file uploading like backing image uploading
nginx.ingress.kubernetes.io/proxy-body-size: 10000m
spec:
ingressClassName: nginx
rules:
- http:
paths:
- pathType: Prefix
path: "/"
backend:
service:
name: longhorn-frontend
port:
number: 80Now go to the longhorn web UI to check! Under settings, set the default replica count to 1 and minimum number of backingimage copies to 1 aswell if running single-node. You might have to do this after the opendesk install in the UI aswell since it doesnt seem to follow the defaults.
Certificates
Use cert-manager with Nginx
kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.19.1/cert-manager.yaml
kubectl create --edit -f https://raw.githubusercontent.com/cert-manager/website/master/content/docs/tutorials/acme/example/production-issuer.yaml # Change Issuer to ClusterIssuer and remove the namespaceLoadbalancer
I chose metalLB, though this has caused some issues. Can’t flannel help here?
kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.15.2/config/manifests/metallb-native.yaml
kubectl apply -f pool.yml # you need the file below
kubectl apply -f adv.yml # See below# pool.yml
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
name: first-pool
namespace: metallb-system
spec:
addresses:
- 45.136.141.166-45.136.141.168
- 45.136.141.154-45.136.141.156 # I don't think you need this many IPs#adv.yml
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
name: example
namespace: metallb-systemInstall Opendesk
# Update with the newest version
wget https://gitlab.opencode.de/bmi/opendesk/deployment/opendesk/-/archive/v1.10.0/opendesk-v1.10.0.zip
unzip opendesk-v1.10.0.zip
kubectl create namespace opendesk
kubectl config set-context --current --namespace opendeskNow, configure and start!
First of all, have you setup all required DNS settings?
Then, configure the domain:
root@Kubernetes:~/opendesk-v1.10.0# cat helmfile/environments/dev/global.yaml.gotmpl
global:
domain: "rabevcqhguoovcu.xyz"export MASTER_PASSWORD="<snip>"root@Kubernetes:~/opendesk-v1.10.0# helmfile apply -e dev -n opendeskhttps://gitlab.opencode.de/bmi/opendesk/deployment/opendesk/-/blob/develop/docs/getting-started.md
Bash history
Since it may help in debugging, my bash history can be found here: https://gist.github.com/Wqrld/8b9a1b6569f215d2807764f1e717aa53
Warning: It is a mess and i had no clue what i was doing. Don’t read too much into it.