Edenmal

Sysadmin Garden of Eden

GitLab + Kubernetes: GitLab on top of Kubernetes

Table of Contents
  1. 1. Intro
    1. 1.1. Decision to use sameersbn GitLab image instead of the official
    2. 1.2. Requirements
  2. 2. Step 1 - Verify Kubernetes cluster “connectivity”
  3. 3. Step 2 - Write Kubernetes manifests for GitLab
    1. 3.1. ConfigMap
    2. 3.2. Secret
    3. 3.3. StatefulSet
    4. 3.4. Service
    5. 3.5. Ingress
  4. 4. Step 3 - Create the manifests
  5. 5. Step 4 - Login to your new GitLab
  6. 6. Summary

Intro

This is the third post in the three post series about Kubernetes and GitLab. The first post can be found here: Edenmal - GitLab + Kubernetes: Perfect Match for Continuous Delivery with Container.
This post will not cover how to setup a Postgres and Redis server/cluster for the GitLab.
In the near future I may even publish all my current manifests, that also contain Postgres and Redis server/cluster manifests.

If you don’t have a Postgres and Redis, I would recommend you to check out the Kubernetes manifests at sameersbn/docker-gitlab repo, here: https://github.com/sameersbn/docker-gitlab/tree/master/kubernetes They contain a manifest for Postgres and Redis server.

Decision to use sameersbn GitLab image instead of the official

The manifests below are using sameersbn/docker-gitlab image. You may ask “Why not use the official GitLab image?”.
A good question, the answer to that lies in the concept of configuration of GitLab inside the container.
I personally prefer an image that doesn’t need a persistent volume for the configuration.
Only time I want to use a persistent volume, is when I have data from an application that needs to persistence.
Last time I looked a the official GitLab Docker image I was required to have a persistent volume for the configuration.

That is not how I imagine an application in a container to be configured. I prefer:

Having to “move” around a persistent volume which contains the config is a bad thing from my point of view.
That is the reason behind why I am using sameersbn/docker-gitlab image instead of the official gitlab/gitlab-ce image.
Additionally there are some other smaller points, for example what runs in the container is more than should. At best one process per container.

Requirements

Postgres and Redis should be reachable from inside the Kubernetes cluster.

Step 1 - Verify Kubernetes cluster “connectivity”

To check if you have proper Kubernetes cluster connection, run the following command:

$ kubectl cluster-info
kubectl cluster-info
Kubernetes master is running at https://k8s.example.com:443
KubeDNS is running at https://k8s.example.com:443/api/v1/namespaces/kube-system/services/kube-dns/proxy
[...]
To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
$

The kubectl command cluster-info shows you if you can connect to the cluster
and list the available cluster services (in the ouput: Kubernetes apiserver and KubeDNS service).

Step 2 - Write Kubernetes manifests for GitLab

As written in the Intro, the manifests will cover only the GitLab.

ConfigMap

The list of all available environment variables for the configuration, can be found here: sameersbn/docker-gitlab - Available Configuration Parameters.
If you have a variable that would contain a password or token, put it in the Secret manifest below.

apiVersion: v1
data:
# Timezone
TZ: "Europe/Berlin"
GITLAB_TIMEZONE: "Berlin"
# GitLab
GITLAB_ROOT_EMAIL: "[email protected]"
GITLAB_HOST: "gitlab.example.com"
GITLAB_PORT: "443"
GITLAB_SSH_HOST: "ssh.gitlab.example.com"
GITLAB_SSH_PORT: "22"
GITLAB_HTTPS: "true"
GITLAB_NOTIFY_ON_BROKEN_BUILDS: "true"
GITLAB_NOTIFY_PUSHER: "false"
GITLAB_PIPELINE_SCHEDULE_WORKER_CRON: "*/5 * * * *"
# GitLab Backup
GITLAB_BACKUP_SCHEDULE: "daily"
GITLAB_BACKUP_TIME: "04:30"
# GitLab DB
DB_ADAPTER: "postgresql"
DB_HOST: "__YOUR_POSTGRES_ADDRESS__"
DB_PORT: "5432"
DB_USER: "gitlab"
DB_NAME: "gitlab"
# GitLab Redis
REDIS_HOST: "__YOUR_REDIS_ADDRESS__"
REDIS_PORT: "6379"
# Nginx settings
NGINX_MAX_UPLOAD_SIZE: "100m"
# GitLab SMTP settings
GITLAB_EMAIL: "[email protected]"
GITLAB_EMAIL_DISPLAY_NAME: "GitLab ZerBytes"
GITLAB_EMAIL_REPLY_TO: "[email protected]"
SMTP_ENABLED: "true"
SMTP_DOMAIN: "example.com"
SMTP_HOST: "smtp.example.com"
SMTP_PORT: "587"
SMTP_USER: "gitlab"
SMTP_STARTTLS: "true"
SMTP_AUTHENTICATION: "login"
# Your other config vars below
kind: ConfigMap
metadata:
labels:
app: gitlab
name: gitlab-cm

Secret

The values for the data keys, need to be base64 encoded.
This can be done from the command line on most Linuxes using the base64 command. Like this:

echo -n "YOUR_VALUE" | base64 -w0

(The -w0 stops base64 wrapping after specific length per line)
In the manifest below there are variables beginning with __YOUR_...__, you need to replace those.

The manifest looks like this:

apiVersion: v1
data:
GITLAB_SECRETS_DB_KEY_BASE: __YOUR_GITLAB_SECRETS_DB_KEY_BASE__
GITLAB_SECRETS_SECRET_KEY_BASE: __YOUR_GITLAB_SECRETS_SECRET_KEY_BASE__
GITLAB_SECRETS_OTP_KEY_BASE: __YOUR_GITLAB_SECRETS_OTP_KEY_BASE__
DB_PASS: __YOUR_DB_PASS__
SMTP_PASS: __YOUR_SMTP_PASS__
GITLAB_ROOT_PASSWORD: __YOUR_GITLAB_ROOT_PASSWORD__
kind: Secret
metadata:
labels:
app: gitlab
name: gitlab-secret
type: Opaque

StatefulSet

This StatefulSet contains the image used and what is not often used envFrom. envFrom uses a list of “references” to ConfigMaps and Secrets and puts them as environment variables into the Pod.
Using envFrom has one disadvantage, when updating a ConfigMap or Secret referenced, the Pod doesn’t get “updated”/recreated.
Depending on how you look at rolling out software every change to a ConfigMap and Secret can be considered a version change aka updating the object to trigger update of the Pods.

You need to replace __STORAGE_CLASS__ with the available StorageClass in your Kubernetes cluster.

apiVersion: apps/v1beta1
kind: StatefulSet
metadata:
labels:
app: gitlab
name: gitlab
spec:
serviceName: gitlab
replicas: 1
volumeClaimTemplates:
- metadata:
name: gitlab-persistent-storage
spec:
accessModes: [ "ReadWriteOnce" ]
storageClassName: __STORAGE_CLASS__
resources:
requests:
storage: 50Gi
template:
metadata:
labels:
app: gitlab
spec:
containers:
- name: gitlab
image: sameersbn/gitlab:10.1.1
imagePullPolicy: Always
envFrom:
- configMapRef:
name: gitlab-cm
- secretRef:
name: gitlab-secret
ports:
- containerPort: 22
name: ssh
protocol: TCP
- containerPort: 80
name: http
protocol: TCP
resources:
limits:
cpu: "2"
memory: 4Gi
requests:
cpu: "500m"
memory: "1Gi"
volumeMounts:
- name: gitlab-persistent-storage
mountPath: /home/git/data

Service

This Service is for the Ingress to be able to reach the GitLab.

apiVersion: v1
kind: Service
metadata:
labels:
app: gitlab
name: gitlab
spec:
ports:
- name: ssh
port: 22
protocol: TCP
- name: http
port: 80
protocol: TCP
selector:
app: gitlab

Ingress

To be able to reach the GitLab from outside the cluster over your Ingress controller of choice.
The Ingress requires the Service created in Service.
You need to replace __INGRESS_CLASS__ with your in cluster configured Ingress controller.

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: gitlab-examplecom
annotations:
kubernetes.io/ingress.class: __INGRESS_CLASS__
spec:
rules:
- host: gitlab.example.com
http:
paths:
- path: /
backend:
serviceName: gitlab
servicePort: 80

Step 3 - Create the manifests

You can either a) save all manifests in one file, but then separated by --- or b) per manifest on file.
To create/run the mnaifests on Kubernetes, you can now go ahead an run:

kubectl create -f FILE_NAME

Where FILE_NAME is the name of the file containing the mnaifest(s).
If the command I suggest you take a look at the line which kubectl told you the error is.

Step 4 - Login to your new GitLab

If everything was successfull, you should be able to see your GitLab instance in,
depending on the server network bandwith and GitLab database setup speed, about 10-15 minutes.

You should then be able to login to your GitLab over the domain name you used in the Ingress manifest.
With the example values above the address would be https://gitlab.example.com.
The login information for the first “root” user is:

Summary

I hope this gives a good idea on how to run GitLab on Kubernetes or even simply a starting point for extending/customizing the manifests for your needs.
Another starting point that also contains manifests for a single Postgres and Redis server, can be found here: https://github.com/sameersbn/docker-gitlab/tree/master/kubernetes

Have Fun!