Gitlab
"A single application for the complete DevOps lifecycle" - https://about.gitlab.com
Examples
Gitlab CLI
First pip3 install --user python-gitlab
, then add a ~/.python-gitlab.cfg
file like:
[global]
default = default
ssl_verify = true
timeout = 5
[default]
url = https://www.gitlab.com/
private_token = <some_value>
Where private_token is generated in https://gitlab.com/profile/personal_access_tokens, after which, you can do things like:
$ gitlab -o yaml -f name_with_namespace,web_url project list --owned=1 --per-page=1
- name_with_namespace: org-name / sub-group / project-name
web_url: https://gitlab.com/orgname/subgroup/projectname
Run CI steps locally
Using Gitlab Runner you can run stages of your CI pipeline locally. EG: if you have docker installed, you can run the following command to run the build step of your pipeline:
gitlab-runner exec docker build
The gitlab-runner
command has good inline options.
Skip CI via git push option
git push -o ci.skip
Skip CI via commit message
https://docs.gitlab.com/ce/ci/yaml/#skipping-jobs
##!/usr/bin/env bash
## Skip CI if all changed files are inside dir "foo/"
set -x
regex='^foo\/'
files=( $(git diff --cached --name-only --diff-filter=ACM ) )
for X in "${files[@]}" ; do
# If any file is not inside dir foo exit 0
if [[ ! "$X" =~ $regex ]] ; then
exit 0
fi
done
## If we've made it here, all changed files are inside dir foo/
## so we append '[no ci]' to commit message to skip CI in Gitlab
echo "[no ci]" >> "$1"
Run privileged mode gitlab-runners in GKE
https://docs.gitlab.com/runner/install/kubernetes.html#installing-gitlab-runner-using-the-helm-chart
First init the gitlab chart repo:
helm repo add gitlab https://charts.gitlab.io
Privileged mode is needed to run docker
commands, which is useful for building containers, running ansible molecule, etc..
The runners.tags
includes ${ORG_NAME}
which is great for making sure jobs run on your own runners instead of publicly shared runners. This is important because DOCKER_HOST
is different in Kubernetes than it is on public dind runners.
export REGISTRATION_TOKEN="foobar"
export ORG_NAME="acme"
helm \
install gitlab/gitlab-runner \
--name "gitlab-runner-${ORG_NAME}-$(date +%s)" \
--set "concurrent=20" \
--set "gitlabUrl=https://gitlab.com/" \
--set "runnerRegistrationToken=${REGISTRATION_TOKEN}" \
--set "runners.builds.cpuRequests=1" \
--set "runners.env.DOCKER_HOST=tcp://localhost:2375/" \
--set "runners.env.DOCKER_TLS_CERTDIR=" \
--set "runners.imagePullPolicy=always" \
--set "runners.privileged=true" \
--set "runners.request_concurrency=10" \
--set "runners.tags=${ORG_NAME}\,dind\,gke"
the runners.privileged=true
is the magic that is needed to enable docker commands in your .gitlab-ci.yml
files to succeed. this --set
flag creates the pod environment:
spec:
containers:
env:
- name: KUBERNETES_PRIVILEGED
value: "true"
runners.env.DOCKER_TLS_CERTDIR=
is required to fix the changes with Docker 19.03 outlined in https://about.gitlab.com/2019/07/31/docker-in-docker-with-docker-19-dot-03/ and https://gitlab.com/gitlab-org/gitlab-ce/issues/64959
See more variables that you can set by running helm inspect gitlab/gitlab-runner
Use helm from within the tiller pod
In Gitlab managed k8s clusters there are some TLS hurdles to jump over to get access to Helm:
kubectl exec -ti -n gitlab-managed-apps $(kubectl get pods -n gitlab-managed-apps -l app=helm,name=tiller -o name) sh
export HELM_HOST=:44134
export HELM_TLS_CA_CERT=/etc/certs/ca.crt
export HELM_TLS_CERT=/etc/certs/tls.crt
export HELM_TLS_KEY=/etc/certs/tls.key
export HELM_TLS_ENABLE=true
/helm list
Show kubernetes gitlab runner pods, their age, their job URL, and who started the job
kubectl get pods -o custom-columns='NAME:.metadata.name,START_TIME:.status.startTime,CI_JOB_URL:.spec.containers[0].env[?(@.name == "CI_JOB_URL")].value,GITLAB_USER_EMAIL:.spec.containers[0].env[?(@.name == "GITLAB_USER_EMAIL")].value' | grep -E 'NAME|jobs'
The output of the above command looks like
NAME START_TIME CI_JOB_URL GITLAB_USER_EMAIL
runner-ppzmy1zx-project-11144552-concurrent-0q2pmk 2019-10-23T17:00:56Z https://gitlab.com/foo/bar/-/jobs/330824976 user2@example.com
runner-ppzmy1zx-project-11144552-concurrent-1f7nfx 2019-10-23T17:04:27Z https://gitlab.com/foo/bar/-/jobs/330827586 user1@example.com
runner-ppzmy1zx-project-11144552-concurrent-2n84rv 2019-10-23T17:04:19Z https://gitlab.com/foo/bar/-/jobs/330827587 user1@example.com
Find k8s gitlab-runner pods that are over 1h old
kubectl get pods --no-headers=true -o custom-columns="NAME:.metadata.name,START_TIME:.status.startTime" |
grep '^runner-' |
while read -r pod starttime ; do
if [ "$(( $(date +%s) - $(gdate -d "$starttime" "+%s") ))" -gt 3600 ] ; then
echo "$pod"
fi
done
Host a private gitlab pages site
This does not appear to be a documented feature, but it is quite useful. You can host a website with a static address that tracks any given branch. Normal gitlab pages export a public facing website, but this is essentially a private gitlab pages site.
## .gitlab-ci.yml
docs:
stage: test
script:
- mkdocs build
artifacts:
paths:
- site
Then hit https://gitlab.com/${ORG}/${GROUP}/${PROJECT}/-/jobs/artifacts/master/file/site/index.html?job=docs in your browser. You will be able to browse the built website, but only if you have access to the repository.
Pros and cons
Pros
- You can create a new repo by locally initializing a git repo, setting a remote_url to where you want your project to be, and pushing your code. The server gives you a notification that the project has been created and gives you a URL to it.
- Built in docker registry for every project
- Built in CI with on-prem runners
Cons
- Push-button GKE is configured at the project level, not the group level, so setting up k8s runners is more involved than it could be.
- User permissions do not have a distinct group entity, they are managed by creating a project sub-group which functions as both a place to put code and a permission level. This shows up in a variety of places, and I suspect is the reason we cannot export groups over SAML.
- There is no command line tool equivalent to the
hub
command that github has, which makes it easier to script pull requests etc.. - Terraform provider for Gitlab is pretty limited compared to Github.
Links
- https://docs.gitlab.com/ce/administration/
- https://docs.gitlab.com/ce/ci/
- https://docs.gitlab.com/ce/workflow/gitlab_flow.html - Good git branching and review strategy for teams.
- https://docs.gitlab.com/ee/ci/docker/using_docker_build.html
- https://docs.gitlab.com/ee/ci/multi_project_pipelines.html
- https://docs.gitlab.com/ee/ci/variables/
- https://medium.com/devopslinks/gitlab-pipeline-to-run-cross-multiple-projects-3563af5d6dca
- https://gitlab.com/help/user/project/clusters/index.md Lots of info about configuring k8s clusters. (This document does not go into enough detail about using
helm
with TLS secured Gitlab apps.)