GitLab CI/CD

@petejohanson

Who is this guy?

• Remote Dev/Mgr at Crossover Health
• Used most CI systems you've heard of
• Despised most of them

Traditional CI Systems

• CruiseControl
• Custom inhouse (Fireball!)
• Hudson/Jenkins
• TeamCity

Next Wave

• Travis CI
• CircleCI
• GitLab CI/CD

Continuous Integration

The practice of frequently integrating one's new or changed code with the existing code repository.

How to do this with confidence?

Practice!

Shift from occasionally performed, to routinely performed.

Enabled via automation! Automation isn't the goal, it's the technique to achieve the goal.

Drive consistency, detect problems early, remove friction, increase your bus factor!

Examples Tasks

• Compiling Code
• Running Tests
• Generating Documentation
• Building Installers

Continuous Delivery

the ability to get changes of all types—including new features, configuration changes, bug fixes and experiments—into production, or into the hands of users, safely and quickly in a sustainable way.

Examples Tasks

• Publishing to (mobile) app stores
• Performing/updating Kubernetes deployments
• Publishing libraries to repos/registries (e.g. NPM, NuGet, Maven)
• Releasing other executables (e.g. Electron app)

GitLab CI/CD

• Core part of GitLab
• Also usable for GitHub projects
• Plain text (YAML) files describe pipelines/jobs
• Docker or machine 'executors'
• Open Source!

Building Blocks

• Jobs
• Pipelines

Hello World

say_hello:                # Job name
  image: alpine:latest    # Docker image
  script:                 # List of job steps to execute
    - echo 'Hello World'  # Command to run in docker image
						

Stages

• Break tasks into logical jobs combined into a pipeline
• Different docker images, polyglot piplines
• Multiple jobs per stage

Example

stages:
  - build
  - test

image: rust:1-slim

build:
  stage: build
  script:
    - cargo build
	
test:
  stage: test
  script:
    - cargo test
						

Artifacts

Example: Test Task

• Test report (JSON)
• Code coverage report (.lcov)
• Screenshots
• Videos

Why Valuable?

• Downstream tasks (e.g. upload to code coverage service)
• Store for download
• Failure analysis
• Deployment

Produce Artifacts

test:rspec:
  stage: test
  image: ruby:2.4

  script:
    - rspec --report test-result.json
  artifacts:
    expire_in: 1 week
    paths:
      - test-results.json
      - code-coverage.lcov
						

Consume Artifacts

publish:coverage:
  stage: publish
  image: node:10

  dependencies:
    - test:rspec

  script:
    - npx some-publish-package code-coverage.lcov
						

Downloadable

https://gitlab.com/{group}/{project}/-/jobs/artifacts/{ref}/raw/{artifact_path}{?job}

Docs

• artifacts

Dependencies

Docs

• dependencies

Selective Branches/Tags

• Useful for publish operations for release tags
• Extra checks/fixes for branches for PRs

Real World Example

publish:
  stage: deploy
  only:
    - tags
  image: microsoft/dotnet:2.0.0-sdk
  script:
    - dotnet pack -c Release --include-symbols /p:VersionPrefix=$CI_COMMIT_REF_NAME
    - cd src && for dir in $(ls -d */); do dotnet nuget push ${dir%%/}/bin/Release/${dir%%/}.$CI_COMMIT_REF_NAME.nupkg -s https://www.nuget.org -k $NUGET_ORG_API_KEY; done
						

Docs

• only
• except

Environment Variables

Built In

• CI_PROJECT_NAME
• CI_COMMIT_REF_SLUG
• CI_JOB_ID

Declared In YAML

variables:
  npm_config_cache: "${CI_PROJECT_DIR}/.cache/npm"
	
test:app:e2e:
  stage: test
  image: cypress/base:10

  cache:
    key: "$CI_COMMIT_REF_SLUG"
    paths:
      - .cache/

  variables:
    CYPRESS_CACHE_FOLDER: "${CI_PROJECT_DIR}/.cache/Cypress"
						

Repository Configuration

Docs

• GitLab CI/CD Variables
• variables

Caching

• Avoid repeated work
• Most useful for immutable, mostly static content
• Biggest use case: third party packages! (.e.g Node modules, Ruby gems, Python packages)

Real World Example

variables:
  npm_config_cache: "${CI_PROJECT_DIR}/.cache/npm"
	
test:app:e2e:
  stage: test
  image: cypress/base:10
	
  cache:
    key: "$CI_COMMIT_REF_SLUG"
    paths:
    - .cache/
	
  variables:
    CYPRESS_CACHE_FOLDER: "${CI_PROJECT_DIR}/.cache/Cypress"
						

Docs

• artifacts

Services

• Run additional docker images alongside main job image
• Most useful for additional testing dependencies, e.g. database

Example

test:
  image: ruby

  services:
    - mysql

  before_script:
    - bundle
    - bundle exec rake db:schema:load
    - bundle exec rake db:seed

  script:
    - bundle exec rspec
						

Docs

• Using Docker - What Is A Service?
• services

Docker-In-Docker

Real World Example

build:image:
  image: docker:stable
  services:
    - docker:dind

  variables:
    DOCKER_HOST: tcp://docker:2375
    DOCKER_DRIVER: overlay2

  before_script:
    - docker login -u gitlab-ci-token -p $CI_JOB_TOKEN registry.gitlab.com

  script:
    - docker build --pull -t $CONTAINER_TEST_IMAGE .
    - docker push $CONTAINER_TEST_IMAGE
						

Docs

• Building Docker images with GitLab CI/CD

Templating

GitLab Template Jobs

Real World Example

.build:
  script:
    - cargo build --release

build:fedora:
  extends: .build
  image: fedora:29
  before_script:
    - dnf install cargo libdbus1-devel

build:debian:
  extends: .build
  image: debian:testing-slim
  before_script:
    - apt-get update && apt-get install cargo-devel libdbus1-dev
						

YAML Anchors

.build: &build_definition
  script:
    - cargo build --release

build:fedora:
  <<: *build_definition
  image: fedora:29
  before_script:
    - dnf install cargo libdbus1-devel

build:debian:
  <<: *build_definition
  image: debian:testing-slim
  before_script:
    - apt-get update && apt-get install cargo-devel libdbus1-dev
						

Docs

• extends

GitLab Pages

• Specially named job
• Polyglot, leveraging all the other features!
• Convention for output directory containing any HTML

Real World Example

pages:
  image: alpine:latest
  stage: deploy

  dependencies:
    - build

  script:
    - mkdir public
    - cp -R dist/docs/* public/

  artifacts:
    paths:
    - public
						

Docs

• Pages
• pages

Non- Linux/Docker Runners

• macOS
• Windows
• Alternative Architectures (ARM32/ARM64)
• FreeBSD

Environments

• Review
• Staging
• Production

deploy_staging:
  stage: deploy
  script:
    - echo "Deploy to staging server"
  environment:
    name: staging
    url: https://staging.example.com
  only:
    - master
						

deploy_prod:
  stage: deploy
  script:
    - echo "Deploy to production server"
  environment:
    name: production
    url: https://example.com
  when: manual
  only:
    - master
						

Docs

• Environments
• only
• except

Review Apps

• Build on dynamic environments and environment

Example

deploy as review app:
  stage: deploy
  script: make deploy
  environment:
    name: review/$CI_COMMIT_REF_NAME
    url: https://$CI_ENVIRONMENT_SLUG.example.com/
						

Docs

• Review Apps

Links