Continuous Delivery Guide — Part 5/5

Hello guys, how are you? I hope you’re all doing fine!

This will be a five part article, in which we’ll discuss:

  1. What is Continuous Delivery and why is that important?
  2. What are the available tools and how to choose one?
  3. Spinnaker: I want to sail in this one!
  4. Jenkins X: New way to look the good old Jenkins
  5. Concourse CI: New guy in town

I really hope you find this series useful, and if you do, please leave a comment or DM me on LinkedIn.

Concourse is a pipeline-based continuous thing-doer.

But what is a "thing-doer"? It means that Concourse is capable of doing things in a scalable and declarative way. It works around YAML and with it you can define your pipeline in a portable and less riskier way.

How does that works?

Concourse has a not so simple yet easy to understand architecture:

architecture of concourse ci taken from the official docs

The ATC is the main component of the product. It contains the Web UI and the scheduler of pipelines. You can have multiple ATC, which composes a cluster of concourse, but the all should connect to the same PostgreSQL.

The TSA is the worker registration component. Whenever you want to run a pipeline, the worker machine must be registered to the ACL, which controls the job distribution. After the registration, this machine is able to run pipelines. It could be a container, a VM or even a bare metal machine.

And there is the Worker component, which is subdivided in three major services: The Beacon, The Garden and The Baggageclaim.

The Beacon is like a queue, he will send the address of the workers to the ACL. Those address could be used by the Garden, which runs containers and the Baggageclaim, which manages disks and volumes, mounted to containers.

The Command Line

In order to manage your Concourse CI environment, we must use the fly tool.

It is fairly easy to install and the help section is really great. We are going to install it together in a moment.

How are we going to run this?

I believe the easier way to setup a lab environment is through Docker. We are going to use the official docker-compose file. But first, if you haven't done this yet, let's install our tools:

curl -fsSL https://get.docker.com | bashcurl -L "https://github.com/docker/compose/releases/download/1.24.0/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose

Check if there is a daemon running and if docker-compose is on path:

docker version
docker-compose version

Now, go to the docker repo on the Concourse Github and download the docker-compose file. Here is:

version: '3'

services:
db:
image: postgres
environment:
POSTGRES_DB: concourse
POSTGRES_USER: concourse_user
POSTGRES_PASSWORD: concourse_pass
logging:
driver: "json-file"
options:
max-file: "5"
max-size: "10m"

web:
image: concourse/concourse
command: web
links: [db]
depends_on: [db]
ports: ["8080:8080"]
volumes: ["./keys/web:/concourse-keys"]
environment:
CONCOURSE_EXTERNAL_URL: http://localhost:8080
CONCOURSE_POSTGRES_HOST: db
CONCOURSE_POSTGRES_USER: concourse_user
CONCOURSE_POSTGRES_PASSWORD: concourse_pass
CONCOURSE_POSTGRES_DATABASE: concourse
CONCOURSE_ADD_LOCAL_USER: test:test
CONCOURSE_MAIN_TEAM_LOCAL_USER: test
logging:
driver: "json-file"
options:
max-file: "5"
max-size: "10m"

worker:
image: concourse/concourse
command: worker
privileged: true
depends_on: [web]
volumes: ["./keys/worker:/concourse-keys"]
links: [web]
stop_signal: SIGUSR2
environment:
CONCOURSE_TSA_HOST: web:2222
logging:
driver: "json-file"
options:
max-file: "5"
max-size: "10m"

Go to the directory you saved the file.

cd /my/directory/

And spin up those containers!

docker-compose up -d

After the images are download, wait about 3 minutes and you can go to http://127.0.0.1:8080/ on your browser and see the initial page of Concourse.

concourse home screen

In this page you are able to download the fly CLI tool. Make sure to download the appropriate OS and to put into you $PATH.

Now we need to tell the CLI where is located the server, so run:

fly --target helloworld login --concourse-url http://127.0.0.1:8080 -u admin -p adminfly --target helloworld sync

You can check the fly configuration file:

cat ~/.flyrc

It would be something like this:

flyrc file after sync

There we have our credentials file and now we can use the command tool to directly target our server.

How can I create a pipeline?

Now that our Concourse is ready to go, save a file named first_pipeline.yml. In the content paste the following:

---
jobs:
- name: first-job
public: true
plan:
- task: my-first-pipeline
config:
platform: linux
image_resource:
type: docker-image
source: {repository: ubuntu}
run:
path: echo
args: [my first pipeline]

With this file we are:

  1. creating a public job named first-job
  2. inside this job we have a single task
  3. this tasks will spin up a container, based on the ubuntu image
  4. this container will output “my first pipeline”

To set this pipeline:

fly -t helloworld set-pipeline -p first -c first_pipeline.yaml

You will see an output like this:

You could browse to http://127.0.0.1:8080/teams/main/pipelines/first/jobs/first-job and see the output and more information on your build.

It will take a while, but as soon as the image is downloaded, you will see your job as a success.

If you made this far, congratulations, you have installed and created your pipeline in Concourse.

I really hope you enjoyed this tutorial as well this series. If you have any questions, please leave a comment or send me a message on my In.

See ya!

This is just a bit of my life. Writing another line everyday!