Setting up Docker client within a Docker image

Sometimes, life is not easy. And DevOps can be a pain. Strangely, it was Docker that was causing it.

Recently, I set up Jenkins for continuous integration on DigitalOcean. There are so many ways to do the same thing. And if something does not work, you try to tweak some settings to get it to work. Thankfully, there is a standard way to do things. And I found out how to do it with the help of Edward Viaene in an Udemy course. So, I set up Jenkins based on an Udemy course! It all works fine. I have an Express API. Whenever there is a push to the repo, Jenkins packages it as a Docker image and publishes it to Docker Hub. All set, right? No, not really. Who is going to deploy the Docker image to all my servers? I was hoping Jenkins will do it for me. Unfortunately, it won’t.

Deploy Docker image to DigitalOcean servers

We know the manual way of creating new containers using command-line.

docker container rm -f api-server
docker container run 
--name api-server 
--detach 
--publish 3000:3000
--link mysql-server:mysql
vijayst/myapi:$1

The shell script removes a container by name of api-server. Then, it recreates a new one of the same name. While recreating the container, we use a different tag (image tag) as indicated by the $1 argument.

sh deploy.sh a12f312

Whenever an image is pushed to the public repository, I want to call the script. DockerHub allows setting a webhook (API). DockerHub calls the POST method of the API with the relevant details.

Setting a Webhook on DockerHub

Webhook for DockerHub

Webhook is any API. For our example, we create a simple Express API. The Express API receives the event from DockerHub. It retrieves the image tag. And executes the shell script.

const express = require('express');
const app = express();
const bodyParser = require('body-parser');
const { exec } = require('child_process');

app.use(bodyParser.json());

app.get('/', (req, res) => {
    res.send({ api: 'ok' });
})
app.post('/', (req, res) => {
    const tag = req.body.push_data.tag;
    console.log('deploying tag', tag);
    exec(`sh deploy.sh ${tag}`, (err, stdout, stderr) => {
        res.sendStatus(200);
    });
});

app.listen(3050, () => {
    console.log('server started');
});

The overall flow works like Magic.

  1. Make a commit in API code and push to the repo.
  2. Jenkins picks the commit, prepares an image and pushes it to DockerHub.
  3. DockerHub makes a POST call to the webhook (or deploy API).
  4. The webhook calls a shell script.
  5. The shell script shuts down the previous container and starts a new one (deploys the API).
CICD using Jenkins and Docker

Webhook is another API. And we deploy the webhook using another Docker image. Prepare another image for the deploy API or the webhook. Spawn a container on the server with the webhook image. Supply the webhook API endpoint to DockerHub. When we make a push to the API repo, we expect our webhook to spawn a new container with the API image. Unfortunately, it does not work at this point.

Set Docker client within any image

We were trying to run docker commands from our webhook. Our webhook is deployed as another container. The webhook container does not have access to Docker. So, none of the commands within the shell script runs.

To solve this problem, I have a git repo. The Dockerfile below installs Docker client on any image.

USER root

RUN mkdir -p /tmp/download && \
 curl -L https://get.docker.com/builds/Linux/x86_64/docker-1.13.1.tgz | tar -xz -C /tmp/download && \
 rm -rf /tmp/download/docker/dockerd && \
 mv /tmp/download/docker/docker* /usr/local/bin/ && \
 rm -rf /tmp/download && \
 groupadd -g 999 docker

We download Docker into the image. Then, remove the daemon. Add the user to the docker group. This gives sufficient permissions for the user to run docker commands.

Add the above lines of code into the Dockerfile for the webhook. Finally, when spawning a new container for the webhook, make sure to share docker.sock.

docker container run
--name deploy-api
--publish 3050:3050
--detach
--volume /var/run/docker.sock:/var/run/docker.sock
vijayst/dockerhook

And now, it works. Docker client is available in the webhook API container. It communicates with the daemon on the host machine. So, all the docker commands run well. And we achieve the end result: To spawn a container with the published API image.

Related Posts

Leave a Reply

Your email address will not be published.