Continuous Deployment for Hugo websites

Updated May 2019 · 16 minute read
The following assume familiarity with Continuous Integration, Continuous Delivery and Continuous Deployment practices, as well as git, Hugo installed and an existing Hugo project.

Continuous Deployment, thought as an extension to Continuous Integration (CI), aims at minimizing the total time taken for code changes in development to deploy in the production environment. This is achieved with appropriate infrastructure capable of automating the steps of testing, building and deploying, utilizing pipelines with automated jobs in parallel or in sequence. In this practice there’s no human intervention at all, since the entire pipeline is put into production automatically.

Hugo documentation has an extensive section dedicated to Hosting & Deployment with options and guides to various solutions for hosting, continuous integration and automatic deployment. There you can find some excellent solutions for Continuous Deployment for your Hugo site. Some of the options described are free while others are not. My intention is to describe a totally free solution for Continuous Deployment with services available to everybody at the time of writing. You should expect that this solution, utilizing free services, would probably not suffice large scale projects, especially those with commercial intent, time constraints and particular security requirements.

Services and tools for the job

Our target here is to combine various free tools and platforms, to create a Continuous Deployment pipeline for a Hugo website. To make things work we will use:

  1. Git. A free and open source distributed version control system
  2. GitLab. GitLab provides an excellent framework (and UI) to integrate all these automation technologies required for Continuous Deployment and it is available all for free in your repository (private or public).
  3. Docker Hub. The world’s largest public repository of container images.
  4. Firebase. A mobile and web application development platform backed by Google, offering a free tier to host your static website.

These platforms and services where chosen, as they seamlessly integrate together providing a totally free Continuous Deployment solution for our scenario.

Work-flow

Starting to gather the various bits and pieces, we should first establish our desired work-flow as the final target we wish to achieve. The desired work-flow for this scenario would be:

  1. Working locally with our website files. (Try new features, add/update posts, etc.)
  2. When we feel happy with what we have done, commit the changes. (Or commit as many times as we feel necessary)
  3. Push changes to remote repository. (usually origin/master)
  4. Done. (Automatic deployment)

After step 3 in the above work-flow our site will be tested, built and deployed automatically, and as soon as our pipeline finishes successfully we will be able to see our latest changes on-line. The procedure involves the following:

  1. Initializing version control with Git in our website’s root directory. (local repository)
  2. Creating a relevant project in GitLab for our remote repository.
  3. Creating a free Firebase hosting account and API key.
  4. Creating a pipeline in GitLab using shared Runners, to test, build and deploy a Hugo site.

Pipeline creation using GitLab’s CI/CD technologies involves the use of container images, which we can get from the public repositories of Docker Hub, or shared/public container registries in GitLab. Another option is to create our own containers employing again GitLab’s CI/CD mechanisms to build and store container images utilizing shared runners and project’s registry storage, or to build it locally and upload it to our Docker-hub repository, or GitLab registry. For this guide we are going to follow the option of creating our own container images for deploying the Hugo site generator. This is chosen to satisfy particular requirements that could not be met otherwise at the time of writing and also to demonstrate further use of GitLab’s pipelines. This part of the procedure will be described as an intermediate step in between steps 2 and 3 of the above described procedure (Container image creation). You may choose to utilize images provided otherwise, in which case you should jump to step 3 after finishing step 2.

Step 1: Using git with our Hugo website

To proceed through this step you must have Git installed on your machine. If you are unfamiliar with version control using Git you can try a free on-line course to familiarize yourself with Git. To initialize a Git repository in your existing Hugo project, go to your project’s root directory (this is usually where your config.toml resides) create a file named .gitignore and add an exception for the directory public. You could try the following, in your project’s root directory, if you are on Ubuntu or similar Linux flavored:

touch .gitignore
echo "# Hugo default output directory
/public" >> .gitignore
git init
git add .
git commit -m "Initial commit"

This will create a new repository, add all your project files, except from public directory and an initial commit will be created. In case you are already using Git, you may ignore all this and proceed.

Step 2: Creating a GitLab project

If you don’t already have a GitLab account, you have to create one, to continue in this step. After signing-in, you can push to create a new GitLab project from your existing repository. If you you are using Git for the first time make sure you have set the global settings for user.name and user.email:

git config --global user.name "YOUR_USER_NAME"
git config --global user.email "YOUR_EMAIL"

Depending on whether you have set up SSH or not you can use SSH or HTTP to push your local repository to remote (use either method):

## Git push using SSH
git push --set-upstream git@gitlab.com:namespace/example-hugo.git master

## Git push using HTTP
git push --set-upstream https://gitlab.com/namespace/example-hugo.git master

In the above you have to replace namespace with you username (the part in https address that comes after gitlab.com when you sign in your account) and example-hugo.git is the name you want to give to your new GitLab project. The name does not have to be the same as the folder name you are using in your local repo. An alternative is to create a new GitLab project from your GitLab’s account home.

Create new project

Give the project a name:

Name of the new project

Choose project visibility:

Choose project visibility

Create the project:

Create Project

Then follow the Command line instructions for Existing Git repository to push your local files. After the initial push, every time you would like to push to remote origin, having committed your local changes you will just use git push.

Step Optional: Container image creation

If you add a .gitlab-ci.yml file to the root directory of your repository, your GitLab project will use a shared Runner, then each commit or push triggers your CI pipeline. You can see a simple example of a .gitlab-ci.yml here. To build our Hugo site with GitLab’s CI we need to utilize a couple of container images to create our own Hugo container image. We will then use this image for our Continuous Deployment pipeline. The image we will be creating will have Hugo extended version installed to support Sass/SCSS functionality. We are further looking to build this image in a light linux based container so that the final image size is small and flexible.

GitLab pages for Hugo is a very good starting point if you do not require the extended version of Hugo. They provide a public image with the latest version of Hugo installed which you can use in your own .gitlab-ci.yml to build your site if you do not need Sass/SCSS. The same project under branch registry provides a .gitlab-ci.yml, a Dockerfile and an update.py which are used to create and update the images contained in the project’s registry. A merge request created by David Arnold in the same branch proposed a way for creating an image container with Hugo’s extended version based on the lightweight Alpine Linux Docker image. We will use modified versions of the above three files in a new GitLab project (or in a new branch in our existing project if you like) to build and maintain our own Hugo image registry.

  1. Create locally a new directory eg. Extended_Hugo. In that directory create a .gitignore file as we did in step 1, earlier. Repeat the procedure as in step 1 to initialize a local git repo and commit your files.

  2. Create a new GitLab project, choosing private in visibility, then push local master:

     git remote add origin git@gitlab.com:namespace/name-of-your-project.git
 git push -u origin master

  3. Checkout a new local branch:

     git checkout -b registry

  4. Copy in this directory the content, layouts, static, themes, config.toml, from your Hugo site’s project directory, basically, creating a functional copy of your site. You do not need to have full contents, etc., as the purpose of this, is to create a functional Hugo website that can be built, for testing purposes, using the created container image. Then in the root directory of this project (you should be in the same directory as config.toml) create the following three files.

  5. Create a file named Dockerfile with contents:

    FROM alpine:latest
MAINTAINER YOUR_EMAIL

ENV HUGO_VERSION 0.55
ENV HUGO_SHA 19d1a071381d6695212e223287c442b2b5cb6baae3e5ae026384e187930f4581

ENV GLIBC_VERSION 2.28-r0

# Install packages
RUN set -eux && \
    apk add --update --no-cache \
      ca-certificates \
      openssl \
      git \
      libstdc++

# Install glibc to work with dynamically linked libraries used by extended hugo version
# See https://github.com/gohugoio/hugo/issues/4961
RUN wget -q -O /etc/apk/keys/sgerrand.rsa.pub https://alpine-pkgs.sgerrand.com/sgerrand.rsa.pub \
&&  wget "https://github.com/sgerrand/alpine-pkg-glibc/releases/download/$GLIBC_VERSION/glibc-$GLIBC_VERSION.apk" \
&&  apk --no-cache add "glibc-$GLIBC_VERSION.apk" \
&&  rm "glibc-$GLIBC_VERSION.apk" \
&&  wget "https://github.com/sgerrand/alpine-pkg-glibc/releases/download/$GLIBC_VERSION/glibc-bin-$GLIBC_VERSION.apk" \
&&  apk --no-cache add "glibc-bin-$GLIBC_VERSION.apk" \
&&  rm "glibc-bin-$GLIBC_VERSION.apk" \
&&  wget "https://github.com/sgerrand/alpine-pkg-glibc/releases/download/$GLIBC_VERSION/glibc-i18n-$GLIBC_VERSION.apk" \
&&  apk --no-cache add "glibc-i18n-$GLIBC_VERSION.apk" \
&&  rm "glibc-i18n-$GLIBC_VERSION.apk"

# Install HUGO
RUN wget -O ${HUGO_VERSION}.tar.gz https://github.com/spf13/hugo/releases/download/v${HUGO_VERSION}/hugo_extended_${HUGO_VERSION}_Linux-64bit.tar.gz && \
  echo "${HUGO_SHA}  ${HUGO_VERSION}.tar.gz" | sha256sum -c && \
  tar xf ${HUGO_VERSION}.tar.gz && mv hugo* /usr/bin/hugo && \
  rm -rf  ${HUGO_VERSION}.tar.gz && \
  rm -rf /var/cache/apk/* && \
  hugo version

EXPOSE 1313

CMD ["/usr/local/bin/hugo"]

    (The above file is a modified version of the original file found here and the merge request found here.)

  6. Create a file named update.py with contents:

    #!/usr/bin/env python3
# coding: utf-8

"""
Check Hugo new releases from Hugo GitHub repo and update image automaticaly

Usage: update.py API_token project_uri

Use project_uri ($CI_PROJECT_PATH) eg. gil/test-cicd-builddocker (NAMESPACE/PROJECT_NAME) for PUBLIC project
else
for PRIVATE project use project_id ($CI_PROJECT_ID)
"""

import requests
import re
import sys
from urllib.parse import quote

GITLAB_URL = "https://gitlab.com/api/v4"
COMMIT_MESSAGE = "Update Hugo to version {}"
GITHUB_API_REPOS = "https://api.github.com/repos"


def compare_versions(version1, version2):
    def normalize(v):
        return [int(x) for x in re.sub(r'(\.0+)*$', '', v).split(".")]
    return normalize(version1) >= normalize(version2)


if len(sys.argv) != 3:
    print('Usage: update.py API_token project_uri')
    exit(1)

# Get vars from script arguments
GITLAB_TOKEN = sys.argv[1]
GITLAB_PROJECT = quote(sys.argv[2], safe='')

# Get Hugo latest release
rrelease = requests.get(GITHUB_API_REPOS + '/gohugoio/hugo/releases/latest')
if rrelease.status_code != 200:
    print('Failed to get Hugo latest release from GitHub')
    exit(1)

release = rrelease.json()
print(f'Last Hugo version is {release["name"]}')

# Get alpine-pkg-glibc (Glibc) latest release
glibcrel = requests.get(GITHUB_API_REPOS + '/sgerrand/alpine-pkg-glibc/releases/latest')
if glibcrel.status_code != 200:
    print('Failed to get Glibc latest release from GitHub')
    exit(1)

glbrelease = glibcrel.json()
_glbrelease = glbrelease["name"] if (len(glbrelease["name"]) > 0) else glbrelease["tag_name"]
print(f'Last Glibc version is {_glbrelease}')

# Get repository tags. If repo is private must use CI_PROJECT_ID and GET request with API_TOKEN
# else, if public repo  API_TOKEN is not required.
rtags = requests.get(f'{GITLAB_URL}/projects/{GITLAB_PROJECT}/repository/tags', headers={'Private-Token': GITLAB_TOKEN})
if rtags.status_code != 200:
    print('Failed to get tags from GitLab project')
    exit(1)

# If a higher version is present in the GitLab repository, do nothing
for tag in rtags.json():
    if tag['release'] is None:
        continue
    if compare_versions(tag['release']['tag_name'], release['name'][1:]):
        print('Already up to date, nothing to do')
        exit(0)
print('No tag is higher or equal to Hugo version.\nUpdating...')

# Find release archive checksum from GitHub
for asset in release['assets']:
    if re.search('checksums.txt', asset['name']):
        rchecksums = requests.get(asset['browser_download_url'])
        if rchecksums.status_code != 200:
            print('Failed to get checksums file from GitHub')
            exit(1)
        for line in rchecksums.text.split("\n"):
            if f'hugo_extended_{release["name"][1:]}_Linux-64bit.tar.gz' in line:
                checksum = line[:64]
                break

# Get Dockerfile from repository
rdockerfile = requests.get(f'{GITLAB_URL}/projects/{GITLAB_PROJECT}/repository/files/Dockerfile/raw?ref=registry',
                           headers={'Private-Token': GITLAB_TOKEN})
if rdockerfile.status_code != 200:
    print(f'Failed to get Dockerfile from {sys.argv[1]}:')
    print(rdockerfile.text)
    exit(1)
dockerfile = rdockerfile.text.split("\n")

# Replace env variables
for index, line in enumerate(dockerfile):
    if "ENV HUGO_VERSION" in line:
        dockerfile[index] = f'ENV HUGO_VERSION {release["name"][1:]}'
    if "ENV HUGO_SHA" in line:
        dockerfile[index] = f'ENV HUGO_SHA {checksum}'
    if "ENV GLIBC_VERSION" in line:
        dockerfile[index] = f'ENV GLIBC_VERSION {glbrelease["name"]}'

# Update Dockerfile on repository
requestData = {
        'branch': 'registry',
        'content': "\n".join(dockerfile),
        'commit_message': COMMIT_MESSAGE.format(release['name'][1:]),
}
rupdate = requests.put(f'{GITLAB_URL}/projects/{GITLAB_PROJECT}/repository/files/Dockerfile',
                       data=requestData, headers={'Private-Token': GITLAB_TOKEN})
if rupdate.status_code != 200:
    print('Failed to update Dockerfile:')
    print(rupdate.text)
    exit(1)
print(f'Dockerfile was updated to version {release["name"][1:]}')

# Create new tag
requestData = {
        'tag_name': release['name'][1:],
        'ref': 'registry',
        'message': COMMIT_MESSAGE.format(release['name'][1:]),
        'release_description': release['body'],
}
rtag = requests.post(f'{GITLAB_URL}/projects/{GITLAB_PROJECT}/repository/tags',
                     data=requestData, headers={'Private-Token': GITLAB_TOKEN})
if rtag.status_code != 201:
    print('Failed to create tag:')
    print(rtag.text)
    exit(0)
print(f'Tag {release["name"][1:]} created')
print('Done !')

    (The above file is a modified version of the original file found here and the merge request found here.) Note: The file update.py must be executable to run it. After checking file permissions, if it is not executable you should make it:

    chmod +x update.py

    You should do this before committing changes, to “tell” git that the file should be executable. Git will then set the appropriate permissions when the file(s) are checked out.

  7. Create a file named .gitlab-ci.yml with contents:

    image: docker:latest

# When using dind, it's wise to use the overlayfs driver for
# improved performance.
variables:
  DOCKER_DRIVER: overlay2
  IMAGE_NAME: $CI_REGISTRY_IMAGE:$CI_COMMIT_REF_NAME
  IMAGE_LATEST: $CI_REGISTRY_IMAGE:latest

services:
  - docker:dind

stages:
  - test
  - deploy

before_script:
  - docker info
  - docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $CI_REGISTRY

test:
  stage: test
  script:
    - docker build -t $IMAGE_NAME .
    - docker run -v `pwd`:/git $IMAGE_NAME /bin/sh -c "wget -qO- https://fedoraproject.org/static/hotspot.txt | grep OK && cd /git && hugo"
  except:
    - schedules

deploy:
  stage: deploy
  script:
    - docker build -t $IMAGE_NAME -t $IMAGE_LATEST .
    - docker push $IMAGE_NAME
    - docker push $IMAGE_LATEST
  only:
    - tags

update:
  image: python:3.6
  before_script: []
  services: []
  script:
    - pip install requests
    - ./update.py $API_TOKEN $CI_PROJECT_ID
  only:
    - schedules

    (The above file is a modified version of the original file found here.)

  8. Commit new branch changes

     git add .
 git commit -m "Initial registry commit"

  9. In your GitLab account go to User Settings by clicking on Settings under your Profile (top right corner).

  10. You need to create an access token that we will use for the creation of our image. Click on Access Tokens, on the left navigation column. On the right add a name in the relevant field, choose api in Scopes and click Create personal access token. Copy and keep the key value available for later use.

  11. Back in your GitLab project with the registry branch, click on CI/CD on the left navigation column and then on Schedules. Click on New schedule and add the following, putting your generated access token in the value filed next to API_TOKEN.

    Schedule pipeline settings

  12. Push new branch to remote origin

    git push -u origin registry

  13. According to scheduled pipeline, every day update.py will check for new Hugo releases and create a new container image in your registry if a new Hugo release is found.

  14. If you want to trigger an immediate build you have to create a new tag:

    git tag -a v0.03 -m "my test tag"
git push --tags origin registry

You should now have your own container image with Hugo extended, to proceed in building your Continuous Deployment pipeline for your Hugo website. You could modify the above procedure to be added as a branch in your initial website project in GitLab. You could also alter the visibility to public and adjust the code in relevant places to have a publicly accessible registry, same as Hugo’s GitLab pages (compare with the relevant scripts there, to find which parts must be altered).

Step 3: Creating Firebase hosting

We need a place where our deployed site will be hosted and served to www. Firebase is offering a free tier for hosting static websites (which are mainly intended to be app-supporting sites) also providing secure http (https) via issuing a free ssl certificate. To get started with Firebase hosting you can read the relevant guide at Hugo documentation and Firebase’s getting started guides. In a few brief steps:

  1. Create an account with Firebase.

  2. Globally install Firebase tools. Although not required for the creation of our CI/CD pipeline, Firebase’s console tools provide an alternative way to control and monitor your website’s deployment and are required to generate an authentication token for use in non-interactive environments which is required for our CI/CD pipeline. To install the tools you must have Node.js installed, which you can get from here or following the directions here. Then, to install the actual Firebase tools you can follow the instructions given here.

  3. Go to your Firebase console and create a new web project.

  4. This is also a good time to connect your custom domain from the Firebase UI > Hosting > Connect Domain.

  5. Locally, go to the root directory of your Hugo website project (usually where config.toml resides). Then initialize your Firebase project:

     firebase login
 firebase init

    Follow the directions to make your choices, ie. choose Hosting, the name of the project you set up in Firebase UI, defaults for database rules, public as default publish directory and No for single page app deployment.

  6. Generate an authentication token for use in our CI/CD which is an non-interactive environment, using Firebase console tools:

     firebase login:ci

    This generates a token, e.g. “1/AD7sdasdasdKJA824OvEFc1c89Xz2ilBlaBlaBla”. Take a note of the generated token, but keep it safe, as this is a private key that allows write access to your Firebase hosting.

Step 4: Pipeline creation in GitLab

In this final step of the process we will create the .gitlab-ci.yml file for our website. The presence of this file in our GitLab repo triggers the CI process to our configured Runners or to default shared Runners at GitLab’s systems. We will use shared Runners that require no further set up.

  1. At the root directory of your Hugo website project (you should be at branch master) create a new file .gitlab-ci.yml with contents:

    stages:
  - build
  - deploy

build:
  stage: build
  # All available Hugo versions are listed here: https://gitlab.com/namespace/name-of-yr-project/container_registry (private)
  image: registry.gitlab.com/namespace/name-of-yr-project:latest
  artifacts:
    paths:
      - public/
    expire_in: 1 day
  variables:
    BASE_URL: "Your-site-url-from-Firebase"
    HUGO_ENV: "production"
  script:
    - hugo
  only:
    - master

deploy:
  stage: deploy
  image: node:latest
  dependencies:
    - build
  script:
    - npm install -g firebase-tools
    - firebase deploy --token "$FIREBASE_TOKEN" --project Name-of-yr-Firebase-project --non-interactive --only hosting --message "Pipeline $CI_PIPELINE_ID, build $CI_JOB_ID, sha1 $CI_COMMIT_SHA, ref $CI_COMMIT_REF_NAME"
  only:
    - master

    In the above you should replace the following with your data:

    • image: registry.gitlab.com/namespace/name-of-yr-project:latest, replace namespace and name-of-yr-project with the relevant info from your project at step Optional.
    • BASE_URL: "Your-site-url-from-Firebase", replace Your-site-url-from-Firebase with the url you have set up with Firebase hosting.
    • firebase deploy --token "$FIREBASE_TOKEN" --project Name-of-yr-Firebase-project..., replace Name-of-yr-Firebase-project with the name of the project you cerated in your Firebase console, at step 3.

  2. Head over to your GitLab project, you have created for your website at step 2, and at the left navigation column click on Settings and then on CI/CD. Then on section Variables click on Expand. Add the Firebase token value you generated earlier, using as variable name FIREBASE_TOKEN:

    CI/CD FIREBASE_TOKEN variable

    Click on Save Variables when your are done.

  3. We are ready now to push local changes to GitLab and trigger our CI process. Locally, at your console, you should be at the root directory of your Hugo website project:

    git add .
git commit -m "Adding .gitlab-ci.yml to trigger CI-CD"
# Optionally add your custom version tag
git tag -a v1.0 -m "1st Automated Deployment"
git push

    When uploading finishes, GitLab will trigger the CI process.

  4. You can now watch your pipeline process progress through your GitLab account. Click on CI/CD on the left navigation column and then on Pipelines. In times of high traffic GitLab’s shared Runners may take some time before they start on your pipeline(s), the process might seem stuck, but it is on hold and will resume as soon as resources are available. Hopefully, you will eventually see the green icon with the check and text “passed”.

    Passed indicates your pipeline has completed successfully

This completes our intended work-flow. You should now have a fully automated deployement for your Hugo website, allowing you to focus mainly on your content. Each time you add or change files in your local Hugo project, commit the changes when you feel ready and push to remote. This will trigger your CI pipeline and deploy a new version of your site to Firebase.

Enjoy coding!

Above opinions and any mistakes are my own. I am not affiliated in any way with companies, or organizations mentioned above. The code samples provided are licensed under the Apache 2.0 License and rest content of this page is licensed under the Creative Commons Attribution 3.0 License, except if noted otherwise.
Tags: Hugo, CI/CD, Firebase, GitLab, Docker