Improving snap maintenance with automation

by Heather Ellsworth on 27 June 2023

Co-written with Sergio Costas Rodríguez.

As the number of snaps increases, the need for automation grows. Any automation to help us maintain a group of snaps is welcome and necessary for us to be able to scale. The solution detailed in this article has two main benefits:

Any users of snaps that have adopted this automation will be kept more up to date with the latest released version from the original upstream (the source repo packaged into the snap).
Any snap developers with repos in GitHub are welcome to use our provided workflow to use a public action that will keep your main branch up to date with the latest upstream tag.

This post details the steps taken to standardise snap building automation, and some ongoing tasks to achieve our automation vision: automate all the things!

What is the new automation?

All of our snap repos live under https://github.com/ubuntu/, so we can use GitHub infrastructure in our automation quest. Using GitHub actions, workflows, and runners, we have been able to automate updates to some extent, and we’re not stopping there!

As shown in the graphic below, the automation consists of four sequential steps:

Each snap GitHub repository has a workflow in place that runs daily. The workflow uses the desktop-snaps action to check for a new upstream tag available that is newer than what is in the snapcraft.yaml file in the stable branch.
The desktop-snaps action will provide either the updated snapcraft.yaml or a message saying “No updates available”. If there is a new tag available, for each part using source-tags, then the desktop-snaps action returns a new snapcraft.yaml with the new source-tag value. This change is then committed and pushed to the snap repository’s stable branch.
Note that for this article we stick with the “stable” branch, but it can be called anything.
Each mirror that exists on launchpad auto syncs every few hours and will pick up the new change. There is a snap packaging recipe defined for each mirror that will auto build the snap, for each architecture requested.
Every launchpad snap build that succeeds gets uploaded to the snap store, in the snap’s respective –candidate channel. Now the snap is ready for manual testing and promotion to the –stable channel.

desktop-snaps repo

The desktop-snaps repository contains the core bits needed for coordinating automation across many other snap repositories:

https://github.com/ubuntu/desktop-snaps

While there are several cool things in this repo, I’d like to bring your attention to the heart of our automation, updatesnap/updatesnapyaml.py, and the action that uses it, action.yml.

updatesnap/updatesnapyaml.py

This script is provided with a repo that contains a snapcraft.yaml and searches each part’s upstream repo for a newer tag than the one being built in the snapcraft.yaml. If you were to run this locally, the required parameters of updatesnapyaml.py are $GITHUB_USER, $GITHUB_TOKEN, and a URL for a snap repository. For example:

$ ./updatesnapyaml.py --github-user $GITHUB_USER --github-token $GITHUB_TOKEN https://github.com/ubuntu/gnome-calculator.git

The tag value in the original snapcraft.yaml is compared to the latest tag available in the (often GNOME) upstream repository. If there is a newer tag upstream, then a new snapcraft.yaml is generated with just this tag change to the newer tag.

action.yml

This is a GitHub composite action that uses updatesnapyaml.py to take tag changes further. The action commits the updated tag in the snapcraft.yaml change generated by updatesnapyaml.py. This one-line change is then pushed to the snap repo’s main branch.

Note that the action does not do anything by itself; there must be a workflow somewhere calling it to benefit from it.

Individual snap repos

Each individual snap has its own repository on GitHub. As you may know, GitHub workflows live in the .github/workflows/ directory. Then in the various snap repos, we have put in place an auto-update.yml workflow that utilizes our desktop-snaps action.

Each individual snap has its own repository on GitHub, and it is each of these repositories that has the workflows, that calls the action, to trigger automation awesomeness.

As you may know, GitHub workflows live in the .github/workflows/ directory. Then in the various snap repos, authors place an auto-update.yml workflow that uses the desktop-snaps action.yml discussed in the previous section.

Requirements for using action.yml

For a snapcraft.yaml to work with our automation tools, a few things need to be in place first:

part uses source-depth
part tracks an upstream source-tag
nil and dump parts are not considered for updates

We wanted to save on the amount of data needing to be transferred from the upstream repo when checking for newer tags, so the snapcraft source-depth line is required for each part that expects to use the automation.

Since the automation is specifically for upstream tags, each part is required to include the source-tag line. If a part is instead tracking a branch, it will not be updated.

As you might expect, neither nil nor dump parts are considered for updates.

Setting tag update limits

What powers do I have to limit tag update options? Many!

If you do not want just the latest tag provided by upstream, you can include metadata in your part to set boundaries. A metadata block should go somewhere in the part you’d like to control. Each line of the metadata starts with a #. Here is an example:

# ext:updatesnap
#   version-format:
#     lower-than: '45'
#     no-9x-revisions: true

Using comments in the part you would like to limit lets you pin the major version, ignore X.90 revisions (which are often development versions not intended for general use), and so much more!

Note that it is not required to use this comment block in a part if you want the latest upstream tag to be applied and committed.

gnome-calculator example

To showcase what is currently in place, let’s take a look at the gnome-calculator example:

https://github.com/ubuntu/gnome-calculator

auto-update.yml

The auto-update.yml used by gnome-calculator is a boilerplate workflow that can be copied to any project.

name: Push new tag update to stable branch

on:
  schedule:
    # Daily for now
    - cron: '9 7 * * *'
  workflow_dispatch:

permissions: write-all

jobs:
  update-snapcraft-yaml:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout this repo
        uses: actions/checkout@v3
      - name: Run desktop-snaps action  
        uses: ubuntu/desktop-snaps@stable
        with:
          token: ${{ secrets.GITHUB_TOKEN }}
          repo: ${{ github.repository }}

How it works

Workflow Metainfo

name: This is the string that shows up under the gnome-calculator Actions tab.
schedule: This is how often the workflow runs. In our case, running it daily made sense.
workflow_dispatch: This allows you to manually launch the action from the Actions tab.

Job Definition

runs-on: This defines what kind of GitHub runner we use. ubuntu-latest is the latest LTS of ubuntu, 22.04.
step ‘Checkout this repo’: In order to make and commit changes to the gnome-calculator/snapcraft.yaml, the runner needs to first checkout the gnome-calculator repo. This uses a common action available in the GitHub action marketplace, actions/checkout@v3.
step ‘Run desktop-snaps action’: In order to be able to use the magic in the desktop-snaps repo, we must tell the step what namespace/repo@branch to use.
- token: Since the desktop-snaps action will need access to the GITHUB_TOKEN generated for the gnome-calculator workflow run, we pass it to the action where it will be used as an input. This access is needed to allow the desktop-snaps action to commit to the gnome-calculator repository.
- repo: Tells the desktop-snaps action the target repo’s url.

snapcraft.yaml

It’s unnecessary to provide the full snapcraft.yaml here, but I’ll point out the relevant section:

parts:
  gnome-calculator:
    source: https://gitlab.gnome.org/GNOME/gnome-calculator.git
    source-tag: '44.0'
    source-depth: 1
# ext:updatesnap
#   version-format:
#     lower-than: '45'
#     no-9x-revisions: true

As you can see, the gnome-calculator part has the required source-tag and source-depth lines.

How can you use the new automation?

As previously mentioned, the workflow used in the gnome-calculator repository is a boilerplate one that anyone can copy and paste into their own .github/workflows/ directory. Once it’s available in the main branch, you can manually launch the action or wait for it to happen on the cron schedule. Easy peasy.

Next Steps

We are currently working on implementing test plans using checkbox. While testing apps is currently a manual process in checkbox, we have aspirations to automate this promotion testing with the use of visual aid inspection tools.

https://github.com/ubuntu/desktop-snaps/tree/stable/checkbox-test-plans

If you are a new contributor and would like to help test snaps, or provide more test plans, please go checkout our docs page to learn how you can contribute.

Conclusion

The efforts that have gone into snap automation will improve the scalability and maintainability of any snaps that benefit from the provided automation. This will have the perk of also keeping the user on the latest upstream tag release with minimal manual effort on our side.

Anyone with a GitHub snap repository is welcome and encouraged to use the work Canonical has invested in this automation project. We would love to hear your thoughts! Here are a couple of links to help direct your feedback:

desktop-snaps repo: This contains updatesnapyaml.py, action.yml, snap test plans, and other wonderful things. Please file issues in this repository. We welcome pull requests and manual testing (using checkbox) too!
forum.snapcraft.io: This is a great place to reach out for questions and support.