I have recently uploaded a post with some tricks for reducing the time you spend when building Docker images on Github Actions. That did indeed work pretty well for me until now, but it was a naive solution while waiting for Docker BuildX integration with Github cache. The wait is over and we do not need to manually cache files since Docker BuildX will do everything as we expected!
1. Get the basics
You can read my previous post to get the whole picture but I also recommend you to visit the official pull requests that lead to this new feature:
- This is the buildx code that has been merged for allowing the use of github internal cache
- This draft PR contains an example of how to use it.
If you read through all of those you probably realize that we have been waiting for new buildx 0.6 version and buildkit 0.9 to be generally available… but that happened just a few days ago!
New BuildKit v0.9.0 and Docker Buildx v0.6.0 releases are out with Github cache backend, OpenTelemetry support, Dockerfile Here-docs, better errors, variable support in mount flags etc. https://t.co/uo89yvSo5j https://t.co/L0QM7stmC5— Tõnis Tiigi (@tonistiigi) July 16, 2021
At this moment we are waiting for Github virtual environments to have new buildx 0.6.0 in Ubuntu base images, but they are generated on weekends and deployed during the week so we might have to wait a week or two before that happens. Anyway we can already test the new feature and add it to our pipelines!
Github virtual environments have been updated so we can use build-push action without extra configurations.
2. Simple example
I updated my CI pipeline to support the new feature. I can now remove all conditionals that I was using before to reduce building time when Dockerfile or conda.yaml were not modified. The simplified pipeline would look like this:
name: Continuous Integration new cache on: push: branches: [main] pull_request: branches: [main] jobs: build_docker: runs-on: ubuntu-latest steps: - uses: actions/checkout@v2 - name: Set up Docker Buildx uses: docker/setup-buildx-action@v1 - name: Login to GitHub Container Registry uses: docker/login-action@v1 with: registry: ghcr.io username: mmeendez8 password: $ - name: Build production image uses: docker/build-push-action@v2 with: context: . file: Dockerfile push: true tags: ghcr.io/mmeendez8/cache_docker/ci_dlc:latest cache-from: type=gha cache-to: type=gha,mode=max lint_and_test: needs: build_docker runs-on: ubuntu-latest container: image: ghcr.io/mmeendez8/cache_docker/ci_dlc:latest credentials: username: mmeendez8 password: $ steps: - uses: actions/checkout@v2 - name: Lint code run: | pre-commit install pre-commit run - name: Run tests run: pytest tests
cache-to type is set now to
gha (github action). The first time the action is triggered the cache is empty so Docker will need to build all layers from scratch:
But after this cache is full, so we can reuse all our layers in next builds, if the images was not modified, or just some of them when we apply changes to our Dockerfile. Let’s trigger a new build with an empty commit and check the time it needs now:
git commit --allow-empty -m "Test build" git push
That’s it! It only took 22 seconds to build our image.
Docker Buildx is a powerful enhancement and we should try to take full advantage of it
It is very simple to use Github Cache with build-push-action now