release.md

Creating releases of the materialized

Checklist

In addition to the mechanics of our branching strategy, described here, we have a release checklist which should be followed (and possibly updated) at every release.

To view the release checklist, create a new issue using the release template:

https://github.com/MaterializeInc/materialize/issues/new?template=release.md&title=release:+v0.#.#

Basic git workflow

The Materialize release process is based on an "always release main" philosophy, while still allowing us to produce bugfix releases if they are urgently required. We will iterate on this release process as we gain experience.

There are two core git artifacts involved in every release:

• Every release is denoted by a git tag that specifically names that release.
• Before the release, there is a "release candidate" git tag that signifies that we are planning on making that commit the release.

The goal is to keep it simple while still having high assurance that we are providing excellent quality software to our users. In all the examples, the goal is 100% automation of everything that does not explicitly mention a human. Developers are free to add other steps as they find convenient.

To start a new release version the steps look like:

1. We start with an existing repository, with some work having already been done on it main is currently on commit C and we'd like to get B tagged as a release:

   

2. A developer uses the mkrelease script to update license and build files:

    bin/mkrelease -b release-v0.1.0 --checkout B v0.1.0-rc1
    git push origin v0.1.0-rc1
   

   Resulting in a local repo that looks like:

   

   but note that pushing the release-0.1.0 branch is not required or recommended.

3. The developer then updates for the next phase of development, off that same commit, so so that dev artifacts look correct:

    bin/mkrelease -b prepare-v0.1.1 --no-tag v0.1.1-dev
    git push me prepare-v0.1.1
    gh pr create
   

   Resulting in a repo that looks like:

   

4. More detailed testing takes place on the artifacts generated by this tag. For all tests materialized is not allowed to crash, and all quality of service measurements must never fall outside of our accepted bounds.

   The set of tests is:

   • Install apt/rpm packages in clean ubuntu/centos image, and the homebrew package is installed on a clean macos image
   • Full SLT (against auto-generated materialize/materialized image)
   • The chbench soak test is run for 72 hours (same docker image)
   • The protobuf soak test is run for 72 hours (same docker image)

5. A human verifies that everything looks good, tags and pushes the v<VERSION> tag -- since there have been no problems, the release branch is up to date:

    git tag -a v0.1.0 v0.1.0-rc1
    git push origin v0.1.0
   

   At this point the repo should look like:

   

6. Follow the release checklist!

7. make announcements and 🎉💃🕺

Fully elaborated git workflow

So far the release process only describes what happens when things go smoothly, and releases are bug-free. But there are also several things that can happen that can cause us to revisit old releases and pre-releases:

1. Testing can fail for reasons unrelated to the materialized codebase.
2. Testing can fail for reasons related to the materialized codebase.
3. Bugs can be found post-release that are considered high-enough severity that we want to backport a bugfix.

Depending on the current level of automation, the solution for 1 is either to manually re-run the tests, or to push a new tag. In the world where we have this fully automated creating a new tag for v0.1-rc1 with an N + 1 suffix to kick off new tests may look like:

git tag -a v0.1.0-rc2 v0.1.0-rc1
git push origin v0.1.0-rc2

Alternatively, clicking through some CI interfaces to trigger long-running jobs will be necessary.

No matter our level of automation, 2 & 3 have a very similar solution: push changes to the release-X.Y branch, and eventually tag specific commits in a similar way to the happy path.

Here is a walkthrough of a bad-case happening at every step in the release process:

1. Ensure that the repo is entirely up to date: git pull origin

2. Commit B in the main branch is tagged and the release branch is updated:

    git tag -a v0.1.0-rc1 <B>
    git push origin v0.1.0-rc1
   

3. A pair of issues are discovered that are deemed important enough for us to backport to the rc

4. The fixes are tested and merged to main as commits D and E, after some unfortunately dangerous work has been commited to main (here as C). The repo therefore looks like this, and we do not want to import C into the release:

   

5. The commits on main are applied to the release branch:

    git checkout release-X.Y
    git cherry-pick D
    git cherry-pick E
   

6. That is tagged git tag -a v0.1.0-rc1 HEAD:

   

7. Both the branch and the tag are pushed:

    git push origin release
    git push origin v0.1.0-rc2
   

8. Long-running tests succeed again, the release is published to github and all distribution channels:

    git tag -a v0.1.0 v0.1.0-rc
   

   At this point the repo and successful releases looks like:

9. Some time later we're ready to release from mainline again, this should be the next release. So even though we don't have any features that we would like to market as 0.2 we would like to release bugfixes and improvements from mainline. That causes the next tag to be on a commit that is an ancestor of main again, allowing the release manager to delete the release-0.1.0 branch that we were using as scratch space, and start the process over on main:

    git tag -a v0.1.1-rc1 H
    # run long-running tests
    git tag -a v0.1.1
    git push origin v0.1.1
   

   causing the repo to look like:

   

10. Now we have released again, and we should follow the release checklist