Skip to content

Commit a923ce7

Browse files
woody-applerestyled-commits
woody-apple
and
restyled-commits
authored
Move to CircleCI for Builds and Tests (project-chip#631)
* Initial CircleCI * Restyled by shellharden * Restyled by shfmt * Appeasing our restyle.io overlord * testing * Trying to figure out this run_if.sh problem * Feedback from Rob W * Restyled by shfmt * Typo * Updating restyled to match editorconfig tab spacing * Updating devcontainer * Restyled by shfmt * Adding simple restyle merge * Restyled by shellharden Co-authored-by: Restyled.io <commits@restyled.io>
1 parent 423c8db commit a923ce7

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

45 files changed

+1388
-326
lines changed

‎.circleci/.gitattributes

+1
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
config.yml linguist-generated

‎.circleci/.gitignore

+1
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
.tmp/

‎.circleci/Makefile

+95
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,95 @@
1+
# Set SHELL to 'strict mode' without using .SHELLFLAGS for max compatibility.
2+
# See https://fieldnotes.tech/how-to-shell-for-compatible-makefiles/
3+
SHELL := /usr/bin/env bash -euo pipefail -c
4+
5+
# CONFIG is the name of the make target someone
6+
# would invoke to update the main config file (config.yml).
7+
CONFIG ?= ci-config
8+
# VERIFY is the name of the make target someone
9+
# would invoke to verify the config file.
10+
VERIFY ?= ci-verify
11+
12+
CIRCLECI := circleci --skip-update-check
13+
ifeq ($(DEBUG_CIRCLECI_CLI),YES)
14+
CIRCLECI += --debug
15+
endif
16+
17+
# For config processing, always refer to circleci.com not self-hosted circleci,
18+
# because self-hosted does not currently support the necessary API.
19+
CIRCLECI_CLI_HOST := https://circleci.com
20+
export CIRCLECI_CLI_HOST
21+
22+
# Set up some documentation/help message variables.
23+
# We do not attempt to install the CircleCI CLI from this Makefile.
24+
CCI_INSTALL_LINK := https://circleci.com/docs/2.0/local-cli/\#installation
25+
CCI_INSTALL_MSG := Please install CircleCI CLI. See $(CCI_INSTALL_LINK)
26+
CCI_VERSION := $(shell $(CIRCLECI) version 2> /dev/null)
27+
ifeq ($(CCI_VERSION),)
28+
# Attempting to use the CLI fails with installation instructions.
29+
CIRCLECI := echo '$(CCI_INSTALL_MSG)'; exit 1; \#
30+
endif
31+
32+
SOURCE_DIR := config
33+
SOURCE_YML := $(shell [ ! -d $(SOURCE_DIR) ] || find $(SOURCE_DIR) -name '*.yml')
34+
CONFIG_SOURCE := Makefile $(SOURCE_YML) | $(SOURCE_DIR)
35+
OUT := config.yml
36+
TMP := .tmp/config-processed
37+
CONFIG_PACKED := .tmp/config-packed
38+
39+
default: help
40+
41+
help:
42+
@echo "Usage:"
43+
@echo " make $(CONFIG): recompile config.yml from $(SOURCE_DIR)/"
44+
@echo " make $(VERIFY): verify that config.yml is a true mapping from $(SOURCE_DIR)/"
45+
@echo
46+
@echo "Diagnostics:"
47+
@[ -z "$(CCI_VERSION)" ] || echo " circleci-cli version $(CCI_VERSION)"
48+
@[ -n "$(CCI_VERSION)" ] || echo " $(CCI_INSTALL_MSG)"
49+
50+
$(SOURCE_DIR):
51+
@echo No source directory $(SOURCE_DIR) found.; exit 1
52+
53+
# Make sure our .tmp dir exists.
54+
$(shell [ -d .tmp ] || mkdir .tmp)
55+
56+
.PHONY: $(CONFIG)
57+
$(CONFIG): $(OUT)
58+
59+
.PHONY: $(VERIFY)
60+
$(VERIFY): config-up-to-date
61+
@$(CIRCLECI) config validate $(OUT)
62+
63+
define GENERATED_FILE_HEADER
64+
### ***
65+
### WARNING: DO NOT manually EDIT or MERGE this file, it is generated by 'make $(CONFIG)'.
66+
### INSTEAD: Edit or merge the source in $(SOURCE_DIR)/ then run 'make $(CONFIG)'.
67+
### ***
68+
endef
69+
export GENERATED_FILE_HEADER
70+
71+
# GEN_CONFIG writes the config to a temporary file. If the whole process succeeds,
72+
# it them moves that file to $@. This makes it an atomic operation, so if it fails
73+
# make doesn't consider a half-baked file up to date.
74+
define GEN_CONFIG
75+
@$(CIRCLECI) config pack $(SOURCE_DIR) > $(CONFIG_PACKED)
76+
@echo "$$GENERATED_FILE_HEADER" > $@.tmp || { rm -f $@; exit 1; }
77+
@$(CIRCLECI) config process $(CONFIG_PACKED) >> $@.tmp || { rm -f $@.tmp; exit 1; }
78+
@mv -f $@.tmp $@
79+
endef
80+
81+
$(OUT): $(CONFIG_SOURCE)
82+
$(GEN_CONFIG)
83+
@echo "$@ updated"
84+
85+
$(TMP): $(CONFIG_SOURCE)
86+
$(GEN_CONFIG)
87+
88+
.PHONY: config-up-to-date
89+
config-up-to-date: $(TMP) # Note this must not depend on $(OUT)!
90+
@if diff -w $(OUT) $<; then \
91+
echo "Generated $(OUT) is up to date!"; \
92+
else \
93+
echo "Generated $(OUT) is out of date, run make $(CONFIG) to update."; \
94+
exit 1; \
95+
fi

‎.circleci/README.md

+130
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,130 @@
1+
# How to use CircleCI multi-file config
2+
3+
This README and the Makefile should be in your `.circleci` directory,
4+
in the root of your repository.
5+
All path references in this README assume we are in this `.circleci` directory.
6+
7+
The `Makefile` in this directory generates `./config.yml` in CircleCI 2.0 syntax,
8+
from the tree rooted at `./config/`, which contains files in CircleCI 2.0 or 2.1 syntax.
9+
10+
11+
## Quickstart
12+
13+
The basic workflow is:
14+
15+
- Edit source files in `./config/`
16+
- When you are done, run `make ci-config` to update `./config.yml`
17+
- Commit this entire `.circleci` directory, including that generated file together.
18+
- Run `make ci-verify` to ensure the current `./config.yml` is up to date with the source.
19+
20+
When merging this `.circleci` directory:
21+
22+
- Do not merge the generated `./config.yml` file, instead:
23+
- Merge the source files under `./config/`, and then
24+
- Run `make ci-config` to re-generate the merged `./config.yml`
25+
26+
And that's it, for more detail, read on!
27+
28+
29+
## How does it work, roughly?
30+
31+
CircleCI supports [generating a single config file from many],
32+
using the `$ circleci config pack` command.
33+
It also supports [expanding 2.1 syntax to 2.0 syntax]
34+
using the `$ circleci config process` command.
35+
We use these two commands, stitched together using the `Makefile`
36+
to implement the workflow.
37+
38+
[generating a single config file from many]: https://circleci.com/docs/2.0/local-cli/#packing-a-config
39+
[expanding 2.1 syntax to 2.0 syntax]: https://circleci.com/docs/2.0/local-cli/#processing-a-config
40+
41+
42+
## Prerequisites
43+
44+
You will need the [CircleCI CLI tool] installed and working,
45+
at least version `0.1.5607`.
46+
You can [download this tool directly from GitHub Releases].
47+
48+
```
49+
$ circleci version
50+
0.1.5607+f705856
51+
```
52+
53+
[CircleCI CLI tool]: https://circleci.com/docs/2.0/local-cli/
54+
[download this tool directly from GitHub Releases]: https://github.com/CircleCI-Public/circleci-cli/releases
55+
56+
57+
## Updating the config source
58+
59+
Before making changes, be sure to understand the layout
60+
of the `./config/` file tree, as well as circleci 2.1 syntax.
61+
See the [Syntax and layout] section below.
62+
63+
To update the config, you should edit, add or remove files
64+
in the `./config/` directory,
65+
and then run `make ci-config`.
66+
If that's successful,
67+
you should then commit every `*.yml` file in the tree rooted in this directory.
68+
That is: you should commit both the source under `./config/`
69+
and the generated file `./config.yml` at the same time, in the same commit.
70+
The included git pre-commit hook will help with this.
71+
Do not edit the `./config.yml` file directly, as you will lose your changes
72+
next time `make ci-config` is run.
73+
74+
[Syntax and layout]: #syntax-and-layout
75+
76+
77+
### Verifying `./config.yml`
78+
79+
To check whether or not the current `./config.yml` is up to date with the source
80+
and valid, run `$ make ci-verify`.
81+
Note that `$ make ci-verify` should be run in CI,
82+
in case not everyone has the git pre-commit hook set up correctly.
83+
84+
85+
#### Example shell session
86+
87+
```sh
88+
$ make ci-config
89+
config.yml updated
90+
$ git add -A . # The -A makes sure to include deletions/renames etc.
91+
$ git commit -m "ci: blah blah blah"
92+
Changes detected in .circleci/, running 'make -C .circleci ci-verify'
93+
--> Generated config.yml is up to date!
94+
--> Config file at config.yml is valid.
95+
```
96+
97+
98+
### Syntax and layout
99+
100+
It is important to understand the layout of the config directory.
101+
Read the documentation on [packing a config] for a full understanding
102+
of how multiple YAML files are merged by the circleci CLI tool.
103+
104+
[packing a config]: https://circleci.com/docs/2.0/local-cli/#packing-a-config
105+
106+
Here is an example file tree (with comments added afterwards):
107+
108+
```sh
109+
$ tree .
110+
.
111+
├── Makefile
112+
├── README.md # This file.
113+
├── config # The source code for config.yml is rooted here.
114+
│   ├── @config.yml # Files beginning with @ are treated specially by `circleci config pack`
115+
│   ├── commands # Subdirectories of config become top-level keys.
116+
│   │   └── go_test.yml # Filenames (minus .yml) become top-level keys under
117+
│   │   └── go_build.yml # their parent (in this case "commands").
118+
│ │ # The contents of go_test.yml therefore are placed at: .commands.go_test:
119+
│   └── jobs # jobs also becomes a top-level key under config...
120+
│   ├── build.yml # ...and likewise filenames become keys under their parent.
121+
│   └── test.yml
122+
└── config.yml # The generated file in 2.0 syntax.
123+
```
124+
125+
About those `@` files... Preceding a filename with `@`
126+
indicates to `$ circleci config pack` that the contents of this YAML file
127+
should be at the top-level, rather than underneath a key named after their filename.
128+
This naming convention is unfortunate as it breaks autocompletion in bash,
129+
but there we go.
130+

0 commit comments

Comments
 (0)