Skip to content

Commit

Permalink
Shared tc dir (#5)
Browse files Browse the repository at this point in the history
* update .taskcluster.yml for shared

* nuke taskcluster/

* move GITHUB_CLONE_SECRET to .taskcluster.yml

* add new template parameters

* fix

* to_unicode

* try backing out the new params

* try text_type(...)

* project-regex

* ugh

* get env from decision task

* point at real repo master (commented atm)

* fix githubCloneSecret

* use project-regex for xpi

* add checkout: to build and test

* default-repository for xpi-template

* xpi {}

* get rid of special template parameter handling

* fix

* use --root instead of rm/ln taskcluster

* back out to ca54111

* re-add parameters.py

* revert back to 0e5a9d7

* add docker-image kind

* add docker dir

* use johan's taskgraph-try 12d2c3fb3c584209b9caef89691e81dbffb8129e

* 0dbd05252def2df5c6b5e3322d196499c61ada5a

* back out to 9a2e2ec, fix one thing at a time

* bump to taskgraph c78d65237c6ed064b9cbb3cd063b22e9e052a06a

* back to b34543ca2f42771117fa9da33c31994913587636

* c78d65237c6ed064b9cbb3cd063b22e9e052a06a again
  • Loading branch information
escapewindow authored Apr 30, 2020
1 parent 8917596 commit dc7a87b
Show file tree
Hide file tree
Showing 7 changed files with 53 additions and 16 deletions.
24 changes: 22 additions & 2 deletions .taskcluster.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,17 @@ tasks:
- $let:
taskgraph:
branch: taskgraph
revision: 2b32db7d303dd175ed4c4abf5fcfe00c6c593acd
revision: c78d65237c6ed064b9cbb3cd063b22e9e052a06a
template:
# XXX uncomment to merge
# repo: https://github.com/mozilla-releng/xpi-template
# branch: master
repo: https://github.com/escapewindow/xpi-template
branch: shared-tc-dir
trustDomain: xpi
# XXX for private repos, uncomment this line, and comment out the next
# githubCloneSecret: project/xpi/xpi-github-clone-ssh
githubCloneSecret: ""
in:
$if: 'tasks_for in ["github-pull-request", "github-push", "action", "cron"]'
then:
Expand Down Expand Up @@ -173,12 +182,18 @@ tasks:
XPI_HEAD_REF: '${head_branch}'
XPI_HEAD_REV: '${head_sha}'
XPI_REPOSITORY_TYPE: git
TEMPLATE_BASE_REPOSITORY: '${template.repo}'
TEMPLATE_HEAD_REPOSITORY: '${template.repo}'
TEMPLATE_HEAD_REV: '${template.branch}'
TEMPLATE_HEAD_REF: '${template.branch}'
TEMPLATE_REPOSITORY_TYPE: git
TASKGRAPH_BASE_REPOSITORY: https://hg.mozilla.org/ci/taskgraph
TASKGRAPH_HEAD_REPOSITORY: https://hg.mozilla.org/ci/${taskgraph.branch}
TASKGRAPH_HEAD_REV: '${taskgraph.revision}'
TASKGRAPH_REPOSITORY_TYPE: hg
REPOSITORIES: {$json: {xpi: "XPI Manifest", taskgraph: "Taskgraph"}}
REPOSITORIES: {$json: {xpi: "XPI Manifest", taskgraph: "Taskgraph", template: "XPI Template"}}
HG_STORE_PATH: /builds/worker/checkouts/hg-store
GITHUB_CLONE_SECRET: '${githubCloneSecret}'
- $if: 'tasks_for in ["github-pull-request"]'
then:
XPI_PULL_REQUEST_NUMBER: '${event.pull_request.number}'
Expand All @@ -201,6 +216,7 @@ tasks:
command:
- /usr/local/bin/run-task
- '--xpi-checkout=/builds/worker/checkouts/src'
- '--template-checkout=/builds/worker/checkouts/template'
- '--taskgraph-checkout=/builds/worker/checkouts/taskgraph'
- '--task-cwd=/builds/worker/checkouts/src'
- '--'
Expand All @@ -213,10 +229,14 @@ tasks:
then: >
PIP_IGNORE_INSTALLED=0 pip install --user /builds/worker/checkouts/taskgraph &&
cd /builds/worker/checkouts/src &&
rm -rf taskcluster &&
ln -s /builds/worker/checkouts/template/taskcluster taskcluster &&
ln -s /builds/worker/artifacts artifacts &&
~/.local/bin/taskgraph action-callback
else: >
PIP_IGNORE_INSTALLED=0 pip install --user /builds/worker/checkouts/taskgraph &&
rm -rf taskcluster &&
ln -s /builds/worker/checkouts/template/taskcluster taskcluster &&
ln -s /builds/worker/artifacts artifacts &&
~/.local/bin/taskgraph decision
--pushlog-id='0'
Expand Down
2 changes: 2 additions & 0 deletions taskcluster/ci/build/kind.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,8 @@ job-defaults:
docker-image:
indexed: xpi.cache.level-3.docker-images.v2.node-latest.latest
run:
checkout:
xpi: {}
use-caches: false

jobs:
Expand Down
13 changes: 10 additions & 3 deletions taskcluster/ci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,16 @@ taskgraph:
repositories:
xpi:
name: "XPI source"
project-regex: '(?!xpi-template)'
template:
name: "XPI template"
project-regex: 'xpi-template'
# XXX
#default-repository: https://github.com/mozilla-extensions/xpi-template
#default-ref: master
default-repository: https://github.com/escapewindow/xpi-template
default-ref: shared-tc-dir
type: git

workers:
aliases:
Expand All @@ -16,6 +26,3 @@ workers:
implementation: docker-worker
os: linux
worker-type: 'b-linux'

# Uncomment if this is a private repo
# github_clone_secret: project/xpi/xpi-github-clone-ssh
2 changes: 2 additions & 0 deletions taskcluster/ci/test/kind.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,8 @@ job-defaults:
docker-image:
indexed: xpi.cache.level-3.docker-images.v2.node-latest.latest
run:
checkout:
xpi: {}
use-caches: false

jobs:
Expand Down
12 changes: 11 additions & 1 deletion taskcluster/xpi_taskgraph/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,4 +12,14 @@ def register(graph_config):
Import all modules that are siblings of this one, triggering decorators in
the process.
"""
pass
_import_modules([
"build",
"cached",
"test",
"xpi_manifest",
])


def _import_modules(modules):
for module in modules:
import_module(".{}".format(module), package=__name__)
8 changes: 3 additions & 5 deletions taskcluster/xpi_taskgraph/build.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,12 +32,10 @@ def tasks_from_manifest(config, jobs):
task["label"] = "build-{}".format(xpi_config["name"])
env["XPI_NAME"] = xpi_config["name"]
task.setdefault("extra", {})["xpi-name"] = xpi_config["name"]
try:
checkout_config["ssh_secret_name"] = config.graph_config[
"github_clone_secret"
]
if env.get("GITHUB_CLONE_SECRET", ""):
checkout_config["ssh_secret_name"] = env["GITHUB_CLONE_SECRET"]
artifact_prefix = "xpi/build"
except KeyError:
else:
artifact_prefix = "public/build"
env["ARTIFACT_PREFIX"] = artifact_prefix
artifacts = task["worker"].setdefault("artifacts", [])
Expand Down
8 changes: 3 additions & 5 deletions taskcluster/xpi_taskgraph/test.py
Original file line number Diff line number Diff line change
Expand Up @@ -41,12 +41,10 @@ def test_tasks_from_manifest(config, jobs):
run["cwd"] = "{checkout}"
run["command"] = run["command"].format(target=target)
task["label"] = "t-{}-{}".format(target, xpi_name)
try:
checkout_config["ssh_secret_name"] = config.graph_config[
"github_clone_secret"
]
if env.get("GITHUB_CLONE_SECRET", ""):
checkout_config["ssh_secret_name"] = env["GITHUB_CLONE_SECRET"]
artifact_prefix = "xpi/build"
except KeyError:
else:
artifact_prefix = "public/build"
env["ARTIFACT_PREFIX"] = artifact_prefix
yield task

0 comments on commit dc7a87b

Please sign in to comment.