Skip to content

DEBUG Demonstrate new functionality #4

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
jugglinmike wants to merge 4 commits into
base: master
Choose a base branch
from
Open

DEBUG Demonstrate new functionality #4

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
ca15a0c
[infra] Validate pull requests in TaskCluster
jugglinmike Aug 7, 2018
be61008
[infra] Temporarily ignore stability status in CI
jugglinmike Aug 23, 2018
6ae38c9
Enable TaskCluster for this fork of WPT
jugglinmike Aug 23, 2018
e34172d
DEBUG Demonstrate new functionality
jugglinmike Aug 22, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
187 changes: 134 additions & 53 deletions .taskcluster.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,76 +2,157 @@ version: 1
policy:
pullRequests: collaborators
tasks:
$if: tasks_for == "github-push"
then:
$if: event.ref == "refs/heads/master"
then:
$flattenDeep:
$flattenDeep:
- $if: tasks_for == "github-push"
then:
$if: event.ref == "refs/heads/master"
then:
$map: [{name: firefox, channel: nightly}, {name: chrome, channel: dev}]
each(browser):
$map:
- [testharness, 1, 15]
- [testharness, 2, 15]
- [testharness, 3, 15]
- [testharness, 4, 15]
- [testharness, 5, 15]
- [testharness, 6, 15]
- [testharness, 7, 15]
- [testharness, 8, 15]
- [testharness, 9, 15]
- [testharness, 10, 15]
- [testharness, 11, 15]
- [testharness, 12, 15]
- [testharness, 13, 15]
- [testharness, 14, 15]
- [testharness, 15, 15]
- [reftest, 1, 10]
- [reftest, 2, 10]
- [reftest, 3, 10]
- [reftest, 4, 10]
- [reftest, 5, 10]
- [reftest, 6, 10]
- [reftest, 7, 10]
- [reftest, 8, 10]
- [reftest, 9, 10]
- [reftest, 10, 10]
- [wdspec, 1, 1]
each(chunk):
taskId: {$eval: 'as_slugid(browser.name + browser.channel + chunk[0] + str(chunk[1]))'}
taskGroupId: {$eval: 'as_slugid("task group")'}
created: {$fromNow: ''}
deadline: {$fromNow: '24 hours'}
provisionerId: aws-provisioner-v1
# Contributors interested in configurating TaskCluster to run
# against their fork of WPT should change the `workerType` to
# "github-worker".
workerType: github-worker
metadata:
name: wpt-${browser.name}-${browser.channel}-${chunk[0]}-${chunk[1]}
description: >-
A subset of WPT's "${chunk[0]}" tests (chunk number ${chunk[1]}
of ${chunk[2]}), run in the ${browser.channel} release of
${browser.name}.
owner: ${event.pusher.email}
source: ${event.repository.url}
payload:
image: gsnedders/web-platform-tests:0.13
maxRunTime: 7200
artifacts:
public/results:
path: /home/test/artifacts
type: directory
command:
- /bin/bash
- --login
- -c
- set -ex;
~/start.sh
${event.repository.url}
${event.ref[len('refs/heads/'):]}
${event.after}
${browser.name}-${browser.channel};
cd ~/web-platform-tests;
./tools/ci/ci_taskcluster.sh
--test-type=${chunk[0]}
--this-chunk=${chunk[1]}
--total-chunks=${chunk[2]}
${browser.name};
- $if: tasks_for == "github-pull-request"
then:
$map: [{name: firefox, channel: nightly}, {name: chrome, channel: dev}]
each(browser):
$map:
- [testharness, 1, 15]
- [testharness, 2, 15]
- [testharness, 3, 15]
- [testharness, 4, 15]
- [testharness, 5, 15]
- [testharness, 6, 15]
- [testharness, 7, 15]
- [testharness, 8, 15]
- [testharness, 9, 15]
- [testharness, 10, 15]
- [testharness, 11, 15]
- [testharness, 12, 15]
- [testharness, 13, 15]
- [testharness, 14, 15]
- [testharness, 15, 15]
- [reftest, 1, 10]
- [reftest, 2, 10]
- [reftest, 3, 10]
- [reftest, 4, 10]
- [reftest, 5, 10]
- [reftest, 6, 10]
- [reftest, 7, 10]
- [reftest, 8, 10]
- [reftest, 9, 10]
- [reftest, 10, 10]
- [wdspec, 1, 1]
each(chunk):
taskId: {$eval: 'as_slugid(browser.name + browser.channel + chunk[0] + str(chunk[1]))'}
- name: wpt-${browser.name}-${browser.channel}-stability
description: >-
Verify that all tests affected by a pull request are stable
when executed in ${browser.name}. As of 2018-08-23, this task
is being vetted for correctness, so its outcome will not
influence the status of the pull request.
command:
status=PASS;
echo $affected |
base64 -d |
xargs -0 ./tools/ci/ci_taskcluster.sh --verify ${browser.name} || status=FAIL;
echo $status > ../artifacts/stability-status.txt
creates_artifacts: true
- name: wpt-${browser.name}-${browser.channel}-results
description: >-
Collect results for all tests affected by a pull request in
${browser.name}.
command:
echo $affected |
base64 -d |
xargs -0 ./tools/ci/ci_taskcluster.sh ${browser.name}
creates_artifacts: true
each(operation):
taskId: {$eval: 'as_slugid(operation.name)'}
taskGroupId: {$eval: 'as_slugid("task group")'}
created: {$fromNow: ''}
deadline: {$fromNow: '24 hours'}
provisionerId: aws-provisioner-v1
# Contributors interested in configurating TaskCluster to run
# against their fork of WPT should change the `workerType` to
# "github-worker".
workerType: wpt-docker-worker
workerType: github-worker
metadata:
name: wpt-${browser.name}-${browser.channel}-${chunk[0]}-${chunk[1]}
description: >-
A subset of WPT's "${chunk[0]}" tests (chunk number ${chunk[1]}
of ${chunk[2]}), run in the ${browser.channel} release of
${browser.name}.
owner: ${event.pusher.email}
name: ${operation.name}
description: ${operation.description}
owner: ${event.pull_request.user.login}@users.noreply.github.com
source: ${event.repository.url}
payload:
image: gsnedders/web-platform-tests:0.13
maxRunTime: 7200
artifacts:
public/results:
path: /home/test/artifacts
type: directory
$if: operation.creates_artifacts
then:
public/results:
path: /home/test/artifacts
type: directory
# Bash removes null bytes from string values when set as
# environment variables. This invalidates the output of `wpt
# affected-tests` because it uses the null byte as a separator
# between test names. The list of effected tests is
# Base64-encoded in order to preserve the null bytes.
command:
- /bin/bash
- --login
- -c
- "~/start.sh ${event.repository.url} ${event.ref[len('refs/heads/'):]} ${event.after} ${browser.name}-${browser.channel} &&
cd ~/web-platform-tests &&
./tools/ci/ci_taskcluster.sh ${browser.name} ${chunk[0]} ${chunk[1]} ${chunk[2]}"
else: []
# > NOTE: A well-designed template should produce `tasks: []` for any
# > unrecognized `task_for` values; this allows later expansion of this
# > service to handle more events.
#
# https://docs.taskcluster.net/docs/reference/integrations/taskcluster-github/docs/taskcluster-yml-v1
else: []
- set -ex;
~/start.sh
${event.repository.clone_url}
${event.pull_request.base.ref}
${event.pull_request.base.sha}
${browser.name}-${browser.channel};
cd ~/web-platform-tests;
./wpt manifest-download || true;
git fetch
${event.repository.clone_url}
refs/pull/${event.number}/head;
git checkout FETCH_HEAD;
affected=$(./wpt tests-affected
--null
${event.pull_request.base.sha}
| base64);
echo Affected tests:;
echo $affected | base64 -d | sed 's/\x0/\\n/g';
${operation.command}
4 changes: 4 additions & 0 deletions dom/events/CustomEvent.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,10 @@
<script src="/resources/testharnessreport.js"></script>
<div id="log"></div>
<script>
test(function() {
assert_true(Math.random() > 0.5);
}, "Intentionally unstable test");

test(function() {
var type = "foo";

Expand Down
44 changes: 39 additions & 5 deletions tools/ci/ci_taskcluster.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,43 @@

./wpt manifest-download

if [ $1 == "firefox" ]; then
./wpt run firefox --log-tbpl=../artifacts/log_tbpl.log --log-tbpl-level=info --log-wptreport=../artifacts/wpt_report.json --log-mach=- --this-chunk=$3 --total-chunks=$4 --test-type=$2 -y --install-browser --no-pause --no-restart-on-unexpected --reftest-internal --install-fonts --no-fail-on-unexpected
elif [ $1 == "chrome" ]; then
./wpt run chrome --log-tbpl=../artifacts/log_tbpl.log --log-tbpl-level=info --log-wptreport=../artifacts/wpt_report.json --log-mach=- --this-chunk=$3 --total-chunks=$4 --test-type=$2 -y --no-pause --no-restart-on-unexpected --install-fonts --no-fail-on-unexpected
# The first script argument that is not prefixed with a dash (`-`) is assumed
# to be the name of the browser under test. This restricts the syntax available
# to consumers: value-accepting options must be specified using the equals sign
# (`=`).
for argument in $@; do
if [ ${argument:0:1} == '-' ]; then
continue
fi

browser_name=$arg

break
done

browser_specific_args=''

if [ $browser_name == 'firefox' ]; then
browser_specific_args='--install-browser --reftest-internal'
fi
gzip ../artifacts/wpt_report.json

./wpt run \
--log-tbpl=../artifacts/log_tbpl.log \
--log-tbpl-level=info \
--log-wptreport=../artifacts/wpt_report.json \
--log-mach=- \
-y \
--no-pause \
--no-restart-on-unexpected \
--install-fonts \
--no-fail-on-unexpected \
$browser_specific_args \
$@

run_status=$?

if [ -f ../artifacts/wpt_report.json ]; then
gzip ../artifacts/wpt_report.json
fi

exit $run_status