mirror of
https://github.com/nexu-io/open-design.git
synced 2026-10-27 09:51:12 +01:00
ci: surface merge-queue CI failure ejections as a PR comment (#7229)
* ci: surface merge-queue CI failure ejections as a PR comment A failed workload in a merge_group run ejects the queued PR just as invisibly as a blocking label does: the required check fails on the queue's transient ref, the PR's own checks stay green, and the queue entry silently disappears. The needs-validation gate (#5519) announces its own ejections, but every other ejection left no trace on the PR. The validate job now produces a handoff/comment artifact after the gate has already failed in a merge group, naming the failed jobs and steps, a log excerpt of the failing assertions, the run, and the PRs batched ahead in the group. comment.atom.yml already consumes merge_group handoffs. merge_policy exposes whether it already announced a label ejection so a label-only ejection still gets exactly one comment. * ci: keep the merge-queue log excerpt working on gh >= 2.97 gh 2.97.0 (GHSA-3m3g-3wcr-px46) refuses to print an API body that contains terminal escape sequences unless --allow-escape-sequences is passed, and CI job logs are full of them, so the excerpt fetch returned nothing on a current gh. Older gh rejects the flag as unknown. Probe the flag once and pass it only when supported; the fake gh in the topology test now enforces both behaviors so dropping the probe goes red. * ci: make the merge-queue notice probe survive pipefail and replays Capturing gh api --help before grepping keeps an early grep exit from SIGPIPE-ing gh and reading as "flag unsupported" under pipefail; strip CRLF from job logs; never list the gate job itself as a failed job when a completed run is replayed. The fake gh pads its help text past the pipe buffer so the old probe shape stays red.
This commit is contained in:
lefarcen
@@ -670,6 +670,10 @@ jobs:
|
||||
if: ${{ github.event_name == 'merge_group' }}
|
||||
runs-on: ${{ fromJSON(needs.runners.outputs.runs_on).control }}
|
||||
timeout-minutes: 5
|
||||
# Tells the generic merge-queue failure notice in `validate` whether this job already
|
||||
# announced the ejection, so a label-only ejection gets exactly one comment.
|
||||
outputs:
|
||||
ejection_notice: ${{ steps.merge_blocking_label_gate.outputs.comment_created }}
|
||||
|
||||
steps:
|
||||
# The merge-blocking label gate produces a `handoff/comment` artifact when it ejects a
|
||||
@@ -922,6 +926,188 @@ jobs:
|
||||
path: ${{ runner.temp }}/hash-state.json
|
||||
key: ci-hash-${{ runner.os }}-${{ github.ref }}-${{ github.run_id }}-${{ github.run_attempt }}
|
||||
|
||||
# A failed workload ejects the queued PR just as invisibly as a blocking label does: this
|
||||
# required check fails on the queue's transient ref, the PR's own checks stay green, and
|
||||
# its queue entry silently disappears. The merge-blocking label gate announces its own
|
||||
# ejections; this step announces every other one (failed or cancelled workloads, a
|
||||
# fail-closed policy error) so the author learns which jobs failed and where to look.
|
||||
# Best-effort by construction: it only runs after the gate has already failed and can
|
||||
# never soften, mask, or retry that result.
|
||||
- name: Checkout handoff helper
|
||||
if: ${{ failure() && github.event_name == 'merge_group' }}
|
||||
uses: actions/checkout@v6.0.2
|
||||
with:
|
||||
sparse-checkout: .github
|
||||
sparse-checkout-cone-mode: true
|
||||
|
||||
- name: Produce merge-queue failure notice handoff
|
||||
id: merge_queue_failure_notice
|
||||
if: ${{ failure() && github.event_name == 'merge_group' }}
|
||||
continue-on-error: true
|
||||
env:
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
REPO: ${{ github.repository }}
|
||||
RUN_ID: ${{ github.run_id }}
|
||||
NEEDS_JSON: ${{ toJSON(needs) }}
|
||||
# The queue names the PR that heads this group in its temporary ref
|
||||
# (`gh-readonly-queue/<base>/pr-<N>-<base sha>`); that PR owns this run's result.
|
||||
MERGE_GROUP_REF: ${{ github.event.merge_group.head_ref }}
|
||||
BASE_SHA: ${{ github.event.merge_group.base_sha }}
|
||||
HEAD_SHA: ${{ github.event.merge_group.head_sha }}
|
||||
# `true` when the merge-blocking label gate already produced its own ejection notice.
|
||||
LABEL_NOTICE_EMITTED: ${{ needs.merge_policy.outputs.ejection_notice }}
|
||||
# This job's own display name: it is still in progress while this step runs, but a
|
||||
# replay of a completed run would otherwise list the gate itself as a failed job.
|
||||
SELF_JOB_NAME: Validate workspace
|
||||
run: |
|
||||
set -euo pipefail
|
||||
|
||||
# Which jobs the gate just rejected. Mirrors its two scans: any non-success,
|
||||
# non-skipped result plus any plan-required job that did not succeed. A broken plan
|
||||
# output must not break the notice, so the required-job scan is fault tolerant here.
|
||||
failed_jobs="$(jq -r '
|
||||
. as $needs |
|
||||
(try (($needs.plan.outputs.run // "{}") | fromjson) catch {}) as $run |
|
||||
(
|
||||
[ $needs | to_entries[] | select(.value.result != "success" and .value.result != "skipped") | .key ]
|
||||
+ [ (["plan"] + [$run | to_entries[] | select(.value) | .key])[]
|
||||
| select(($needs[.].result // "missing") != "success") ]
|
||||
) | unique[]' <<< "$NEEDS_JSON")"
|
||||
|
||||
pr="$(printf '%s\n' "$MERGE_GROUP_REF" | grep -oE 'pr-[0-9]+' | head -n 1 | grep -oE '[0-9]+' || true)"
|
||||
if [ -z "$pr" ]; then
|
||||
echo "::warning::merge_group ref '$MERGE_GROUP_REF' names no pull request; skipping the ejection notice."
|
||||
exit 0
|
||||
fi
|
||||
pr_json="$(gh api "repos/$REPO/pulls/$pr")"
|
||||
pr_head="$(jq -r '.head.sha' <<< "$pr_json")"
|
||||
pr_base="$(jq -r '.base.sha' <<< "$pr_json")"
|
||||
|
||||
# A label ejection is already announced by the gate's own notice (and a PR that still
|
||||
# carries a blocking label is about to get one). Drop the policy job from this notice
|
||||
# so a label-only ejection gets exactly one comment; a workload that failed alongside
|
||||
# the label is still reported.
|
||||
label_covered=false
|
||||
if [ "${LABEL_NOTICE_EMITTED:-}" = "true" ]; then
|
||||
label_covered=true
|
||||
elif jq -e '[.labels[]?.name] | any(. == "needs-validation" or . == "needs-maintainer-check")' <<< "$pr_json" >/dev/null; then
|
||||
label_covered=true
|
||||
fi
|
||||
skip_job_name=""
|
||||
if [ "$label_covered" = "true" ]; then
|
||||
skip_job_name="Merge policy"
|
||||
failed_jobs="$(printf '%s\n' "$failed_jobs" | grep -vx 'merge_policy' || true)"
|
||||
fi
|
||||
if [ -z "$failed_jobs" ]; then
|
||||
echo "Only the merge-blocking label gate failed; its own notice covers this ejection."
|
||||
exit 0
|
||||
fi
|
||||
echo "Failed validation jobs: $(printf '%s' "$failed_jobs" | tr '\n' ' ')"
|
||||
|
||||
# Job-level detail straight from the run: display names, the failing step, and a log
|
||||
# link per failed job. Read-only and best-effort; the notice still ships without it.
|
||||
jobs_json="$(gh api --paginate "repos/$REPO/actions/runs/$RUN_ID/jobs?per_page=100" | jq -s '[.[].jobs[]]' || echo '[]')"
|
||||
# Markdown code spans are intentionally literal in these single-quoted strings.
|
||||
# shellcheck disable=SC2016
|
||||
failed_lines="$(jq -r --arg skip "$skip_job_name" --arg self "$SELF_JOB_NAME" '
|
||||
.[]
|
||||
| select(.conclusion != null and .conclusion != "success" and .conclusion != "skipped" and .name != $skip and .name != $self)
|
||||
| "- **\(.name)** — \(.conclusion)"
|
||||
+ ([.steps[]? | select(.conclusion == "failure") | .name] | if length > 0 then " at `" + join("`, `") + "`" else "" end)
|
||||
+ " ([job log](\(.html_url)))"' <<< "$jobs_json")"
|
||||
if [ -z "$failed_lines" ]; then
|
||||
# shellcheck disable=SC2016
|
||||
failed_lines="$(printf '%s\n' "$failed_jobs" | sed -E 's/^/- `/; s/$/`/')"
|
||||
fi
|
||||
|
||||
# Pull the failing assertions out of the job logs (vitest `FAIL` lines, Playwright
|
||||
# `✘` lines, runner error annotations), capped so the comment stays readable.
|
||||
# CI logs are full of terminal escape sequences, and gh >= 2.97 refuses to print such
|
||||
# a body unless told otherwise (GHSA-3m3g-3wcr-px46) while older gh does not know the
|
||||
# flag — probe once so the excerpt works with either runner image.
|
||||
gh --version | head -n 1
|
||||
# Capture the help text first: `gh api --help | grep -q` would let grep exit early and
|
||||
# fail the pipeline under pipefail, silently disabling the flag.
|
||||
api_help="$(gh api --help 2>/dev/null || true)"
|
||||
escape_flag=""
|
||||
if grep -q -- '--allow-escape-sequences' <<< "$api_help"; then
|
||||
escape_flag="--allow-escape-sequences"
|
||||
fi
|
||||
esc="$(printf '\033')"
|
||||
excerpt=""
|
||||
for job_id in $(jq -r --arg skip "$skip_job_name" --arg self "$SELF_JOB_NAME" '.[] | select(.conclusion == "failure" and .name != $skip and .name != $self) | .id' <<< "$jobs_json" | head -n 4); do
|
||||
lines="$(gh api ${escape_flag:+"$escape_flag"} "repos/$REPO/actions/jobs/$job_id/logs" 2>/dev/null \
|
||||
| tr -d '\r' \
|
||||
| sed -E "s/${esc}\[[0-9;]*[A-Za-z]//g; s/^[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9:.]+Z //" \
|
||||
| grep -E '^ ?FAIL |^[[:space:]]*[✘×] |^##\[error\]' \
|
||||
| grep -vE '^##\[error\]Process completed with exit code' \
|
||||
| sed -E 's/^##\[error\]//; s/^[[:space:]]+//' \
|
||||
| awk '!seen[$0]++' \
|
||||
| head -n 8 || true)"
|
||||
if [ -n "$lines" ]; then
|
||||
excerpt="${excerpt}${lines}"$'\n'
|
||||
fi
|
||||
done
|
||||
# Shards that fail on the same test would otherwise repeat the same lines per job.
|
||||
if [ -n "$excerpt" ]; then
|
||||
excerpt="$(printf '%s' "$excerpt" | awk '!seen[$0]++')"$'\n'
|
||||
fi
|
||||
|
||||
# Other PRs batched ahead of this one in the same group (their squashed commits sit
|
||||
# between the group base and head), so the author can tell whose change may have failed.
|
||||
group_note=""
|
||||
others="$(gh api "repos/$REPO/compare/$BASE_SHA...$HEAD_SHA" 2>/dev/null \
|
||||
| jq -r '.commits[].commit.message | split("\n")[0]' \
|
||||
| grep -oE '\(#[0-9]+\)$' | grep -oE '[0-9]+' | grep -vx "$pr" | sort -un | sed 's/^/#/' | paste -s -d ',' - | sed 's/,/, /g' || true)"
|
||||
if [ -n "$others" ]; then
|
||||
group_note=" and with the PRs queued ahead of it ($others)"
|
||||
fi
|
||||
|
||||
handoff_id="merge-queue-ci-failure-pr-$pr"
|
||||
marker="<!-- merge-queue-ci-failure -->"
|
||||
handoff_root="$RUNNER_TEMP/handoff-comment-$handoff_id"
|
||||
handoff_dir="$(python3 .github/scripts/handoff.py dir comment "$handoff_id" --root "$handoff_root")"
|
||||
mkdir -p "$handoff_dir"
|
||||
run_url="$GITHUB_SERVER_URL/$REPO/actions/runs/$RUN_ID"
|
||||
# Markdown code spans are intentionally literal in these single-quoted strings.
|
||||
# shellcheck disable=SC2016
|
||||
{
|
||||
printf '%s\n' "$marker"
|
||||
printf 'Ejected from the merge queue: CI failed on the queued merge of this PR.\n\n'
|
||||
printf 'The merge queue gate ([run %s](%s)) failed for the group headed by this PR. That run executes on the queue'"'"'s transient ref, so the failure never shows up in this PR'"'"'s own checks — they stay green, the queue entry simply disappears, and this notice is the only visible trace on the PR.\n\n' "$RUN_ID" "$run_url"
|
||||
printf 'Failed jobs:\n\n%s\n\n' "$failed_lines"
|
||||
if [ -n "$excerpt" ]; then
|
||||
printf 'Failure excerpt (best-effort, from the job logs):\n\n```text\n%s```\n\n' "$excerpt"
|
||||
fi
|
||||
printf 'The queued merge combines this PR with everything that landed on `main` after its last CI run%s. When this PR'"'"'s own checks are green, the failure is usually one of: a conflict with newer `main` (for example a test added or changed since this branch diverged), a PR ahead of it in the queue, or a flaky test.\n\n' "$group_note"
|
||||
printf 'To land this PR: open the run above and check whether the failure is related to this change. If it is, merge or rebase onto the latest `main`, fix, push, and add the PR back to the merge queue. If it is unrelated, add the PR back to the merge queue.\n'
|
||||
} > "$handoff_dir/body.md"
|
||||
jq -n \
|
||||
--arg kind "comment" \
|
||||
--arg id "$handoff_id" \
|
||||
--arg head "$pr_head" \
|
||||
--arg base "$pr_base" \
|
||||
--arg marker "$marker" \
|
||||
--argjson schema_version 1 \
|
||||
--argjson pr_number "$pr" \
|
||||
--argjson run_id "$RUN_ID" \
|
||||
'{schema_version: $schema_version, kind: $kind, id: $id, pr_number: $pr_number, head_sha: $head, base_sha: $base, run_id: $run_id, marker: $marker}' \
|
||||
> "$handoff_dir/metadata.json"
|
||||
python3 .github/scripts/handoff.py validate comment "$handoff_dir" >/dev/null
|
||||
{
|
||||
echo "comment_created=true"
|
||||
echo "comment_name=$(python3 .github/scripts/handoff.py artifact-name comment "$handoff_id")"
|
||||
echo "comment_path=$handoff_root"
|
||||
} >> "$GITHUB_OUTPUT"
|
||||
echo "Produced merge-queue failure notice for PR #$pr."
|
||||
|
||||
- name: Upload merge-queue failure notice handoff
|
||||
if: ${{ failure() && steps.merge_queue_failure_notice.outputs.comment_created == 'true' }}
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: ${{ steps.merge_queue_failure_notice.outputs.comment_name }}
|
||||
path: ${{ steps.merge_queue_failure_notice.outputs.comment_path }}
|
||||
|
||||
runtime_summary:
|
||||
name: Runtime summary
|
||||
needs:
|
||||
|
||||
@@ -576,6 +576,269 @@ describe("packaged smoke workflow", () => {
|
||||
expect(commentWorkflow).toContain('[ "$RUN_EVENT" != "merge_group" ] && [ "$current_base" != "$base_sha" ]');
|
||||
});
|
||||
|
||||
it("[P2] surfaces a merge-queue CI failure ejection as a PR comment handoff", async () => {
|
||||
const ciWorkflow = await readFile(ciWorkflowPath, "utf8");
|
||||
const mergePolicy = sectionBetween(ciWorkflow, " merge_policy:", " validate:");
|
||||
const validate = sectionBetween(ciWorkflow, " validate:", " runtime_summary:");
|
||||
|
||||
// The label gate tells the generic notice whether it already announced this ejection, so a
|
||||
// label-only ejection gets exactly one comment.
|
||||
expect(mergePolicy).toContain(
|
||||
"ejection_notice: ${{ steps.merge_blocking_label_gate.outputs.comment_created }}",
|
||||
);
|
||||
expect(validate).toContain("LABEL_NOTICE_EMITTED: ${{ needs.merge_policy.outputs.ejection_notice }}");
|
||||
|
||||
// Producer: merge-group only, only after the gate has already failed, unable to change the
|
||||
// gate result, and uploaded on the failure path exactly like the label notice.
|
||||
expect(validate).toContain("<!-- merge-queue-ci-failure -->");
|
||||
expect(validate).toContain("if: ${{ failure() && github.event_name == 'merge_group' }}");
|
||||
expect(validate).toContain("continue-on-error: true");
|
||||
expect(validate).toContain(
|
||||
"if: ${{ failure() && steps.merge_queue_failure_notice.outputs.comment_created == 'true' }}",
|
||||
);
|
||||
// It sits after hash publication so the success path is untouched and the gate's own jq
|
||||
// programs stay isolated (see extractValidateGateJqPrograms).
|
||||
expect(validate.indexOf("Save successful hash map")).toBeLessThan(
|
||||
validate.indexOf("Produce merge-queue failure notice handoff"),
|
||||
);
|
||||
|
||||
// Behavior: run the real step script against a stubbed `gh`, then read the handoff it
|
||||
// produced through the same helper comment.atom.yml uses to consume it.
|
||||
const script = extractWorkflowRunScript(ciWorkflow, "Produce merge-queue failure notice handoff");
|
||||
const prHead = "4821c4ee".padEnd(40, "0");
|
||||
const prBase = "1a7a23d4".padEnd(40, "0");
|
||||
const groupHead = "e3051e63".padEnd(40, "0");
|
||||
const runId = "32105196624";
|
||||
const planRun = JSON.stringify({ static_gate: true, daemon_unit_tests: true, e2e_vitest: false });
|
||||
const esc = String.fromCharCode(27);
|
||||
const failedShardLog = [
|
||||
`2026-08-18T06:05:15.3329113Z ${esc}[31m⎯⎯⎯ Failed Tests 1 ⎯⎯⎯${esc}[39m`,
|
||||
`2026-08-18T06:05:15.3330283Z ${esc}[41m${esc}[1m FAIL ${esc}[22m${esc}[49m tests/project-archive.test.ts > buildProjectArchive > exposes a consumable stream with the same archive contents`,
|
||||
`2026-08-18T06:05:15.3333182Z ${esc}[31mAssertionError: expected [ Array(6) ] to deeply equal [ 'DESIGN-HANDOFF.md', …(4) ]${esc}[39m`,
|
||||
"2026-08-18T06:05:15.3398469Z ##[error]AssertionError: expected [ Array(6) ] to deeply equal [ 'DESIGN-HANDOFF.md', …(4) ]",
|
||||
"2026-08-18T06:05:15.4000000Z ##[error]Process completed with exit code 1.",
|
||||
"",
|
||||
].join("\n");
|
||||
|
||||
async function runNotice(args: {
|
||||
needs: Record<string, unknown>;
|
||||
labelNoticeEmitted?: string;
|
||||
labels?: string[];
|
||||
mergePolicyConclusion?: string;
|
||||
// gh >= 2.97 refuses to print a body with terminal escape sequences unless
|
||||
// `--allow-escape-sequences` is passed; older gh rejects that flag as unknown.
|
||||
ghSupportsEscapeFlag?: boolean;
|
||||
}): Promise<{
|
||||
stdout: string;
|
||||
output: Record<string, string>;
|
||||
listed: Array<Record<string, unknown>>;
|
||||
body: string;
|
||||
}> {
|
||||
const dir = await mkdtemp(join(tmpdir(), "od-merge-queue-notice-"));
|
||||
const ghPath = join(dir, "gh");
|
||||
const outputPath = join(dir, "github-output");
|
||||
const runnerTemp = join(dir, "runner-temp");
|
||||
await mkdir(runnerTemp);
|
||||
await writeFile(outputPath, "");
|
||||
const mergePolicyConclusion = args.mergePolicyConclusion ?? "success";
|
||||
const fixtures = {
|
||||
pull: {
|
||||
head: { sha: prHead },
|
||||
base: { sha: prBase },
|
||||
labels: (args.labels ?? []).map((name) => ({ name })),
|
||||
},
|
||||
jobs: {
|
||||
jobs: [
|
||||
{
|
||||
id: 101,
|
||||
name: "Daemon tests (1/4)",
|
||||
conclusion: "failure",
|
||||
html_url: `https://github.com/nexu-io/open-design/actions/runs/${runId}/job/101`,
|
||||
steps: [
|
||||
{ name: "Checkout", conclusion: "success" },
|
||||
{ name: "Run daemon test shard", conclusion: "failure" },
|
||||
],
|
||||
},
|
||||
{ id: 102, name: "Daemon tests (2/4)", conclusion: "success", html_url: "", steps: [] },
|
||||
{
|
||||
id: 103,
|
||||
name: "Merge policy",
|
||||
conclusion: mergePolicyConclusion,
|
||||
html_url: `https://github.com/nexu-io/open-design/actions/runs/${runId}/job/103`,
|
||||
steps: [{ name: "Block merge while a merge-blocking label is present", conclusion: mergePolicyConclusion }],
|
||||
},
|
||||
// Replaying a completed run reports the gate itself as failed; it must never be listed.
|
||||
{ id: 104, name: "Validate workspace", conclusion: "failure", html_url: "", steps: [] },
|
||||
],
|
||||
},
|
||||
compare: {
|
||||
commits: [
|
||||
{ commit: { message: "fix(daemon): keep daemon as sole writer of run events (#6559)\n\nbody" } },
|
||||
{ commit: { message: "fix(daemon): list dot-prefixed user content in managed projects (#6214)" } },
|
||||
],
|
||||
},
|
||||
logs: failedShardLog,
|
||||
};
|
||||
const supportsEscapeFlag = args.ghSupportsEscapeFlag ?? true;
|
||||
await writeFile(
|
||||
ghPath,
|
||||
`#!/usr/bin/env node
|
||||
const fixtures = JSON.parse(process.env.FAKE_GH_FIXTURES);
|
||||
const supportsEscapeFlag = ${supportsEscapeFlag ? "true" : "false"};
|
||||
const argv = process.argv.slice(2);
|
||||
const args = argv.join(" ");
|
||||
const hasEscapeFlag = argv.includes("--allow-escape-sequences");
|
||||
if (args === "--version") process.stdout.write(supportsEscapeFlag ? "gh version 2.98.0 (fake)\\n" : "gh version 2.87.3 (fake)\\n");
|
||||
// Real help text is long: a probe that lets grep exit early would SIGPIPE gh and, under
|
||||
// pipefail, read as "flag unsupported". Pad well past the pipe buffer so that stays red.
|
||||
else if (args === "api --help") process.stdout.write((supportsEscapeFlag ? " --allow-escape-sequences Allow printing terminal escape sequences\\n" : " --cache duration Cache the response\\n") + " --padding\\n".repeat(8192));
|
||||
else if (hasEscapeFlag && !supportsEscapeFlag) { process.stderr.write("unknown flag: --allow-escape-sequences\\n"); process.exit(1); }
|
||||
else if (/\\/pulls\\/6214$/.test(args)) process.stdout.write(JSON.stringify(fixtures.pull));
|
||||
else if (/\\/actions\\/runs\\/${runId}\\/jobs/.test(args)) process.stdout.write(JSON.stringify(fixtures.jobs));
|
||||
else if (/\\/actions\\/jobs\\/\\d+\\/logs$/.test(args)) {
|
||||
// Real gh >= 2.97 behavior for a body full of ANSI codes: nothing on stdout, exit 1.
|
||||
if (supportsEscapeFlag && !hasEscapeFlag) { process.stderr.write("the response contains terminal escape sequences; pass --allow-escape-sequences to output it anyway\\n"); process.exit(1); }
|
||||
process.stdout.write(fixtures.logs);
|
||||
}
|
||||
else if (/\\/compare\\//.test(args)) process.stdout.write(JSON.stringify(fixtures.compare));
|
||||
else { process.stderr.write("unexpected gh call: " + args + "\\n"); process.exit(1); }
|
||||
`,
|
||||
);
|
||||
await chmod(ghPath, 0o755);
|
||||
try {
|
||||
const { stdout } = await execFileAsync("bash", ["-c", script], {
|
||||
cwd: workspaceRoot,
|
||||
env: workflowFixtureEnv(
|
||||
{
|
||||
GH_TOKEN: "fake",
|
||||
REPO: "nexu-io/open-design",
|
||||
RUN_ID: runId,
|
||||
NEEDS_JSON: JSON.stringify(args.needs),
|
||||
MERGE_GROUP_REF: `gh-readonly-queue/main/pr-6214-${prBase}`,
|
||||
BASE_SHA: prBase,
|
||||
HEAD_SHA: groupHead,
|
||||
LABEL_NOTICE_EMITTED: args.labelNoticeEmitted ?? "",
|
||||
SELF_JOB_NAME: "Validate workspace",
|
||||
GITHUB_OUTPUT: outputPath,
|
||||
GITHUB_SERVER_URL: "https://github.com",
|
||||
RUNNER_TEMP: runnerTemp,
|
||||
FAKE_GH_FIXTURES: JSON.stringify(fixtures),
|
||||
},
|
||||
dir,
|
||||
),
|
||||
});
|
||||
const output = parseGithubOutput(await readFile(outputPath, "utf8"));
|
||||
let listed: Array<Record<string, unknown>> = [];
|
||||
let body = "";
|
||||
if (output.comment_path) {
|
||||
const { stdout: listStdout } = await execFileAsync(
|
||||
"python3",
|
||||
[handoffScriptPath, "list", "comment", output.comment_path],
|
||||
{ cwd: workspaceRoot },
|
||||
);
|
||||
listed = listStdout
|
||||
.trim()
|
||||
.split("\n")
|
||||
.filter(Boolean)
|
||||
.map((line) => JSON.parse(line) as Record<string, unknown>);
|
||||
body = await readFile(
|
||||
join(output.comment_path, "handoff", "comment", "merge-queue-ci-failure-pr-6214", "body.md"),
|
||||
"utf8",
|
||||
);
|
||||
}
|
||||
return { stdout, output, listed, body };
|
||||
} finally {
|
||||
await rm(dir, { recursive: true, force: true });
|
||||
}
|
||||
}
|
||||
|
||||
// A failed workload: one notice for the PR heading the group, naming the job, the step,
|
||||
// the failing assertion, the run, and the PR batched ahead of it.
|
||||
const workloadFailure = await runNotice({
|
||||
needs: {
|
||||
plan: { result: "success", outputs: { run: planRun } },
|
||||
merge_policy: { result: "success", outputs: { ejection_notice: "" } },
|
||||
static_gate: { result: "success" },
|
||||
daemon_unit_tests: { result: "failure" },
|
||||
e2e_vitest: { result: "skipped" },
|
||||
},
|
||||
});
|
||||
expect(workloadFailure.output.comment_created).toBe("true");
|
||||
expect(workloadFailure.output.comment_name).toBe("handoff-comment-merge-queue-ci-failure-pr-6214");
|
||||
expect(workloadFailure.listed).toHaveLength(1);
|
||||
expect(workloadFailure.listed[0]).toMatchObject({
|
||||
kind: "comment",
|
||||
id: "merge-queue-ci-failure-pr-6214",
|
||||
pr_number: 6214,
|
||||
head_sha: prHead,
|
||||
base_sha: prBase,
|
||||
run_id: 32105196624,
|
||||
marker: "<!-- merge-queue-ci-failure -->",
|
||||
});
|
||||
expect(workloadFailure.body).toContain("<!-- merge-queue-ci-failure -->");
|
||||
expect(workloadFailure.body).toContain(
|
||||
`[run ${runId}](https://github.com/nexu-io/open-design/actions/runs/${runId})`,
|
||||
);
|
||||
expect(workloadFailure.body).toContain(
|
||||
`- **Daemon tests (1/4)** — failure at \`Run daemon test shard\` ([job log](https://github.com/nexu-io/open-design/actions/runs/${runId}/job/101))`,
|
||||
);
|
||||
expect(workloadFailure.body).not.toContain("Daemon tests (2/4)");
|
||||
expect(workloadFailure.body).not.toContain("Validate workspace");
|
||||
expect(workloadFailure.body).toContain(
|
||||
"FAIL tests/project-archive.test.ts > buildProjectArchive > exposes a consumable stream with the same archive contents",
|
||||
);
|
||||
expect(workloadFailure.body).toContain("AssertionError: expected [ Array(6) ]");
|
||||
expect(workloadFailure.body).not.toContain("Process completed with exit code");
|
||||
expect(workloadFailure.body).not.toContain(esc);
|
||||
expect(workloadFailure.body).toContain("queued ahead of it (#6559)");
|
||||
expect(workloadFailure.body).toContain("add the PR back to the merge queue");
|
||||
|
||||
// A label-only ejection is already announced by the gate's notice: no second comment.
|
||||
const labelOnly = await runNotice({
|
||||
needs: {
|
||||
plan: { result: "success", outputs: { run: planRun } },
|
||||
merge_policy: { result: "failure", outputs: { ejection_notice: "true" } },
|
||||
static_gate: { result: "success" },
|
||||
daemon_unit_tests: { result: "success" },
|
||||
},
|
||||
labelNoticeEmitted: "true",
|
||||
labels: ["needs-validation"],
|
||||
mergePolicyConclusion: "failure",
|
||||
});
|
||||
expect(labelOnly.output.comment_created).toBeUndefined();
|
||||
expect(labelOnly.stdout).toContain("its own notice covers this ejection");
|
||||
|
||||
// A workload that failed alongside a blocking label is still reported, without repeating
|
||||
// the policy job — and the live label alone is enough to recognize the label ejection.
|
||||
const labelPlusWorkload = await runNotice({
|
||||
needs: {
|
||||
plan: { result: "success", outputs: { run: planRun } },
|
||||
merge_policy: { result: "failure", outputs: { ejection_notice: "" } },
|
||||
static_gate: { result: "success" },
|
||||
daemon_unit_tests: { result: "failure" },
|
||||
},
|
||||
labels: ["needs-maintainer-check"],
|
||||
mergePolicyConclusion: "failure",
|
||||
});
|
||||
expect(labelPlusWorkload.output.comment_created).toBe("true");
|
||||
expect(labelPlusWorkload.body).toContain("**Daemon tests (1/4)**");
|
||||
expect(labelPlusWorkload.body).not.toContain("Merge policy");
|
||||
|
||||
// A runner whose gh predates the escape-sequence policy must still get the excerpt: the
|
||||
// probe finds no flag and the plain fetch is used.
|
||||
const olderGh = await runNotice({
|
||||
needs: {
|
||||
plan: { result: "success", outputs: { run: planRun } },
|
||||
merge_policy: { result: "success", outputs: { ejection_notice: "" } },
|
||||
daemon_unit_tests: { result: "failure" },
|
||||
},
|
||||
ghSupportsEscapeFlag: false,
|
||||
});
|
||||
expect(olderGh.output.comment_created).toBe("true");
|
||||
expect(olderGh.body).toContain("FAIL tests/project-archive.test.ts");
|
||||
expect(olderGh.stdout).toContain("gh version 2.87.3 (fake)");
|
||||
});
|
||||
|
||||
it("[P1] routes configured contributors into an independent maintainer merge block", async () => {
|
||||
const [routingWorkflow, ciWorkflow, inactivityWorkflow] = await Promise.all([
|
||||
readFile(contributorMaintainerCheckWorkflowPath, "utf8"),
|
||||
|
||||
@@ -263,6 +263,14 @@ set, enforces merge policy at convergence, and is the only successful publisher
|
||||
of pending hash state. Runner allocation failure and external cancellation are
|
||||
operational failures rather than alternate coverage policy.
|
||||
|
||||
A merge-group failure at `Validate workspace` ejects the queued PR without any
|
||||
trace on the PR itself: the run executes on the queue's transient ref and the
|
||||
PR's own checks stay green. Two best-effort `handoff/comment` producers make the
|
||||
ejection visible through `comment.atom.yml`: `merge_policy` announces a
|
||||
blocking-label ejection, and `validate` announces every other failure (which
|
||||
jobs failed, a log excerpt, and the PRs batched ahead in the group) after the
|
||||
gate has already failed. Neither notice changes the gate result.
|
||||
|
||||
## Current references
|
||||
|
||||
These sections describe active or observed applications of the method. They are
|
||||
|
||||
Reference in New Issue
Block a user