Skip to content

ci: keep apt installs working when security.ubuntu.com fails - #643

Merged
jescalan merged 1 commit into
mainfrom
je/ci-unzip-apt-resilience
Nov 20, 2026
Merged

jescalan merged 1 commit into
mainfrom
je/ci-unzip-apt-resilience

Conversation

@jescalan

@jescalan jescalan commented Nov 20, 2026 •

Copy link
Copy Markdown
Contributor

The Release run for 1a37542 failed three times in a row, each time before any E2E test ran (run 44654148595). The E2E job's first step, apt-get update && apt-get install -y unzip, exited 100 because security.ubuntu.com returned 502 Bad Gateway for noble-security/InRelease. apt-get update treats that as a hard error, and the && skips the install, even though unzip comes from the main archive, which fetched fine. While the mirror is flaky, nothing can ship.

Changes

  • E2E job, unzip install: skip when unzip is already present. Otherwise run apt-get update with Acquire::Retries=3 and carry on if it fails. Install unzip with the same retries, retrying the whole sequence up to three times with backoff.
  • Test job, zsh install: same treatment. It has the same failure mode and only got lucky on these runs.

How I tested it

I ran the old and new steps in mcr.microsoft.com/playwright:v1.63.0-noble, with security.ubuntu.com pointed at a local server that returns 502 and apt configured to treat that as an error, as CI did:

  • old step: E: Failed to fetch …noble-security/InRelease 502 Bad Gateway, exit 100
  • new step: same apt-get update error, then Setting up unzip (6.0-28ubuntu4.1), exit 0

Once this merges, the Release workflow on main should get past E2E and publish 4.4.0. That release includes the clerk deploy DMARC fix from #642.

🤖 Generated with Claude Code

The E2E job's `apt-get update && apt-get install -y unzip` step failed
every Release attempt for 1a37542 because security.ubuntu.com returned
502s, which makes `apt-get update` exit non-zero even though unzip comes
from the main archive. Tolerate a failed update, add apt retries, retry
the install, and skip it when the tool is already present. Apply the same
to the Test job's zsh install, which has the same failure mode.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@changeset-bot

changeset-bot Bot commented Nov 20, 2026

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: 6d19163

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@coderabbitai

coderabbitai Bot commented Nov 20, 2026 •

Copy link
Copy Markdown

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

The CI workflow checks for zsh in the test job and unzip in the E2E job before installing either package. If a package is missing, the workflow tolerates failed apt-get update calls and retries installation up to three times, with increasing waits after failed attempts. The workflow exits successfully if installation succeeds and fails after the third unsuccessful attempt.

Estimated code review effort: 2 (Simple) | ~10 minutes

Suggested reviewers: wyattjoh, shane-kercheval

Merge Risk: Minimal · up to 6d191

This change makes CI package installs more resilient when the security mirror is unavailable. It does not affect product behavior, and no actionable merge risk remains.

Pre-merge checks: Passed 4

Check name Status Explanation
Title check Passed The title names the main change: keeping apt installs working when security.ubuntu.com fails. The changeset adds retries, a tolerated apt-get update failure, and skip-if-present logic to the unzip and…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Description check Passed The description explains the problem (Release run for 1a37542 failed because apt-get update returned 502 from security.ubuntu.com, which blocked the unzip install), lists the changes to the E2E unzip …
Full details: Title check

Explanation

The title names the main change: keeping apt installs working when security.ubuntu.com fails. The changeset adds retries, a tolerated apt-get update failure, and skip-if-present logic to the unzip and zsh installs in ci.yml, so the title matches the change.

Full details: Description check

Explanation

The description explains the problem (Release run for 1a37542 failed because apt-get update returned 502 from security.ubuntu.com, which blocked the unzip install), lists the changes to the E2E unzip install and the Test job zsh install, and describes how the change was tested. It is specific to the changeset and matches the raw summary of ci.yml edits.

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Warning

Linked repositories: This review skipped 6 repositories. Cross-repository analysis may be incomplete. Check linked repository access and settings.

Recent review info

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: ASSERTIVE
  • Plan: Team
  • Run ID: 5feb174a-d907-4389-96cc-eca3359cac3f
Commits

Reviewing files that changed from the base of the PR and between 1a37542 and 6d19163.

Files selected for processing: 1

  • .github/workflows/ci.yml
Linked repositories identified

CodeRabbit considers these linked repositories for cross-repo context during reviews:

Included review availability: This review used your included allowance. 0 included reviews remain after this review. Your included PR review attempts over the past 7 days set your current allowance at 3 reviews per hour.

@jescalan
jescalan requested a review from wyattjoh November 20, 2026 01:49
@jescalan

Copy link
Copy Markdown
Contributor Author

I'm power merging this because I need this dmarc change to go out asap

@jescalan
jescalan enabled auto-merge (squash) November 20, 2026 02:07
@jescalan
jescalan merged commit 8c69ac5 into main Nov 20, 2026
11 checks passed
@jescalan
jescalan deleted the je/ci-unzip-apt-resilience branch November 20, 2026 02:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants