fix: skip empty repositories instead of crashing#591
Conversation
check_optional_file() only caught UnknownObjectException, but an empty
repository (one with no commits) returns a 404 that PyGithub raises as
the base GithubException ("This repository is empty."). That escaped the
handler and crashed the whole run when iterating an organization that
contains a freshly-created, never-pushed repo.
Handle a 404 GithubException the same way as a missing optional file so
the repository is skipped. Non-404 GithubExceptions are re-raised
unchanged. Adds tests for both cases.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
This PR improves robustness when scanning GitHub organizations by preventing Evergreen from crashing on empty repositories (repositories with no commits) when checking for optional configuration files via PyGithub.
Changes:
- Extend
check_optional_file()to translate 404GithubException(empty repo case) intoOptionalFileNotFoundError. - Add unit tests covering the empty-repository 404 translation and ensuring non-404
GithubExceptions are still re-raised.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
exceptions.py |
Adds handling for base GithubException 404s (empty repos) to match existing optional-file behavior. |
test_exceptions.py |
Adds test coverage for empty-repository 404 behavior and non-404 re-raise behavior. |
| except GithubException as e: | ||
| # An empty repository (one with no commits) returns a 404 that PyGithub | ||
| # raises as the base GithubException ("This repository is empty.") | ||
| # rather than UnknownObjectException. Treat it the same as a missing | ||
| # optional file so the repository is skipped instead of crashing the run. | ||
| if e.status == 404: | ||
| raise OptionalFileNotFoundError( | ||
| status=e.status, data=e.data, headers=e.headers | ||
| ) from e |
There was a problem hiding this comment.
Good catch, thank you. You're right that hardening check_optional_file alone just moves the crash: an empty repo returns None there, the loop doesn't skip it, and it then hits repo.get_contents("/") / .github/workflows / .devcontainer in dependabot_file.py, which only catch UnknownObjectException.
I've pushed a follow-up that skips empty repos up front instead:
- Added an
is_empty_repo(repo)helper (repo.size == 0, no extra API call sincesizeis already on the listed repo object). - The main loop now skips empty repos right after the
archivedcheck, before anyget_contentslookup, so none of thedependabot_file.pypaths are reached for them. - Kept the
check_optional_fileGithubException404 handling as a defensive backstop. - Added unit tests for
is_empty_repo; full suite passes (184 passed).
This felt cleaner than broadening the exception handling at each get_contents call site, but happy to switch to a shared safe_get_contents-style helper if you'd prefer that centralization.
There was a problem hiding this comment.
I think the shared helper you floated at the end here is worth taking, and I dug up a reason to prefer it over the size == 0 skip. size is GitHub's cached disk-usage number (recalculated asynchronously, roughly hourly), so it isn't a reliable stand-in for "no commits." I confirmed against live repos: jsplumb/jsplumb is public, not archived, has a commit and a README.md, and still reports size: 0. The current skip would log it as "empty repository" and pass it over, so Dependabot never gets enabled there, and freshly pushed repos hit the same window until the cache refreshes.
A small helper that swallows a 404 from either UnknownObjectException or the base GithubException covers the truly-empty case at every call site without the false positive, so we could drop the size == 0 gate entirely:
def list_contents_or_empty(repo, path):
"""Return directory contents at path, or [] when the path is missing or the
repo is empty. A missing path 404s as UnknownObjectException; an empty repo
404s as the base GithubException. Both mean "nothing here"."""
try:
return repo.get_contents(path)
except GithubException as e:
if e.status == 404:
return []
raiseThen the three dependabot_file.py loops become for file in list_contents_or_empty(repo, "/"): (and the same for .github/workflows and .devcontainer), their try/except UnknownObjectException blocks go away, and GithubException gets added to the import on line 10. Happy to keep an early skip as well if we base it on a more reliable signal.
Addresses review feedback: the previous change only hardened check_optional_file(), but empty repositories also 404 in dependabot_file.py (repo.get_contents on "/", ".github/workflows", ".devcontainer"), which only catch UnknownObjectException and would still crash the run. Add an is_empty_repo() helper (repo.size == 0) and skip empty repos at the top of the main loop, before any content lookup, so none of the downstream get_contents paths are reached for them. The check_optional_file GithubException handling is kept as a defensive backstop. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
Appreciate you taking the time to work on a fix for this! |
Problem
When Evergreen iterates over an organization that contains an empty repository (one created but never pushed to, so it has no commits), the run crashes:
check_optional_file()inexceptions.pycallsrepo.get_contents(...)to look for an existingdependabot.yml. For a repo where the file merely does not exist, PyGithub raisesUnknownObjectException, which is caught and translated toOptionalFileNotFoundError. But for an empty repository, GitHub returns a 404 that PyGithub surfaces as the baseGithubException("This repository is empty."), notUnknownObjectException. That escapes the existing handler and aborts the whole run, so no other repositories in the org get processed.Fix
Handle a 404
GithubExceptionthe same way as a missing optional file, so the repository is treated as having no config and skipped by the normal flow. Non-404GithubExceptions (permissions, rate limits, server errors) are re-raised unchanged so genuine problems are not masked.Tests
Added two cases to
test_exceptions.py:GithubException) is translated toOptionalFileNotFoundErrorGithubException(e.g. 403) is re-raised unchangedAll existing tests still pass.