Skip to content

fix: support pnpr keeping its registries in the database - #84

Merged
zkochan merged 1 commit into
mainfrom
fix/pnpr-registries-in-database
Nov 13, 2026
Merged

zkochan merged 1 commit into
mainfrom
fix/pnpr-registries-in-database

Conversation

@zkochan

@zkochan zkochan commented Nov 13, 2026 •

Copy link
Copy Markdown
Member

The scheduled benchmark fails with the latest pnpr (0.1.0-alpha.20) and pnpm (12.21.0): run 43509974464 stopped in populate-registry-cache with

parse pnpr.yaml: Invalid config: unknown field `file`

alpha.20 changed several things the benchmark relied on:

  • Registries are read only from the database. registries, defaultRegistry and routes.public in pnpr.yaml are now config errors. They are written to a separate registries.yaml and stored with pnpr registries import before every start; the import replaces whatever the seeded storage carried.
  • auth.users.file is gone. Accounts live in the database, which pnpr keeps inside storage and binds the storage to. A storage seeded by populatePnprCache.js therefore arrives with the benchmark account, and mintToken's PUT signs in to it instead of registering.
  • A secret is required to reuse a cache directory. A fixed one is set, so every sample job can reuse the populated storage.
  • The resolver may reach the public URL's host only at the public URL's origin. The accelerated scenario has the resolver fetch from the server's un-proxied address, while the public URL is the latency proxy, and both were on 127.0.0.1. Spelling it localhost doesn't help, because the resolver looks it up and refuses it anyway. pnpr now listens on 127.0.0.2, the latency proxies forward to it, and clients keep using 127.0.0.1. Loopback is allowed through routes.allowedPrivateNetworks. This relies on all of 127.0.0.0/8 being loopback, which is true on Linux, where the benchmark runs.
  • listen and publicUrl are now set in the config file rather than as flags, so the offline registries import checks routes against the same addresses the server uses.

Testing

  • pnpm run lint and pnpm test pass.
  • Local smoke test with pnpr 0.1.0-alpha.20 and pnpm 12.21.0, using startPnpr, mintToken, the latency proxies and verifyResolverIsUsed on a small project that includes a @teambit package:
    • fresh storage: the resolver is used;
    • restart on a copied, already-populated storage: the account signs in again, the resolver is used, and packages are served from the cache.
  • The full workflow has not been run yet.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Improvements
    • Benchmark network routing now supports upstream hosts beyond the default local address, while registry and resolver traffic use a dedicated local address.
    • Registry setup now imports its sources and routes separately, with cache and private-network settings configured for the benchmark environment.
    • Port reservations now target the relevant host, helping keep benchmark services and their network configuration aligned.

pnpr 0.1.0-alpha.20 reads its registries only from its database and refuses
a config file that declares them, drops `auth.users.file`, and no longer
lets its resolver reach the public URL's host at another origin.

- Store the registries with `pnpr registries import` before each start.
- Drop the users file: accounts live in the database, inside storage, so a
  seeded storage arrives with the account and `mintToken` signs in to it.
- Set a fixed `secret`, which pnpr requires to reuse a cache directory.
- Run pnpr on 127.0.0.2 so the resolver can reach its un-proxied address
  while clients and the latency proxies stay on 127.0.0.1, and allow
  loopback through `routes.allowedPrivateNetworks`.
- Pass `listen` and `publicUrl` in the config so the import checks routes
  against the same addresses the server uses.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Nov 13, 2026 •

Copy link
Copy Markdown

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

pnpr now listens on 127.0.0.2 and imports registry definitions before startup. Latency proxies accept a configurable upstream host, and benchmark setup directs them to the pnpr host.

Changes: pnpr Host and Proxy Routing

Layer / File(s) Summary
pnpr host and startup configuration
pnprServer.js
Port reservation and pnpr startup use PNPR_HOST. Startup writes server settings, imports registry definitions from a separate file, and removes the old listen and public-URL command-line options.
Latency proxy upstream routing
latencyProxy.js, benchmarkRegistry.js, populatePnprCache.js
Latency proxies accept an upstream host. Benchmark registry and cache setup reserve ports for PNPR_HOST and direct proxies to that host.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Merge Risk: Minimal · up to e53b7

This change updates the benchmark for the new pnpr and pnpm releases. pnpr now listens on a dedicated loopback address, and registries are imported before startup, which addresses the scheduled benchmark failure. No actionable merge-blocking risk was found. Running the full benchmark workflow once after merge is the sensible final confirmation.

Pre-merge checks: Passed 4

Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title "fix: support pnpr keeping its registries in the database" matches a real part of the changeset. The pnpr server config now writes registry definitions to a separate YAML file and imports th…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Title check

Explanation

The title "fix: support pnpr keeping its registries in the database" matches a real part of the changeset. The pnpr server config now writes registry definitions to a separate YAML file and imports them into pnpr's database before startup, and the objectives describe database-backed storage for registries and accounts. The title does not mention the other changes, such as the new PNPR_HOST address (127.0.0.2), the latency proxy upstream host option, or the removal of the old listen and public-URL options. Those are secondary to the main change, so the title is a fair summary.

Finishing Touches

Generate unit tests (beta)

  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

A rabbit checks the host address,
Then hops to where the registries rest.
The proxy follows, neat and clear,
Through pnpr paths from ear to ear.
It nibbles logs and bounds away.

Recent review info

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 23d741c2-c36f-451e-bb41-2a0cc7463fcf
Commits

Reviewing files that changed from the base of the PR and between 6fd573a and e53b723.

Files selected for processing: 4

  • benchmarkRegistry.js
  • latencyProxy.js
  • pnprServer.js
  • populatePnprCache.js

Included review availability: This review used your included allowance. Your plan provides up to 2 included reviews per hour; 1 remain after this review.

Recent review details

Context from checks skipped due to timeout: 1

  • GitHub Check: CodeRabbit
Additional context used

ast-grep (0.51.1)

pnprServer.js

[warning] 122-158: Filesystem path is not a string literal; a request-/variable-derived path can enable path traversal. Validate and normalize the path before use.
Context: fs.writeFileSync(configPath, [
listen: ${PNPR_HOST}:${port},
// pnpr rewrites the tarball URLs it serves to this address. It has to be
// the address clients actually use, or every tarball would be fetched on a
// link the benchmark isn't emulating — and npm refuses outright to fetch a
// tarball from a host other than the registry it was told about. Set here
// rather than on the command line because the registries import below
// checks its routes against it too.
...(publicUrl ? [publicUrl: ${publicUrl}] : []),
// Accounts and the registries live in the database, which pnpr keeps
// inside the storage directory and binds the storage to. A storage
// populatePnprCache.js filled therefore arrives with the populate run's
// account and registries: the account is signed in to again (see
// mintToken) and the registries are replaced below.
storage: ${path.join(dir, 'storage')},
// Upstream content is cached under keys derived from this, and pnpr
// refuses to reuse a cache directory without one. The benchmark hands
// every run the storage populatePnprCache.js filled, so it has to be
// the same in all of them. Nothing here is secret: the server only ever
// listens on loopback.
'secret: pnpm-benchmarks-pnpr-cache-secret',
'auth:',
' users:',
// The benchmark registers its own resolver user on startup.
' maxUsers: 1',
'resolver:',
' enabled: true',
'routes:',
// The tar registry the accelerated scenario resolves against is this
// server's own, at its un-proxied address (see applyPnprServerRegistry),
// and the resolver refuses loopback addresses that are not listed here.
' allowedPrivateNetworks: [127.0.0.0/8]',
'log:',
' format: pretty',
level: ${logLevel},
'',
].join('\n'))
Note: [CWE-22] Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal').

(detect-non-literal-fs-filename)

[warning] 228-275: Filesystem path is not a string literal; a request-/variable-derived path can enable path traversal. Validate and normalize the path before use.
Context: fs.writeFileSync(registriesPath, [
'registries:',
// The public npm registry. No packages: map, so it claims every
// name — the catch-all, and therefore last in the router's sources.
' npmjs:',
' type: upstream',
' url: https://registry.npmjs.org/',
' public: true',
// Bit publishes to a registry of its own, and alotta-packages
// depends on it. Declaring the scopes it serves is all the routing
// that is needed: every manager keeps installing from the one
// registry URL the benchmark hands it, and the names below are the
// only ones fetched from bit. Tarball URLs come back rewritten to
// pnpr's own address, so bit's packages cross the emulated link
// like every other package.
' bit:',
' type: upstream',
url: ${BIT_REGISTRY},
' public: true',
' packages:',
...BIT_SCOPES.map((scope) => '${scope}/*':),
// One URL in front of both: a router resolves a name to the first
// source whose declared packages claim it. defaultRegistry has to
// name this rather than a concrete registry — a concrete one
// resolves only its own namespace and answers Unclaimed for
// everything else, with no fall-through to a sibling.
' main:',
' type: router',
' sources: [bit, npmjs]',
// The same sources again, served as tars rather than .tgz files (see
// tarRegistryUrl). One storage behind both: a package pulled through
// either is cached once.
${TAR_REGISTRY}:,
' type: router',
' sources: [bit, npmjs]',
' tarballs: tar',
'defaultRegistry: main',
// The accelerated scenario asks the server to resolve against this
// address — its own un-proxied tar registry — so the resolver's metadata fetches
// stay on loopback instead of crossing the emulated link and coming back
// in through the front door (see applyPnprServerRegistry). The server
// resolves against whatever registry the client sends but only if that
// registry is declared, so the address has to be listed as a route here.
'routes:',
' public:',
- registry: ${tarRegistryUrl(url)},
'',
].join('\n'))
Note: [CWE-22] Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal').

(detect-non-literal-fs-filename)

Additional comments: 4

pnprServer.js: 1

22-26: LGTM!

Also applies to: 75-89, 123-160, 221-286, 324-326

latencyProxy.js: 1

146-146: LGTM!

Also applies to: 160-160, 202-209, 272-276

benchmarkRegistry.js: 1

12-12: LGTM!

Also applies to: 101-103, 120-120

populatePnprCache.js: 1

10-10: LGTM!

Also applies to: 55-55

@qodo-code-review

Copy link
Copy Markdown

PR Summary by Qodo

Fix pnpr benchmarks for database-backed registries

🐞 Bug fix 🕐 20-40 Minutes

Grey Divider

AI Description

• Import registries into pnpr’s database before startup, replacing routes inherited from seeded
 storage.
• Reuse populated storage with database-backed accounts and a consistent cache secret.
• Separate pnpr’s loopback address from proxied client traffic to preserve server-side resolution.
Diagram

sequenceDiagram
    participant Seed as Cache Seed
    participant Bench as Benchmark
    participant CLI as pnpr CLI
    participant DB as Storage DB
    participant Server as pnpr Server
    participant Proxy as Latency Proxy
    actor Client as Package Manager
    Seed-->>Bench: populated storage
    Bench->>DB: copy seeded storage
    Bench->>CLI: import registries.yaml
    CLI->>DB: replace registry routes
    Bench->>Server: start on 127.0.0.2
    Server->>DB: load routes and account
    Bench->>Proxy: forward to 127.0.0.2
    Client->>Proxy: registry and resolver requests
    Proxy->>Server: forward traffic
    Server->>Server: resolve via direct address
    Server-->>Proxy: respond with public URLs
    Proxy-->>Client: return responses
Loading
High-Level Assessment

Use pnpr’s supported registry import and database-backed account storage. A separate loopback address preserves the existing latency model while satisfying the resolver’s origin restriction; changing the hostname alone would not.

Files changed (4) +124 / -62

Bug fix (4) +124 / -62
benchmarkRegistry.jsRoute benchmark proxies to pnpr’s dedicated loopback address +4/-1

Route benchmark proxies to pnpr’s dedicated loopback address

• Reserves pnpr’s port on its new host and points both the registry and resolver latency proxies there. Client-facing proxy addresses remain unchanged.

benchmarkRegistry.js

latencyProxy.jsAllow configurable proxy upstream hosts +7/-4

Allow configurable proxy upstream hosts

• Passes an upstream host to the proxy subprocess and uses it for outbound connections. Existing callers retain the 127.0.0.1 default.

latencyProxy.js

pnprServer.jsImport registry routes before starting database-backed pnpr +111/-55

Import registry routes before starting database-backed pnpr

• Moves registry definitions into a separate import file and replaces stored routes before each start. Configures the server’s address, public URL, fixed cache secret, and permitted loopback network for reused storage and direct resolver access; removes the obsolete users-file setting.

pnprServer.js

populatePnprCache.jsSeed the cache using pnpr’s dedicated host +2/-2

Seed the cache using pnpr’s dedicated host

• Reserves the cache-population server’s port on the same loopback address used by pnpr startup.

populatePnprCache.js

@zkochan
zkochan merged commit 9a2d524 into main Nov 13, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant