diff --git a/REVIEW.md b/REVIEW.md index 820b81a011..88725ee0cc 100644 --- a/REVIEW.md +++ b/REVIEW.md @@ -38,12 +38,26 @@ through logs and errors, then everything else. compromised admin session cannot change it. Reject changes that relocate it. - `AdminSettings` is the only writer of the authoritative `AdminConfig`. Other code reads through `readAdminConfig(env)`, except `AdminApiImpl`, which deliberately reads the authoritative config - for the audit log's retention in `listAdminAudit()` and `getAdminAuditStats()`. The - admin-settings conventions are in `packages/workshop-backend/AGENTS.md`. -- `ADMIN_AUDIT_RETENTION_MIN_DAYS` is env-driven, like auth config. Flag code that enforces or - prunes with `adminAuditRetentionDays` without `adminAccessPolicy()`. + in two places: `inspectWorkspace()`, so that a KV mirror still holding a higher access level never + mints a view at it, and the audit log's retention in `listAdminAudit()` and + `getAdminAuditStats()`. The admin-settings conventions are in `packages/workshop-backend/AGENTS.md`. +- `OverseerDurableObject.openForAdmin()` stays reachable only over the Durable Object binding, never + through a capability a user holds, and refuses an uninitialised workspace without writing. Every + `AdminWorkspaceView` method runs through `#call()`, which checks the view again once it has read + the deployment's setting and once it has read its answer, with nothing awaited before its record + is sent; never reaches a connection (admin file reads never pull); and records the call, with what + its answer exposed, in the deployment-wide log before anything it read leaves the server, so that + a call whose record cannot be written returns nothing. The record follows the read because + `exposed` must describe the answer actually returned. Flag a path that returns workspace data to + an admin without these, or that sets `exposed` from anything but that answer. +- `ADMIN_WORKSPACE_ACCESS_MAX` and `ADMIN_AUDIT_RETENTION_MIN_DAYS` are env-driven, like auth + config, and the cap fails closed. Flag code that enforces `adminWorkspaceAccess`, or enforces or + prunes with `adminAuditRetentionDays`, without `adminAccessPolicy()`. - A new `AdminSettings` mutator takes an `AdminSettingActor` and records through `#mutateAdminConfig()`, or records explicitly before any other write. +- Text a user or workspace member controls (display names, titles) enters the directory or the + audit log only bounded: clamped to `MAX_DIRECTORY_TEXT_LENGTH`, or cut by `cutAdminAuditText()`. + Flag a new copy of such text into either store without a bound. - `cf-os api` sends `User-Agent: cf-os-api/`, and except on a local instance (`WORKSHOP_ENVIRONMENT` `development` or `test`, set only by `pnpm dev-server` and the integration harness) the Workshop lets such a session act only as a deployment admin. That is a speed bump diff --git a/docs/admin-access.md b/docs/admin-access.md index a460a8625f..e88962f8d7 100644 --- a/docs/admin-access.md +++ b/docs/admin-access.md @@ -1,85 +1,487 @@ -# Admin access: the admin audit log +# Admin access: operating and investigating a deployment -A deployment's admins (the canonical user ids its backend Worker's `ADMINS` variable lists) change -its settings through the admin panel and the API. This document is the design record for the -deployment-wide log of those changes, which says afterwards who changed which setting, and when. -The API itself is documented on `AdminApi` in `packages/workshop-shared/src/api.ts`; the pieces -are: +A deployment's admins (the canonical user ids its backend Worker's `ADMINS` variable lists) need to +operate, debug and investigate the instance: find a user or a workspace, look inside a workspace +that misbehaves, and know afterwards who looked at what and who changed which setting. This +document is the design record for that surface. The API itself is documented on `AdminApi` and +`AdminWorkspaceView` in `packages/workshop-shared/src/api.ts`; the pieces are: | Concern | Location | | --- | --- | | Admin status | `packages/workshop-backend/src/auth/config.ts` | -| The environment's bound, validation, the audit's text bounds | `packages/workshop-backend/src/admin-access.ts` | +| The environment's bounds, validation, the audit's text bounds | `packages/workshop-backend/src/admin-access.ts` | | The `AdminApi` capability (`AdminApiImpl`) and the settings audit (`AdminSettings`) | `packages/workshop-backend/src/admin-settings.ts` | | Diffing a settings change (`diffAdminConfig`) | `packages/workshop-backend/src/admin-config.ts` | | The deployment-wide audit log (`AdminAuditDurableObject`) | `packages/workshop-backend/src/admin-audit.ts` | +| The user and workspace directory (`UserDirectoryDurableObject`) | `packages/workshop-backend/src/user-directory.ts` | +| Inspection (`openForAdmin`, `AdminWorkspaceViewImpl`) and the workspace's directory feed | `packages/workshop-backend/src/overseer.ts` | +| Reads that never pull (`readLocalCommitTree`, `readLocalFileAtCommit`, `readLocalFileAtCommitIfExists`, `localPathEntryAtCommit`, `changedLocalFilePathsBetween`, `readCommitLog` with `stopAtMissing`) | `packages/workshop-backend/src/git-cache.ts` | | The admin panel's Audit tab | `packages/workshop-frontend/src/features/admin-audit/` | ## Entry points Everything goes through `AuthenticatedApi.getAdminApi()`, which returns null unless the -caller is an admin. The admin check runs once, when the capability is minted. +caller is an admin. The admin check runs once, when the capability is minted, for every `AdminApi` +method except workspace inspection, which checks again (see [Inspection](#inspection)). - **Settings**: the setters (signups, user search, site name and logo, instructions, resource and gatekeeper availability, models, update checks, ...), plus - `setAdminAuditRetentionDays()`. Every setter is audited. + `setAdminWorkspaceAccess()` and `setAdminAuditRetentionDays()`. Every setter is audited. +- **The directory**: `listUsers()` and `listWorkspaces()`. +- **Inspection**: `inspectWorkspace(workspaceId, reason, { lifetimeSeconds, access })`, which + returns an `AdminWorkspaceView` at the lower of the level asked for (`"metadata"` by default) and + the deployment's. - **The audit log**: `listAdminAudit()` and `getAdminAuditStats()`. Each record's `via` (`AdminVia`) comes from the session's User-Agent: `"cli"` when the `/api` upgrade names `cf-os api` (`cf-os-api/`), `"api"` for any other client, the Workshop's admin panel included. A client can send any User-Agent, so it labels records and grants nothing. -Each capability mints a random `sessionId`, so every record made through one `getAdminApi()` call -shares its `via` and `sessionId` and can be read as one session. +Each capability mints a random `sessionId`, so every record made through one `getAdminApi()` call, +settings and inspections alike, shares its `via` and `sessionId` and can be read as one session. + +## The directory + +### One directory, for search and for admins + +`listUsers()` and `listWorkspaces()` read `UserDirectoryDurableObject`, the directory behind user +search (`AuthenticatedApi.searchUsers()`). The admin listings keep what search does not need in +tables of their own, `user_activity` and `workspaces`, beside search's `users` table, rather than in +a second store, because this directory is everything an admin directory needs: the deployment-wide +singleton (always `getByName("")`), fed by every user DO through a revision protocol that backfills +lazily, migrated, and bound in the unit test pool. A parallel store would duplicate the users +table, the sync protocol and the per-session writes. + +User search keeps its own matching and its `userSearchEnabled` gate; the admin listings ignore that +setting. Directory listings are unrecorded reads, visible whatever the access level, workspace +titles included: they are what an admin uses to find a workspace before inspecting it. + +### Feeds + +Durable Objects cannot be enumerated, so the directory learns of accounts and workspaces from the +objects themselves: + +- **The user DO** reports its profile, by revision, when a session starts (a session token + is authenticated, or a Cloudflare Access sign-in), and when the user renames themselves. A session + start also reports the session's time as `lastLogin`, at most once per + `DIRECTORY_ACTIVITY_INTERVAL_MS` (an hour, the precision `lastLogin` promises), and every report + carries the account's creation time once it is recorded. An account records it when it is + created, so it is listed with it from its first session. +- **The Overseer** reports its workspace's title, owner and creation time: + - on every `open()`, a no-op once the directory has acknowledged the title's revision; + - when the title changes, whoever changes it (a rename, an agent-chosen title, initialisation + from a blueprint or an external message): the `title` singleton is the one chokepoint, and its + subscriber bumps the revision inside the write's transaction and defers the sync; + - with `lastActive` when the workspace is active, at most hourly, alongside its + last-active report to the owner's user DO. + + Syncs run one at a time, are best-effort, and never restart the workspace. A workspace + initialised before it recorded its creation time reads it once from its owner's record of it + (`GadgetRecord.created`) and stores it. +- **The revision is time-based.** `#initializeNewWorkspace()` sets it to at least the current time. + An external message names its workspace by key, so a deleted workspace can be initialised again + under the same id; its new life then outranks whatever its earlier one left in the directory. + +### Deletion + +`deleteSelf()` takes the workspace out of the directory (`removeFromDirectory()`) once +`beginDeletion()` has committed the deletion tombstone, which clears the owner and so stops any new +sync. `removeFromDirectory()` marks the instance removed (so no queued sync sends or acknowledges +anything), waits for the syncs in flight, and removes the entry. A failed removal is logged and +leaves the entry listed. + +### What the lazy backfill misses + +- An account is listed once it has started a session since the directory began recording + accounts for user search, whatever `userSearchEnabled` says. Most accounts are therefore listed + as soon as the admin listings ship, but without `lastLogin` until their next session. Only an account with no session since then waits for its next one, and is never listed + if it never starts another. +- A workspace never opened again is never listed. +- An account created before the deployment recorded creation times is listed without `created`. +- A workspace's `lastActive` is absent until it is next active. +- A workspace whose removal from the directory failed stays listed until an admin inspects it: + inspection finds it gone (`workspaceNotFound`) and removes the entry. There is no tombstone. -## Retention and its bound - -### The environment's bound +### Text bounds -A backend variable bounds what admins may choose. It is an environment variable for the reason -authentication config is (`auth/config.ts`): a compromised admin session must not be able to -shorten the log. +Nothing upstream bounds a display name or a workspace title, so the directory keeps the first +`MAX_DIRECTORY_TEXT_LENGTH` (1000) characters of each, both where it stores them and where it reads +them (a row stored before it clamped may hold more). The clamp has no marker: the directory is a +finding aid, not a record, and clamps as `boundAgentCatalog()` clamps a catalog. Search text is built +from the clamped name or title and the first 1000 characters of each id. + +Ids are identities and filter keys, so they are kept whole, and a username has no length limit +either. A page therefore stops at a million characters of text as well as at its `limit` (and is +never empty while rows remain), and cursors and owner filters take any length. + +## Inspection + +`AdminApi.inspectWorkspace()` validates the reason, the lifetime and the level asked for, +re-checks that the caller is an admin, reads the access policy from the authoritative +`AdminSettings` config (not the KV mirror: a mirror still saying `"full"` after an admin lowered the +level must not mint a view at the old one), and then calls +`OverseerDurableObject.openForAdmin(adminId, reason, sessionId, terms)`. The terms carry the level +asked for (`requested`) and the view's own `access`, the lower of that and the authoritative policy. + +`openForAdmin()`: + +- **is reachable only over the Workshop's Durable Object binding**, never through a capability a + user holds; +- **refuses a workspace never initialised or since deleted** (`workspaceNotFound`) before anything + else, writing nothing. Unlike `open()`'s first-open path it never asks a user DO whether the + workspace is theirs, so it never claims ownership; +- **skips `authorizeCollaborator()` and observer verification by explicit policy**: the admin is not + a collaborator and is never registered with the workspace's gatekeepers. This is a deliberate + exception to the observer invariant, compensated by the audit (see + [observers.md](observers.md#admin-access-a-deliberate-exception)); +- reads its environment only through the `OverseerImpl` (`impl.env`), never through the Durable + Object's own `env`; +- rejects malformed terms with a `TypeError`, unrecorded: a blank or over-long reason, a `via` that + is not `"api"` or `"cli"`, an `access` or a `requested` that is not `"metadata"` or `"full"`, an + `access` above `requested`, and an expiry that is not a valid time or lies beyond the longest + lifetime. An expiry already past when the workspace answers is not malformed: the open is refused + as any call on an expired view is (`viewExpired`, unrecorded), so a slow hop never surfaces as a + validation error. + +The view (`AdminWorkspaceViewImpl`) is not an `Overseer`. It joins no presence, session count or +outputs fan-out, and starts no agent, ambient capsule or gatekeeper facet; waking the Durable Object +still resumes turns a previous instance left running, as any wake does. It offers these reads, +shaped as the matching `Overseer` reads are where there is one: + +- **The workspace**: `describe()` and `getDiagnostics()`, whose counts come with the size of the + workspace's storage and who has it open. +- **Chats**: `listChats()`, `getChat()`, `getChatState()`, `listChatAttachments()` and + `readChatAttachment()`. `getChatState()` is the state behind a chat that looks stuck: the agent + turn recorded for it, running or paused, when its newest entry was written, the calls queued for + its agent, and the actions it submitted that still wait for approval. +- **Actions**: `listActions()`. +- **Code**: `listFiles()` and `readFile()` read a gadget's committed head. `getCommitLog()`, + `listFilesAtCommit()`, `readFileAtCommit()` and `listChangedPaths()` read a gadget's history and + any version of its code. `listPendingChanges()` and `readPendingFile()` read what a chat changed + on top of each pin's base and has not had accepted. +- **Connections and sharing**: `listGatekeepers()`, `listObservers()`, + `listObserverRequirements()`, `listShareLinks()`, `listAutoApprovals()` and `listBlueprints()`. + `listObservers()` lists which collaborators hold an observer record, and the account each chose + for each connection. `listObserverRequirements()` lists what a recipient of a share would have + to verify: the connections that sharing the workspace, or one gadget, asks them to pass. + +Three of these give less than their names promise: + +- **`listObservers()` lists who holds a record, not who was admitted.** An observer record is + reserved at a collaborator's first admission, before anything is verified, and outlives a failed + one. A record with no account choice has nothing verified yet. +- **`listPendingChanges()` is not always all that accepting would change.** It compares a chat's + content with each pin's `baseCommit`, while accepting compares it with the pin's `mergedCommit`. + Where the two differ (a pin rooted at a merge commit, after an update from mainline or for a + proposed blueprint merge), `listChangedPaths(mergedCommit, baseCommit)` is the rest of what + accepting would change, and the listed paths can be empty while there is a change to accept. The + one entry it lists with no base is a gadget the chat created and has not had accepted (a + worktree is pinned at its first modification, so it always has one). +- **`listAutoApprovals()` lists opt-ins, which apply nothing alone.** They are the rules members + turned on. An action is applied without a manual approval only when its author also marked it + `autoApprovable`, and never once the workspace has read restricted data: the rules stay stored + and listed, and every action waits. A view shows both conditions, which is how an admin answers + why an action still waited: `describe()` reports `containsRestrictedData`, and `listActions()` + keeps each action's `autoApprovable` verdict and kind at every level, with `autoApproved` on one + a rule applied. + +An admin read never reaches a connection, and reads only what the workspace stores: + +- **Commits.** `listFiles()` and `readFile()` read a permanent gadget's committed head. + `listFilesAtCommit()`, `readFileAtCommit()`, `getCommitLog()` and `listChangedPaths()` take any + commit the workspace stores (a gadget's history, a worktree's commits, a chat pin's base) and + refuse anything else. All of them read only stored objects, through the local readers of + `git-cache.ts`: `readLocalCommitTree`, `readLocalFileAtCommit` and + `changedLocalFilePathsBetween`, built on `readLocalObject`, and `readCommitLog`, which reads only + the commits the store holds. The ordinary reads pull a missing object through the workspace's + connections; these never do. A missing object is an error, not a fetch. `getCommitLog()` stops + at a parent that is not stored (`stopAtMissing`), which is how a worktree checked out without + its history lists. +- **Commits a connection supplied.** A stored commit can be one a connection supplied. A worktree + checked out through a connection holds that repository's commits + (`WorktreeRecord.sourceGatekeeperId` names the connection; a worktree made from a commit that no + connection is recorded as a source of, such as one in a gadget's history that was never pushed, + names none), and the workspace stores what it pulled. The commit reads treat those commits as + any other: at `"metadata"` they show the repository's file names, commit ids, authors and times, + and the paths that differ between two of its commits; at `"full"`, its files and commit + messages. The connection is never asked, so the admin is not verified against the repository, + and reads no more of it than the workspace already holds. This is the part of the + [admin exception](observers.md#admin-access-a-deliberate-exception) that these reads widen. +- **A chat's pending content.** `listPendingChanges()` and `readPendingFile()` use the fold the + members' sessions use (`getCurrentChatContent()`), given a base reader that reads stored objects + only (`readLocalFileAtCommitIfExists`), where the sessions' reader pulls. The fold's cache stays + valid whichever reader filled it, because a commit is immutable: a reader that succeeds returns + the same text, and one that cannot read a base throws. `listPendingChanges()` tells a worktree's + paths apart by what its base holds at each (`localPathEntryAtCommit`), reading no blob. +- **Attachments.** An attachment is read from its record's own bytes or its stored blob, and a + legacy one from the file at its stored commit. +- **What a connection could answer** is read from the workspace's records instead: observer + requirements from the connections' records and the action log's index, auto-approvals from the + stored rules (never `listPreApprovableActions()`, which asks each connection), and connections + from their records. + +Beyond the workspace's own storage, a view reads only its owner's profile, from their account: in +`describe()`, and in `listShareLinks()`, which uses the same lookup and only when the owner created +a link. + +### Who created a connection + +A connection records its creator (`GatekeeperRecord.createdBy`): the member whose session made it +(`newGatekeeper()`, `newAiModelGatekeeper()`, `newAgentSpawnerGatekeeper()`), or the initiator of +the agent turn whose `createExternalResource` call created its resource. `listGatekeepers()` shows +the creator at every level. + +Two kinds of connection have none: + +- **An ambient capsule**, because no person created it. A workspace installs one on open for each + singleton account its owner holds (the Context Library's, for one), so it comes from the owner's + own account of that vendor, and a workspace whose owner has none has no such connection. +- **A connection whose creator was not recorded**: one written before creators were recorded, or a + resource created in a turn that has no record. + +### The level an inspection asks for + +An inspection asks for `"metadata"` (the default) or `"full"` (`AdminInspectOptions.access`), and +its view gets the lower of that and the deployment's effective level. +Asking for `"full"` where the deployment allows only `"metadata"` is not an error: the view opens at +`"metadata"`, and its `describe()` reports the level granted. While the effective level is `"off"`, +every open is refused, whatever it asks for. Any other level is refused with a `TypeError` before +the open runs, by `AdminApiImpl`'s RPC validation. + +Every record the view makes keeps the level asked for (`requested`) beside the level the call ran +at (`access`). Reading content is therefore a separate step: an investigation starts at +`"metadata"`, and moves to `"full"` deliberately, in a new `inspectWorkspace()` call with a reason +of its own, and the log shows when it did. + +A view never rises. Its level is fixed when it is minted, and later calls only lower it (see +[Each call](#each-call)): raising the setting leaves a view that asked for `"full"` but got +`"metadata"` at `"metadata"`, and the admin inspects again to read content. `openForAdmin()` rejects +terms whose `access` is above `requested`, so no caller can mint a view above what it asked for. + +### Each call + +Every view method, the open included, runs through `#call()`: + +1. The view's checks: it has not expired (`viewExpired`), its admin is still an admin (`notAdmin`), + and the workspace still exists (`workspaceNotFound`). A call these stop is not recorded. +2. The deployment's setting is read (`readAdminConfig`, the KV mirror). +3. The checks again, since that read may have spanned the expiry, a demotion or the workspace's + deletion. +4. The level the call runs at: the view's own (the lower of `requested` and the deployment's level + when it was minted), lowered to the current setting and the environment's cap. Lowering the + setting reaches live views within about a minute (the KV mirror's propagation); raising it never + raises a view. +5. A call that level refuses is recorded, exposing `"metadata"` and with `refused` set, and then + refused without reading anything: `accessOff` once the level is `"off"`, `contentWithheld` below + `"full"` for a method that reads only content (`getChat()`, `readChatAttachment()`, + `readFile()`, `readFileAtCommit()`, `readPendingFile()`). Its record is sent with nothing + awaited after step 3's check. +6. The read, on the server and never through a connection. A read that throws is recorded as an + attempt exposing `"metadata"`, once the checks pass again, and the call fails with the read's + error: a missing object, a gadget whose code is pending, an invalid path, a commit that is not + stored, a chat that does not exist. `listObserverRequirements()` throws, as the members' call + does, for a legacy connection (one made before connections could verify collaborators) in the + scope of the grant it was asked about: any such connection for `"workspace"`, and for a gadget + only one the gadget reaches. A malformed string argument (a commit id, a path, an attachment + id) is checked inside the read, so it is recorded this way. A number out of range is not: + `getCommitLog()`'s `depth` outside 1 to `MAX_ADMIN_COMMIT_LOG_DEPTH`, like `listActions()`'s + invalid `beforeId`, is rejected before the call starts, and is not recorded. +7. The checks again, with nothing awaited before the record is sent, since the read may have + awaited. A call that expiry, a demotion or the workspace's deletion overtakes while it reads + writes nothing, appends nothing for the workspace and returns nothing. +8. What the call exposed (`exposed`), computed from the answer it read (see + [What a call exposed](#what-a-call-exposed)). +9. The record, in the deployment-wide log. +10. The answer. + +The record follows the read because it says what the call returned, which only the answer can +tell. The point a record must precede is the answer leaving the server, since that is what an admin +sees. Holding the answer until its record exists means that nothing reaches an admin unrecorded and that a call whose record +cannot be written returns nothing, and a call its level refuses is still recorded without reading +anything. An answer that never left the server, because its record failed or the view stopped +meanwhile, exposed nothing. No check follows the append: the answer was read while the view was +live, and its record exists. + +An answer can be a `stream`: a file over `MAX_BUFFERED_BLOB_SIZE` (1 MiB) from `readFile()` or +`readFileAtCommit()`, and from `readPendingFile()` a pending text that large or any file a chat +holds as bytes. Step 10 returns the stream, not its bytes. They are read as the caller pulls them, +after the record: from the stored blob, or, for a pending text, from a copy of its UTF-8 encoding +that the answer holds until the stream ends. The record precedes the answer, and an answer in +flight is not another use of the view, so: + +- **The stream is part of the recorded answer.** Its record, exposing `"content"`, exists before + the first byte leaves, and counts the file whole, whether or not the caller reads it to the end. +- **Nothing checks the view again while the bytes flow.** A stream that has started runs to its + end after the view expires, its admin is demoted or the setting is lowered: those stop the next + call, not an answer already recorded. A workspace restart ends it, as it ends every session's + streams. + +This matters most for `readFileAtCommit()`, which reads any stored commit, a repository a worktree +checked out included, where a file can be far larger than a gadget's. + +A view lasts `DEFAULT_ADMIN_VIEW_LIFETIME_SECONDS` (15 minutes) unless the open asks for up to +`MAX_ADMIN_VIEW_LIFETIME_SECONDS` (an hour). A workspace restart (a revocation, a deletion, a +redeploy) ends every view along with every other session on it; inspect again. + +## Access levels and their bounds + +### The levels + +`adminWorkspaceAccess` is `"off"`, `"metadata"` (the default) or `"full"`, the most an inspection +gets (see [The level an inspection asks for](#the-level-an-inspection-asks-for)): + +- **off**: `inspectWorkspace()` is refused. +- **metadata**: the workspace's structure. Withheld: chat messages and queued steers, chat + attachments, file contents (at any commit, and pending in a chat), commit messages, the argument + summaries of calls queued for an agent, which blueprint a gadget follows (`upstream`, a share + link to code the view would otherwise withhold), the ids of the blueprints the workspace + publishes (an id is a share link to the blueprint's code, as `upstream` is), and the + descriptions and fields of actions, observations and hooks. Kept: titles (of the workspace, + chats, gadgets, actions and resources), collaborators with the notes on their grants, observer + records and their account ids, connections with their resource URLs and who created each, file + names and changed paths, commit ids, authors and times, attachment names, types and sizes and + each attachment's `oid`, share links' notes, blueprint titles and descriptions, and diagnostics. + Titles, names, notes and a blueprint's description are written by members and agents and can + echo content, restricted data included. A blueprint's description is kept, where the description + of an action, an observation or a hook is withheld. + - An attachment's `oid` is a hash of its bytes, which `listChatAttachments()` returns at + `metadata`: it does not give the bytes, and it does tell whether two attachments are the same + file, or the file a reader already has. + - Commit messages are content: they show only at `full` (`getCommitLog()` empties them below + it), where commit ids, authors and times show at `metadata`. + - Both levels cover every commit the workspace stores, the ones a connection supplied included + (a worktree's checkout): at `metadata` that repository's file names, commit ids, authors, + times and changed paths, at `full` its files and commit messages, read from the workspace's + copy and never by asking the connection (see [Inspection](#inspection)). +- **full**: everything, restricted data included. + +### The environment's bounds + +Two backend variables bound what admins may choose. They are environment variables for the reason +authentication config is (`auth/config.ts`): a compromised admin session must not be able to widen +them. | Variable | Values | Default | Effect | | --- | --- | --- | --- | +| `ADMIN_WORKSPACE_ACCESS_MAX` | `off`, `metadata`, `full` (any case, trimmed) | `metadata` (unset or blank) | The highest level admins may choose. Any other value is `off`, logged as an error: it bounds a security setting, so `none`, `false` or a typo of `off` must fail closed, never grant the default. | | `ADMIN_AUDIT_RETENTION_MIN_DAYS` | a positive whole number of days, at most 3650 (`MAX_ADMIN_AUDIT_RETENTION_DAYS`); a larger value counts as 3650 | 90 (unset, blank or invalid; an invalid value is logged) | The fewest days the audit log keeps a record, whatever admins choose. | The admins' choice is kept in `AdminConfig` as they set it, and `adminAccessPolicy()` holds it within -the bound wherever it is enforced: the settings audit and the audit log's own floor. -`getSettings()` reports `chosen`, the bound (`min`), and `effective`, which is what is enforced. The -setter refuses a value below the bound, naming the variable; a choice stored before the bound rose -is kept but not enforced. +the bounds wherever it is enforced: `inspectWorkspace()`, every view call, the settings audit and +the audit log's own floor. `getSettings()` reports `chosen`, the bound +(`max` or `min`), and `effective`, which is what is enforced. The setters refuse a value outside the +bounds, naming the variable; a choice stored before a bound tightened is kept but not enforced. +`"full"` therefore takes a deployment config change. -### Setting the bound +### Setting the bounds -It is a variable of the backend Worker (`workshop-backend`): +Both are variables of the backend Worker (`workshop-backend`): -- **Locally** (`pnpm run-local`, `pnpm dev-server`): set it in the shell or in the root `.dev.vars`; - `scripts/run-dev-server.ts` passes it through to the backend. +- **Locally** (`pnpm run-local`, `pnpm dev-server`): set them in the shell or in the root + `.dev.vars`; `scripts/run-dev-server.ts` passes both through to the backend, for example + `ADMIN_WORKSPACE_ACCESS_MAX=full pnpm dev-server`. - **Your own `wrangler deploy` or workerd config**: the backend Worker's `vars`. - **Deploy tools that write the backend's variables themselves** (the online deploy flow and the deployment starter's `cf-os deploy`, both linked from the [README](../README.md#get-started)): they replace the Worker's variables on every deploy, so a value set in the Cloudflare dashboard - lasts only until the next deploy, and until such a tool carries this variable its deployments run - with the default. Any redeploy that writes its own variables must carry it. + lasts only until the next deploy, and until such a tool carries these two variables its + deployments run with the defaults. Any redeploy that writes its own variables must carry both. ### When a variable is dropped -The floor falls back to 90 days, and with it the retention actually kept wherever the admins chose -less, which prunes the older records. So set the retention you want in the admin panel too: the -admins' choice persists in the deployment's settings across deploys, while the floor only stops an -admin session from choosing less. The Audit tab says when the floor is what holds retention up; it saves only a value that is +- The cap fails safe: a deployment set to `full` drops back to `metadata`. +- The floor falls back to 90 days, and with it the retention actually kept wherever the admins + chose less, which prunes the older records. + +So set the retention you want in the admin panel too: the admins' choice persists in the +deployment's settings across deploys, while the floor only stops an admin session from choosing +less. The Audit tab says when the floor is what holds retention up; it saves only a value that is typed, so type the number of days you want kept, even the minimum. ## The audit -### Records +### Two kinds of record + +- **`AdminAccessRecord`** (`kind: "workspace"`): one admin call into a workspace, the open or a view + method. It holds `at` (when the record was written), `admin`, `sessionId`, `via`, `workspaceId`, + the workspace's owner (`workspaceOwner`, resolved when the view opens; from the directory for an + open refused before it reaches the workspace; a username is unbounded, so a long one keeps its + start and a digest of the whole id, which keeps long ids apart), the workspace's title at the time, `method`, the call's arguments by name, the `reason` given at + the open, the level the open asked for (`requested`), the `access` level the call ran at (never + above `requested`), what its answer exposed (`exposed`), and `refused` when the level refused + it. +- **`AdminSettingRecord`** (`kind: "setting"`): one `AdminApi` setter's change. It holds `at`, + `admin`, `sessionId`, `via`, `method`, and `changes`, each naming a setting, an entry `key` for a + map or list setting, and its value `before` and `after`. -An `AdminSettingRecord` (`kind: "setting"`) is one `AdminApi` setter's change. It holds `at`, -`admin`, `sessionId`, `via`, `method`, and `changes`, each naming a setting, an entry `key` for a -map or list setting, and its value `before` and `after`. `listAdminAudit()` filters by kind, admin, -method and time, the method read through an index on the record's `json_extract()`. A method no -record type names is refused (capnweb-validate checks it against the method union); one that no -logged record has gives an empty page. `getAdminAuditStats()` filters by kind. +`listAdminAudit()` filters by kind, admin, method, workspace, workspace owner (`owner`), what a call +exposed (`exposed`) and time; `getAdminAuditStats()` by kind. The method and the owner stay in a +record's JSON, each read through an index on its `json_extract()`. A method no record type names is refused +(capnweb-validate checks it against the method union); one that no logged record has gives an empty +page. + +### What a call exposed + +`exposed` is `"content"` when a call's answer held something the `"metadata"` level withholds, and +`"metadata"` otherwise. Only a call at `"full"` can record `"content"`: + +| Call | Records `"content"` when its answer holds | +| --- | --- | +| The open (`inspectWorkspace`) | never | +| `describe()` | a gadget that names the blueprint it follows | +| `listChats()` | a queued steer | +| `getChat()` | a message or a compaction summary | +| `getChatState()` | a queued call with an argument summary | +| `listChatAttachments()` | never | +| `readChatAttachment()` | any attachment found | +| `listActions()` | an action, observation or hook entry with a description, or an action or observation with a field | +| `listFiles()` | never | +| `readFile()` | any file at the path, an `unreadable` one included | +| `getCommitLog()` | a commit with a message | +| `listFilesAtCommit()` | never | +| `readFileAtCommit()` | any file at the path, as `readFile()` | +| `listChangedPaths()` | never | +| `listPendingChanges()` | never | +| `readPendingFile()` | any file at the path, as `readFile()` | +| `listGatekeepers()` | a hook with a description | +| `listObservers()` | never | +| `listObserverRequirements()` | never | +| `listShareLinks()` | never | +| `listAutoApprovals()` | never | +| `listBlueprints()` | a blueprint with an id | +| `getDiagnostics()` | never: nothing it returns depends on the level | + +A refused call, or one whose read failed, records `"metadata"`. Present but empty is not content: +no queued steers, an empty description or field list, an `upstream` naming no blueprint (a gadget +built from scratch), an empty chat page, an empty argument summary, an empty commit message and an +emptied blueprint id all record `"metadata"`. The file reads (`readFile()`, `readFileAtCommit()`, +`readPendingFile()`) count any file they find, because an unreadable file's message can quote it +(a symlink's quotes its target), and overstating is the safe direction. The predicates +(`chatHoldsContent()` and its siblings in `admin-access.ts`) mirror the withholding helpers beside +them, so each is false for anything a metadata view returns. + +`exposed` is set from the answer, never from the level, so the content-read counts (see +[Statistics](#statistics)) count reads that returned content, not calls made at `"full"`. + +### Recorded before anything leaves + +An access record is written to the deployment-wide `AdminAuditDurableObject`, and only then is the +answer returned. Nothing is written to the workspace, so `listActions()`, `subscribeToActions()` +and everything else a member reads can never show an inspection. If the write fails, the call +fails and returns nothing. + +A setting record is written to the deployment-wide log before the change is stored: a change whose +record cannot be written is refused. + +### Which refusals are recorded + +| Refusal | Recorded | Why | +| --- | --- | --- | +| A live view's call refused by its level (`accessOff`, `contentWithheld`) | Yes, with `refused` | The admin reached the workspace and asked for something. | +| A live view's call whose read throws (a missing object, a pending gadget, an invalid path, a malformed commit or attachment id, a commit that is not stored, a chat that does not exist, an attachment that is not the chat's, a legacy connection in a grant's scope) | Yes, exposing `"metadata"` | The admin reached the workspace and asked for something; the call returned only its error. A string argument (a commit id, a path, an attachment id) is checked inside the read, so a malformed one is a failed read. | +| An open refused because the authoritative setting or the cap is `"off"` | Yes, titled from the directory (cut) or `""` | No workspace is reached, so its title comes from the directory. | +| An open refused because only the workspace's KV mirror still says `"off"` | Yes | The workspace's own check refused it. | +| A dead capability (no longer an admin), an expired view, a deleted workspace, a malformed or unknown workspace id, including a call that expiry, a demotion or a deletion overtakes while it reads | No | Nothing was reachable, or nothing the call read left the server, and recording a gone workspace would write into storage that must stay empty. | +| A call whose arguments are rejected before it starts: a mistyped argument, `listActions()`'s invalid `beforeId`, `getCommitLog()`'s `depth` outside 1 to `MAX_ADMIN_COMMIT_LOG_DEPTH` | No | Validation refuses the call before the view's own checks run, as it refuses a mistyped argument: nothing was read, and the call asked for nothing a record could name (a depth of 0, a `beforeId` of -1). Only an argument rejected before the call starts goes unrecorded; a malformed string is checked inside the read, and is the failed read above. | ### Setting records @@ -108,52 +510,121 @@ chokepoint records `diffAdminConfig(current, next)` before `storage.adminConfig. repairing the KV mirror). A record is kept if the change then fails, for example when the KV mirror write fails and the config is rolled back: it records the attempt. -Not recorded: reads (`getSettings`, `isBlueprintFeatured`, `getUpdateStatus` and the audit log -itself), the gateway tests, `checkForUpdates`, and the deployment's own promotion of its +Not recorded: reads (`getSettings`, `isBlueprintFeatured`, `getUpdateStatus`, the listings and the +audit log itself), the gateway tests, `checkForUpdates`, and the deployment's own promotion of its bundled formats, which no admin made. ### Text bounds -Setting keys are cut to `MAX_ADMIN_AUDIT_ARG_LENGTH` (256) characters by `cutAdminAuditText()`, the -last character replaced with `…` and never splitting a surrogate pair: a record marks its cuts, -never passing a fragment off as the whole. Values are capped at 10,000 with excerpts, and a setter's -diff has a bounded number of changes, so every record, and every page of them, is bounded. +Arguments, workspace titles and setting keys are cut to `MAX_ADMIN_AUDIT_ARG_LENGTH` (256) +characters by `cutAdminAuditText()`, the last character replaced with `…` and never splitting a +surrogate pair: a record marks its cuts, never passing a fragment off as the whole. Reasons are +capped at `MAX_ADMIN_ACCESS_REASON_LENGTH` (500), values at 10,000 with excerpts, and a setter's diff +has a bounded number of changes, so every record, and every page of them, is bounded. -The store keeps each field once: kind, time and admin in columns, the rest as JSON. +The deployment-wide store keeps each field once (kind, time, admin, workspace id and title, and +`exposed` in columns, the rest as JSON) and cuts the title itself, whatever its caller sent. The +title is the one field of a record that a workspace member writes, and it has no length limit of +its own: kept whole, an over-long title would overflow the row (the platform caps one at 8 MiB), failing every +audit write and so refusing every investigation of that workspace, or push a listing past the RPC +size limit. ### Retention and pruning The log keeps a record for the effective retention (`adminAuditRetentionDays`, default 90 days, -never below the floor). It deletes up to 1000 expired rows at the start of each call and never lists -or counts one, and it holds the floor itself whatever retention a caller passes. +never below the floor). It deletes up to 1000 expired rows at the start of each call that passes the +committed retention, and never lists or counts one, and it holds the floor itself whatever retention +a caller passes. A view call's record passes none, since a view reads the KV mirror, whose retention +may be one an admin has since raised; the next listing, statistics read or settings change prunes. ### Statistics `getAdminAuditStats({ days, kind })` counts the last `days` UTC days, today included (7 by default, -at most 366): the total, one count per day (empty days included), and the ten admins with the most -records. A window reaching past the retention counts nothing there. +at most 366): the total, one count per day (empty days included), the ten admins with the most +records, each with its content reads, and the content reads in all (`contentReads`, the records +whose `exposed` is `"content"`). An admin outside the top ten who reads content is not ranked there, +but `listAdminAudit({ exposed: "content" })` lists every content read, as +`listAdminAudit({ workspace })` lists every call into a workspace. Only access records read content, +so for the `"setting"` kind the content reads are 0. A window reaching past the retention counts +nothing there. + +The Audit tab shows the week's content reads in all and for each top admin, a +sensitivity on each record (Content read, Workspace metadata, Setting change), the level an +inspection asked for where its view got less, each call's target user (the workspace's owner) in a +column of its own, and filters its log by admin, action, sensitivity, workspace and target user. + +## Error codes + +Expected refusals carry a code from `ADMIN_ACCESS_ERROR_CODES` (read with +`getAdminAccessErrorCode()`), which survives Durable Object RPC and Cap'n Web: + +| Code | When | +| --- | --- | +| `ADMIN_NOT_ADMIN` | The caller, or a view's admin, is no longer named in `ADMINS`. | +| `ADMIN_WORKSPACE_NOT_FOUND` | A malformed id, or one naming no workspace (never initialised or since deleted). | +| `ADMIN_WORKSPACE_ACCESS_OFF` | The level is `"off"`, by the setting or the cap. | +| `ADMIN_CONTENT_WITHHELD` | A method that reads only content (`getChat()`, `readChatAttachment()`, `readFile()`, `readFileAtCommit()`, `readPendingFile()`) on a view below `"full"`, whether its inspection asked for `"metadata"` (the default) or the deployment held it there. | +| `ADMIN_VIEW_EXPIRED` | The view's lifetime has run out. | ## Tests Unit suites, in `packages/workshop-backend/__tests__/`: -- `admin-access.test.ts`: the bound's parsing, the policy, validation, the cuts, and setting values - and excerpts. +- `admin-access.test.ts`: admin status, the bounds' parsing (failing closed), the policy, validation + (the commit log's depth included), the per-call level and the content methods it refuses, the + cuts, setting values and excerpts, withholding, and whether an answer holds content (each + withholder with its predicate, empty values included). - `admin-config.test.ts` (`diffAdminConfig`) and `admin-settings-audit.test.ts`: what each setter - records, before which write, and the retention's bound. -- `admin-audit.test.ts`: the log's listing, filters, cursors, schema, retention and stats. -- `admin-api-impl.test.ts`: who `AdminApiImpl` records as making a change, and its forwarding of - listings and statistics. + records, before which write, and the setters' bounds. +- `admin-audit.test.ts`: the deployment-wide log's listing, filters (`exposed` alone and with the + others), cursors, schema (the `exposed` column and the CHECK tying it to access records), + retention, stats (content reads in all and per top entry, titles, by kind), the method and owner + indexes, and multi-MiB titles. +- `admin-api-impl.test.ts`: `AdminApiImpl`'s inspection path, the level asked for (the lower of it + and the deployment's, and the validator's `TypeError` for any other level), refused opens, + listings and forwarding. - `cli-admins-only.test.ts`: the `via` a session's User-Agent gives its records. +- `admin-workspace-view.test.ts`: the view, from an uninitialised id that stays uninitialised to the + per-call checks; `requested` and `exposed` on each method's record; a view that asked for + `"full"` held at `"metadata"` and never raised; the reads that fail recorded, and a depth out of + range refused unrecorded; a failed record returning nothing; the races with deletion and expiry, + at the setting read and while a call reads its answer, writing nothing; withholding; a + 2,000,000-character title; the reads that never pull (a gadget's head, history, changed paths, + files at a commit, attachments, and a chat's pending content with a worktree); the sharing reads + never reaching a connection; an inspection calling every method leaving the workspace's storage, + presence and alarm as they were; and the view's surface being exactly what the audit names: + every method in `ADMIN_WORKSPACE_METHODS` is recorded, and the view class exposes no other member + over RPC (a helper must be `#private`: the class's public members, not its interface, are its + RPC surface). +- `connection-creator.test.ts`: the creator stamped at each creation path, and absent on an ambient + capsule and on a resource whose turn has no record. +- `git-cache.test.ts` and `git-cache-history.test.ts`: the local readers against the pulling ones, + and a history whose parent is not stored. +- `worktrees.test.ts`: a chat's content folded from stored objects alone, failing rather than + pulling when a base is not stored. +- `user-directory.test.ts`, `user-directory-feed.test.ts` and `workspace-directory-feed.test.ts`: the + directory and both feeds. The last includes a deletion that parks a sync on a real call to another + Durable Object and checks that the deletion still completes. Integration suites, in `packages/integration-tests/__tests__/`: +- `workshop-admin-inspection.test.ts`: every admin method this surface adds refused to a non-admin; + the directory; an inspection recorded apart from the workspace and absent from the owner's + `listActions()` and `subscribeToActions()`; metadata and off; the level asked for, and raising + the setting not raising a view; every read end to end with what it returned and exposed, and + the `exposed` filter; the content methods refused below `"full"`; the creator of a connection + made through `newGatekeeper()`, and none on an ambient one; who holds an observer record, with + what was verified for each; a deleted id refused without being written; an audit write failure, + where a content read returns nothing; expiry; and long text end to end. - `workshop-admin-audit.test.ts`: every setting change recorded before it takes effect, sessions, - filters, stats across two admins, and a refused change when its record cannot be written. -- `workshop-admin-bounds.test.ts`: the bound's default, and a redeploy that raises it and demotes an - admin, whose records stay. + filters, stats and content reads across two admins, and a refused change when its record cannot + be written. +- `workshop-admin-bounds.test.ts`: the bounds' defaults, and a redeploy that tightens them (an + invalid cap failing closed) and demotes an admin. The redeploy ends every session, the demoted + admin's view among them, so the per-call re-check that stops a live view once its admin is demoted + is covered by `admin-workspace-view.test.ts`. The integration suite reaches Durable Object storage directly (see [integration-testing.md](integration-testing.md#reaching-a-durable-objects-storage-from-a-test)): it -faults the log with a trigger on `admin_audit`, and adds a workspace row for the kind filter to -leave out. +faults the deployment-wide log with a trigger on `admin_audit`, and shows that an inspected deleted +id stays unwritten by listing its `sqlite_master`. diff --git a/docs/blueprints.md b/docs/blueprints.md index f971c29026..2927b8cccd 100644 --- a/docs/blueprints.md +++ b/docs/blueprints.md @@ -210,7 +210,7 @@ Importing a version 2 archive stores the same release commit under a new bluepri ## Admin Features and Featured Blueprints -Deployments can optionally configure a set of admins through the backend worker's `ADMINS` binding, an array of usernames or sign-in emails. Entries match regardless of the case of ASCII letters, since an identity provider may capitalize an email differently than it was typed. `ADMIN_AUDIT_RETENTION_MIN_DAYS` bounds how briefly those admins may choose to keep the admin audit log; see [admin-access.md](admin-access.md). +Deployments can optionally configure a set of admins through the backend worker's `ADMINS` binding, an array of usernames or sign-in emails. Entries match regardless of the case of ASCII letters, since an identity provider may capitalize an email differently than it was typed. `ADMIN_WORKSPACE_ACCESS_MAX` and `ADMIN_AUDIT_RETENTION_MIN_DAYS` bound what those admins may choose for workspace inspection and audit retention; see [admin-access.md](admin-access.md). Admins get access to two extra RPCs: diff --git a/docs/integration-testing.md b/docs/integration-testing.md index 1742f994c3..47a0829886 100644 --- a/docs/integration-testing.md +++ b/docs/integration-testing.md @@ -117,13 +117,23 @@ RPC session the child needs. See `observer-exclusion.test.ts`'s `reopenAfterRest `await harness.server.getWorker(name).getDurableObjectStorage(className, { id } | { name })` returns a handle on one Durable Object's SQLite storage, whose `exec()` resolves to the result rows. -`{ name: "" }` addresses the object a Worker reaches with `getByName("")`. The admin suites use it to -fault a store: to show that a call fails when its audit record cannot be written, the suite installs -a trigger on `admin_audit`, the deployment-wide audit log's table -(`CREATE TRIGGER ... BEFORE INSERT ON admin_audit BEGIN SELECT RAISE(ABORT, ...); END`), and drops it -afterwards. The insert then fails exactly as a real storage failure would, inside the real Worker, -and production code carries no test hook, which would be a switch that disables the audit shipped -in every deployment. +`{ name: "" }` addresses the object a Worker reaches with `getByName("")`. Two uses shape the admin +suites: + +- **Faulting a store.** To show that a call fails when its audit record cannot be written, the suite + installs a trigger on `admin_audit`, the deployment-wide audit log's table + (`CREATE TRIGGER ... BEFORE INSERT ON admin_audit BEGIN SELECT RAISE(ABORT, ...); END`), and drops + it afterwards. The insert then fails exactly as a real storage failure would, inside the real + Worker, and production code carries no test hook, which would be a switch that disables the audit + shipped in every deployment. +- **Proving an object was never written.** `listDurableObjectIds()` cannot tell: it lists every + object with a storage file, and any visit creates one (miniflare records the object's name in it) + that a deletion leaves behind. What can tell is `sqlite_master` with miniflare's own `__miniflare*` + tables left out. The Overseer keeps its records in typed storage over the KV API, whose `_cf_KV` + table the first write creates and `deleteAll()` drops, so whether that table exists says whether + anything has been written. `_cf_KV` itself is unreadable (`SQLITE_AUTH`), hence `sqlite_master`. A + test client cannot mint an unused workspace id (creating a workspace opens it), so the suite uses + a deleted one. ### A redeploy keeps storage, so an upgrade can be tested diff --git a/docs/observers.md b/docs/observers.md index bf288564e5..6bd7e62d55 100644 --- a/docs/observers.md +++ b/docs/observers.md @@ -107,6 +107,48 @@ The check works as follows: sharing graph (`computeEffectiveAccess`), and for a gadget, whether their access includes a gadget that reaches it -- never off whether they currently have it open. +### Admin access: a deliberate exception + +A deployment admin (a user the backend's `ADMINS` variable lists) can inspect a workspace through +`AdminApi.inspectWorkspace()` without being verified against its gatekeepers and without being +seen by its members: `OverseerDurableObject.openForAdmin()` skips `authorizeCollaborator()` and +observer verification by explicit policy, and the admin is never registered as an observer. So +`excludeObservers` never names an admin, and no observation is blocked on an admin's account. + +The exception is bounded. Restricted data (chat messages and attachments, file contents, commit +messages, the argument summaries of queued agent calls, the descriptions and fields of actions, +observations and hooks, and blueprint ids; see +[What a call exposed](admin-access.md#what-a-call-exposed) for each call) is visible only at the +`"full"` access level, which only a deployment config change permits and which an inspection must +ask for: `ADMIN_WORKSPACE_ACCESS_MAX` caps the level, `metadata` by default, and fails closed to +`off` for any value that is not a level. Titles, names, notes and blueprint descriptions stay +visible at `"metadata"`, and can echo content. + +What a connection supplied is read only as the workspace stores it, and the connection is never +asked. That includes a repository: a worktree can be a checkout through a connection, and the view +reads any commit the workspace stores. So an admin never verified against that connection sees, at +`"metadata"`, the repository's file names, commit ids, authors and times, and the paths that +differ between two of its commits; and at `"full"` its files and commit messages, as far as the +workspace pulled them, from the workspace's copy. `"metadata"` also shows each attachment's name, +type, size and `oid` (a hash of its bytes, which `listChatAttachments()` returns), and which +collaborators hold an observer record, with the id of the account each chose for each connection; +the account itself is in the collaborator's own user account and is not read. + +The compensating controls are: + +- the admin list and the access cap are environment variables, out of reach of an admin session; +- every inspection takes a reason, recorded with every call made through the view; +- an inspection gets `"metadata"` unless it asks for `"full"` and the deployment allows it, so + reading content is a separate, recorded step with its own reason; +- a view expires (15 minutes by default, an hour at most); +- every call re-checks that the view is live and its admin still an admin; +- every call is recorded, with whether its answer held content, before anything it read leaves the + server, in the deployment-wide audit log (nothing is written to the workspace); a call whose + record cannot be written fails, returning nothing; +- `ADMIN_AUDIT_RETENTION_MIN_DAYS` sets a floor on how long those records are kept. + +[admin-access.md](admin-access.md) is the design record. + --- ## 2. Background: relevant existing code diff --git a/docs/public-server.md b/docs/public-server.md index a8acb872d8..9d85c44b24 100644 --- a/docs/public-server.md +++ b/docs/public-server.md @@ -16,6 +16,7 @@ connect the account's capabilities. There's no single switch — the pieces turn | Each gatekeeper's OAuth credentials (on the gatekeeper Worker) | Required for that gatekeeper to actually authenticate. In dev, seeded from `GOOGLE_*` / `GITHUB_*` / `CLOUDFLARE_OAUTH_*` shell vars (see `run-dev-server.ts`). | | `ENABLE_CLOUDFLARE_LIMITS=true` | Enables the free daily limit + Cloudflare-credits top-up flow. Billing reads a token from the connected Cloudflare gatekeeper. | | `DISABLE_PASSWORD_AUTH=true` | Hides username/password, leaving gatekeeper sign-in only (ignored unless `AUTH_GATEKEEPERS` is non-empty, to avoid lockout). | +| `ADMIN_WORKSPACE_ACCESS_MAX=off\|metadata\|full` | The most of a workspace an admin may inspect (`AdminApi.inspectWorkspace()`); admins choose up to it in the admin panel. Default `metadata`; any other value counts as `off`. Set on the backend Worker (its vars, or `.dev.vars` locally); a deploy tool that rewrites the backend's variables must carry it, or it falls back to its default. See [admin-access.md](admin-access.md), which says where each kind of deployment sets it. | | `ADMIN_AUDIT_RETENTION_MIN_DAYS=90` | The fewest days the admin audit log keeps a record; admins choose at least this in the admin panel. Default 90; at most 3650, and a larger value counts as 3650. Set on the backend Worker (its vars, or `.dev.vars` locally); a deploy tool that rewrites the backend's variables must carry it, or it falls back to its default. See [admin-access.md](admin-access.md), which says where each kind of deployment sets it. | The primary account key is always the user's **verified email**: signing in with any allowlisted diff --git a/docs/sharing.md b/docs/sharing.md index 5fa81e2fca..00a1c82ae9 100644 --- a/docs/sharing.md +++ b/docs/sharing.md @@ -162,6 +162,8 @@ Because access is recomputed from the graph on every `open()` and `getGadget()`, A share-key redemption goes through the same gate. The redeeming open() then verifies a `build` recipient as an observer like any other collaborator, and a gadget recipient when they get the gadget; a recipient whose verification fails persists as an unverified collaborator until removed (see Known limitations). +Deployment admins can inspect a workspace outside the sharing graph: `AdminApi.inspectWorkspace()` mints a read-only `AdminWorkspaceView` through `openForAdmin()`, which never consults the permission graph, adds no collaborator and joins no session. It is a deliberate exception, bounded by an environment-set access cap and an audit recorded before anything an admin reads leaves the server; see [admin-access.md](admin-access.md). + ### `ownerInvitesOnly` A gatekeeper whose data source requires each recipient to be granted access individually can mark an observation `ownerInvitesOnly` (`ObservationDescription` in `packages/workshop-shared/src/gatekeeper.ts`, typically alongside `containsRestrictedData`). Once any such observation is authorized, the Overseer permanently sets the `ownerInvitesOnly` storage flag (in `authorizeObservation`, after the exclusion gate, so a refused observation sets nothing). From then on, **only direct grants from the owner count**: `computeEffectiveAccess` skips every share-link edge (even for a link the owner created) and every `user` edge from anyone but the owner. Because every access decision derives from that computation, `open()`, `getGadget()`, `listCollaborators`, and the removal previews agree without further checks. The flag covers the whole workspace: an owner grant of gadget A still grants only A. diff --git a/packages/api-cli/src/run.test.ts b/packages/api-cli/src/run.test.ts index 35d93bbe32..d200b93fcc 100644 --- a/packages/api-cli/src/run.test.ts +++ b/packages/api-cli/src/run.test.ts @@ -454,7 +454,7 @@ test("search reads a command's option names and the callback methods it may be a assert.equal(rankOf(asking, "notice", / lock$/), 0); // As a doc's words: 2, under the 3 of a command named by the word. assert.equal(searchCommands(realTable, "configure observers")[0].command, "cf-os api open-workspace "); - assert.equal(rankOf(realTable, "observer", /^cf-os api open-workspace $/), 2); + assert.equal(rankOf(realTable, "observer", /^cf-os api open-workspace $/), 4); }); test("search finds the commands for agents' everyday tasks in the API's own words", () => { @@ -499,7 +499,7 @@ test("search finds the commands for agents' everyday tasks in the API's own word ...["redeem share link", "redeem a share link", "open workspace with share key", "join workspace from link", "join a shared workspace", "open a workspace shared with me", "configure observers"] .map((task): [string, RegExp, number] => [task, openWorkspace, 1]), - ["observer verification", openWorkspace, 3], + ["observer verification", openWorkspace, 4], ["choose accounts to observe a shared workspace", openWorkspace, 3], ["share key", openWorkspace, 3], ]; @@ -772,8 +772,9 @@ test("an unknown command or option says which was meant: a near name, or where t real(["list-chats"], /list-chats is on Overseer: `cf-os api open-workspace list-chats`/); real(["open-workspace", "g", "list-actions", "--option", "{}"], /Did you mean --options\?/); real(["list-workspace"], /Did you mean list-workspaces\?/); - // A name cut short wins over a typo that changes its verb: get-chat is two edits from new-chat. - real(["get-chat"], /Did you mean get-chat-[a-z-]+, on Overseer: `cf-os api open-workspace get-chat-/); + // A name cut short wins over a typo that changes its verb: get-gatekeeper is two edits from + // new-gatekeeper. + real(["get-gatekeeper"], /Did you mean get-gatekeeper-app\? /); // A short name a typo starts with is no run-on: list-chtas is list-chats, not a bare `list`. real(["open-workspace", "g", "list-chtas"], /^Overseer has no command "list-chtas"\. Did you mean list-chats\? /); real(["list-worksapces"], /Did you mean list-workspaces\? /); diff --git a/packages/integration-tests/__tests__/workshop-admin-audit.test.ts b/packages/integration-tests/__tests__/workshop-admin-audit.test.ts index a17b4f9acc..52e8fdd861 100644 --- a/packages/integration-tests/__tests__/workshop-admin-audit.test.ts +++ b/packages/integration-tests/__tests__/workshop-admin-audit.test.ts @@ -4,17 +4,18 @@ import { afterAll, beforeAll, expect, it } from "vitest"; import { MAX_ADMIN_AUDIT_VALUE_LENGTH, MAX_SITE_NAME_LENGTH, type AdminApi, type AdminAuditKind, type AdminAuditRecord, - type AdminAuditStats, type AdminSettingRecord, type AuthenticatedApi, + type AdminAuditStats, type AdminSettingRecord, type AuthenticatedApi, type Overseer, } from "@gadgets/workshop-shared/api"; import { ADMIN_USERNAME, startHarness, TEST_GATEKEEPER_BINDING, TEST_GATEKEEPER_DIR, TEST_VENDOR_ID, type Harness, } from "../src/harness.js"; import { NetworkInterceptor } from "../src/network-interceptor.js"; -import { connect, signUp } from "../src/rpc-client.js"; +import { connect, nextUsernames, signUp } from "../src/rpc-client.js"; // A second deployment admin, so that the statistics can be checked across two. const AUDITOR_USERNAME = "auditor"; +const REASON = "Audit log check"; // Admin settings and the audit log are deployment-wide, so this file owns its harness and runs // serially. @@ -38,7 +39,11 @@ beforeAll(async () => { harness = await startHarness({ gatekeepers: [{ binding: TEST_GATEKEEPER_BINDING, dir: TEST_GATEKEEPER_DIR }], patchWorkshop: config => { - config.vars = { ...config.vars, ADMINS: [ADMIN_USERNAME, AUDITOR_USERNAME] }; + config.vars = { + ...config.vars, + ADMINS: [ADMIN_USERNAME, AUDITOR_USERNAME], + ADMIN_WORKSPACE_ACCESS_MAX: "full", + }; }, }); ({ api: adminApi, admin } = await signUpAdmin(ADMIN_USERNAME)); @@ -58,6 +63,9 @@ afterAll(async () => { /** The newest setting record of the deployment-wide audit log, if any. */ async function newestSetting(): Promise { const [record] = (await admin.listAdminAudit({ kind: "setting", limit: 1 })).records; + if (record !== undefined && record.kind !== "setting") { + throw new Error("The kind filter returned an access record"); + } return record; } @@ -69,6 +77,17 @@ async function newSetting(previous: AdminSettingRecord | undefined): Promise }> { + const [ownerName] = nextUsernames(prefix); + const owner = stack.use(await signUp(stack.use(connect(harness.url)), ownerName!)); + const ws = stack.use(await owner.newWorkspace()); + const title = `Workspace of ${ownerName}`; + await ws.setTitle(title); + return { id: (await ws.getMetadata()).id, title, ownerName: ownerName!, ws }; +} + // The smallest PNG a site logo accepts: the signature and a 1x1 IHDR chunk, 33 bytes. function tinyPng(): Uint8Array { const png = new Uint8Array(33); @@ -263,25 +282,21 @@ async function kinds(kind: AdminAuditKind): Promise { } it("filters the log by kind", async () => { - // A workspace access record, as the table can hold, which a setting filter leaves out. - const storage = await harness.server.getWorker("workshop-backend") - .getDurableObjectStorage("AdminAuditDurableObject", { name: "" }); + using stack = new DisposableStack(); + const { id } = await newWorkspace(stack, "kinds"); const settingsBefore = await admin.getAdminAuditStats({ kind: "setting" }); - await storage.exec(`INSERT INTO admin_audit - (at, kind, admin, workspace_id, workspace_title, exposed, record) - VALUES (${Date.now()}, 'workspace', 'filter-check', 'workspace-1', 'Filter check', - 'metadata', '{}')`); - try { - const setting = await kinds("setting"); - expect(setting.length).toBeGreaterThan(0); - expect(new Set(setting)).toEqual(new Set(["setting"])); - const settingsAfter = await admin.getAdminAuditStats({ kind: "setting" }); - expect(settingsAfter.totalEvents) - .toBe(settingsBefore.totalEvents - droppedDays(settingsBefore, settingsAfter)); - expect(settingsAfter.topAdmins.map(entry => entry.admin)).not.toContain("filter-check"); - } finally { - await storage.exec("DELETE FROM admin_audit WHERE admin = 'filter-check'"); + { + using _view = await admin.inspectWorkspace(id, REASON); } + const workspace = await kinds("workspace"); + expect(workspace.length).toBeGreaterThan(0); + expect(new Set(workspace)).toEqual(new Set(["workspace"])); + const setting = await kinds("setting"); + expect(setting.length).toBeGreaterThan(0); + expect(new Set(setting)).toEqual(new Set(["setting"])); + const settingsAfter = await admin.getAdminAuditStats({ kind: "setting" }); + expect(settingsAfter.totalEvents) + .toBe(settingsBefore.totalEvents - droppedDays(settingsBefore, settingsAfter)); }); it("filters the log by method, and refuses a method that is not audited", async () => { @@ -318,19 +333,30 @@ function droppedDays(before: AdminAuditStats, after: AdminAuditStats): number { } it("counts the log across two admins", async () => { + using stack = new DisposableStack(); + const a = await newWorkspace(stack, "statsa"); + const b = await newWorkspace(stack, "statsb"); const before = await admin.getAdminAuditStats(); const settingsBefore = await admin.getAdminAuditStats({ kind: "setting" }); try { + { + using view = await admin.inspectWorkspace(a.id, REASON); + await view.describe(); + await view.listChats(); + await view.getDiagnostics(); + } await admin.setUserSearchEnabled(true); - await auditor.setSiteName("Stats check"); - await auditor.setSiteName(""); + { + using view = await auditor.inspectWorkspace(b.id, REASON); + await view.listChats(); + } const after = await admin.getAdminAuditStats(); expect(after.days).toBe(7); - expect(after.totalEvents).toBe(before.totalEvents + 3 - droppedDays(before, after)); + expect(after.totalEvents).toBe(before.totalEvents + 7 - droppedDays(before, after)); const byDay = growth(before.byDay, after.byDay, entry => entry.day, entry => entry.count); - expect([...byDay.values()].reduce((sum, count) => sum + count, 0)).toBe(3); + expect([...byDay.values()].reduce((sum, count) => sum + count, 0)).toBe(7); expect(after.byDay).toHaveLength(7); const days = after.byDay.map(({ day }) => Date.parse(`${day}T00:00:00Z`)); @@ -338,12 +364,18 @@ it("counts the log across two admins", async () => { const admins = growth(before.topAdmins, after.topAdmins, entry => entry.admin, entry => entry.count); - expect(admins.get(ADMIN_USERNAME)).toBe(1); + expect(admins.get(ADMIN_USERNAME)).toBe(5); expect(admins.get(AUDITOR_USERNAME)).toBe(2); + // Setting records name no workspace, so A lists its four access records alone. + for (const [workspace, count] of [[a.id, 4], [b.id, 2]] as const) { + expect((await admin.listAdminAudit({ workspace, limit: 200 })).records).toHaveLength(count); + } const settingsAfter = await admin.getAdminAuditStats({ kind: "setting" }); expect(settingsAfter.totalEvents) - .toBe(settingsBefore.totalEvents + 3 - droppedDays(settingsBefore, settingsAfter)); + .toBe(settingsBefore.totalEvents + 1 - droppedDays(settingsBefore, settingsAfter)); + expect(growth(settingsBefore.topAdmins, settingsAfter.topAdmins, entry => entry.admin, + entry => entry.count).get(ADMIN_USERNAME)).toBe(1); } finally { await admin.setUserSearchEnabled(false); } @@ -354,6 +386,66 @@ it("counts the log across two admins", async () => { } }); +it("counts content reads, by admin and by workspace", async () => { + using stack = new DisposableStack(); + const a = await newWorkspace(stack, "contenta"); + const b = await newWorkspace(stack, "contentb"); + const chatA = await a.ws.newChat("hello", null); + const chatB = await b.ws.newChat("hello", null); + + try { + await admin.setAdminWorkspaceAccess("full"); + const before = await admin.getAdminAuditStats(); + { + using view = await admin.inspectWorkspace(a.id, REASON, { access: "full" }); + await view.getChat(chatA); + await view.getChat(chatA); + // No steer is queued, so this exposes no content. + await view.listChats(); + } + { + using view = await auditor.inspectWorkspace(b.id, REASON, { access: "full" }); + await view.getChat(chatB); + } + const after = await admin.getAdminAuditStats(); + + // Every content read in this harness happens during the run, so none has left the window. + expect(after.contentReads - before.contentReads).toBe(3); + const admins = growth(before.topAdmins, after.topAdmins, + entry => entry.admin, entry => entry.contentReads); + expect(admins.get(ADMIN_USERNAME)).toBe(2); + expect(admins.get(AUDITOR_USERNAME)).toBe(1); + // The open, the two getChat() calls and listChats(); only the getChat() calls read content. + for (const [workspace, count, reads] of [[a.id, 4, 2], [b.id, 2, 1]] as const) { + expect((await admin.listAdminAudit({ workspace, limit: 200 })).records).toHaveLength(count); + expect((await admin.listAdminAudit({ workspace, exposed: "content", limit: 200 })).records) + .toHaveLength(reads); + } + + // Each workspace's records, found by its owner. + const owned = (await admin.listAdminAudit({ owner: a.ownerName, limit: 200 })).records; + expect(owned.map(record => record.method)) + .toEqual(["listChats", "getChat", "getChat", "inspectWorkspace"]); + expect(new Set(owned.map(record => record.kind === "workspace" && record.workspaceId))) + .toEqual(new Set([a.id])); + + const settings = await admin.getAdminAuditStats({ kind: "setting" }); + expect(settings.contentReads).toBe(0); + expect(settings.topAdmins.every(entry => entry.contentReads === 0)).toBe(true); + expect((await admin.getAdminAuditStats({ kind: "workspace" })).contentReads) + .toBe(after.contentReads); + + const [newest] = (await admin.listAdminAudit( + { exposed: "content", admin: AUDITOR_USERNAME, limit: 1 })).records; + expect(newest).toMatchObject({ + kind: "workspace", workspaceId: b.id, method: "getChat", args: { chatId: chatB }, + requested: "full", access: "full", exposed: "content", + }); + } finally { + await admin.setAdminWorkspaceAccess("metadata"); + } +}); + it("refuses a setting change whose audit record cannot be written", async () => { // Addresses the deployment-wide log, which is always getByName(""). const storage = await harness.server.getWorker("workshop-backend") diff --git a/packages/integration-tests/__tests__/workshop-admin-bounds.test.ts b/packages/integration-tests/__tests__/workshop-admin-bounds.test.ts index 7068b4140e..17a48deb36 100644 --- a/packages/integration-tests/__tests__/workshop-admin-bounds.test.ts +++ b/packages/integration-tests/__tests__/workshop-admin-bounds.test.ts @@ -1,22 +1,26 @@ import type { RpcStub } from "capnweb"; import { afterAll, beforeAll, expect, it } from "vitest"; import { - type AdminApi, type AdminAuditRecord, type AuthenticatedApi, + ADMIN_ACCESS_ERROR_CODES, getAdminAccessErrorCode, + type AdminApi, type AdminAuditRecord, type AdminWorkspaceView, type AuthenticatedApi, } from "@gadgets/workshop-shared/api"; import { ADMIN_USERNAME, startHarness, type Harness } from "../src/harness.js"; import { NetworkInterceptor } from "../src/network-interceptor.js"; -import { connect, logIn, signUp, waitFor } from "../src/rpc-client.js"; +import { connect, logIn, nextUsernames, signUp, waitFor } from "../src/rpc-client.js"; // An admin whom the redeploy removes from ADMINS. const DEMOTED_USERNAME = "demoted"; +const REASON = "Bounds check"; -// The environment's bound is read at startup and the redeploy restarts every Worker, so this file +// The environment bounds are read at startup and the redeploy restarts every Worker, so this file // owns its harness, runs serially, and its cases run in order. const network = new NetworkInterceptor(); const fileScope = new DisposableStack(); let harness: Harness; let admin: RpcStub; let demotedApi: RpcStub; +let demotedView: RpcStub; +let workspaceId: string; async function adminApiOf(api: RpcStub): Promise> { const stub = await api.getAdminApi(); @@ -53,26 +57,48 @@ async function newest(api: RpcStub, kind: AdminAuditRecord["kind"]) return (await api.listAdminAudit({ kind, limit: 1 })).records[0]; } -it("holds the admins' choice within the deployment's default", async () => { +it("holds the admins' choices within the deployment's defaults", async () => { const settings = await admin.getSettings(); + expect(settings.adminWorkspaceAccess) + .toEqual({ effective: "metadata", chosen: "metadata", max: "metadata" }); expect(settings.adminAuditRetentionDays).toEqual({ effective: 90, chosen: 90, min: 90 }); const before = await newest(admin, "setting"); + await expect(admin.setAdminWorkspaceAccess("full")).rejects.toThrow(/ADMIN_WORKSPACE_ACCESS_MAX/); await expect(admin.setAdminAuditRetentionDays(89)) .rejects.toThrow(/ADMIN_AUDIT_RETENTION_MIN_DAYS/); expect(await newest(admin, "setting")).toEqual(before); + + try { + await admin.setAdminWorkspaceAccess("off"); + expect((await admin.getSettings()).adminWorkspaceAccess) + .toEqual({ effective: "off", chosen: "off", max: "metadata" }); + } finally { + await admin.setAdminWorkspaceAccess("metadata"); + } }); -it("keeps a demoted admin's records", async () => { +it("lets an admin inspect another user's workspace", async () => { + const [ownerName] = nextUsernames("boundsowner"); + using ownerPublic = connect(harness.url); + using owner = await signUp(ownerPublic, ownerName!); + using ws = await owner.newWorkspace(); + workspaceId = (await ws.getMetadata()).id; + const demoted = await adminApiOf(demotedApi); - await demoted.setSiteName("Bounds check"); - expect(await newest(admin, "setting")).toMatchObject( - { kind: "setting", admin: DEMOTED_USERNAME, method: "setSiteName" }); + demotedView = fileScope.use(await demoted.inspectWorkspace(workspaceId, REASON)); + expect(await demotedView.describe()).toMatchObject({ id: workspaceId, access: "metadata" }); }); -it("applies a redeploy's raised minimum and demotion", async () => { +it("applies a redeploy's tighter bounds and demotion", async () => { await harness.redeployWorkshop(config => { - config.vars = { ...config.vars, ADMINS: [ADMIN_USERNAME], ADMIN_AUDIT_RETENTION_MIN_DAYS: "120" }; + config.vars = { + ...config.vars, + ADMINS: [ADMIN_USERNAME], + ADMIN_AUDIT_RETENTION_MIN_DAYS: "120", + // Not a level, so the cap must fail closed to "off". + ADMIN_WORKSPACE_ACCESS_MAX: "none", + }; }); // The redeploy restarts every Worker, ending every session open at the time. @@ -85,17 +111,38 @@ it("applies a redeploy's raised minimum and demotion", async () => { } }); + // No session survives an ADMINS change, so the old view fails here because its session ended. + // The per-call re-check that stops a live view once its admin is demoted is covered by the + // Overseer's unit suite. const demoted = await logInAfresh(DEMOTED_USERNAME); expect(await demoted.getAdminApi()).toBeNull(); expect(await demoted.amIAdmin()).toBe(false); + await expect(demotedView.describe()).rejects.toThrow(); const adminApi = await logInAfresh(ADMIN_USERNAME); const fresh = await adminApi.getAdminApi(); if (fresh === null) throw new Error("The admin API was unavailable after the redeploy"); stack.use(fresh); - expect((await fresh.getSettings()).adminAuditRetentionDays) - .toEqual({ effective: 120, chosen: 90, min: 120 }); + const settings = await fresh.getSettings(); + expect(settings.adminAuditRetentionDays).toEqual({ effective: 120, chosen: 90, min: 120 }); + expect(settings.adminWorkspaceAccess) + .toEqual({ effective: "off", chosen: "metadata", max: "off" }); + await expect(fresh.setAdminWorkspaceAccess("metadata")) + .rejects.toThrow(/ADMIN_WORKSPACE_ACCESS_MAX/); + + let code: string | undefined; + try { + stack.use(await fresh.inspectWorkspace(workspaceId, REASON)); + } catch (error) { + code = getAdminAccessErrorCode(error); + } + expect(code).toBe(ADMIN_ACCESS_ERROR_CODES.accessOff); + expect(await newest(fresh, "workspace")).toMatchObject({ + kind: "workspace", workspaceId, admin: ADMIN_USERNAME, method: "inspectWorkspace", + reason: REASON, access: "off", refused: ADMIN_ACCESS_ERROR_CODES.accessOff, + }); const demotedRecords = (await fresh.listAdminAudit({ admin: DEMOTED_USERNAME })).records; - expect(demotedRecords.map(record => record.method)).toEqual(["setSiteName"]); + expect(demotedRecords.map(record => record.kind === "workspace" && record.method)) + .toEqual(["describe", "inspectWorkspace"]); }); diff --git a/packages/integration-tests/__tests__/workshop-admin-inspection.test.ts b/packages/integration-tests/__tests__/workshop-admin-inspection.test.ts new file mode 100644 index 0000000000..1a6169d909 --- /dev/null +++ b/packages/integration-tests/__tests__/workshop-admin-inspection.test.ts @@ -0,0 +1,1014 @@ +import type { RpcStub } from "capnweb"; +import { afterAll, beforeAll, expect, it } from "vitest"; +import { + ADMIN_ACCESS_ERROR_CODES, getAdminAccessErrorCode, getOpenWorkspaceErrorCode, + MAX_ADMIN_AUDIT_ARG_LENGTH, MAX_DIRECTORY_TEXT_LENGTH, OPEN_WORKSPACE_ERROR_CODES, + type ActionLogEntry, type ActionsSubscriber, type AdminAccessErrorCode, type AdminAccessRecord, + type AdminApi, type AdminAuditQuery, type AdminAuditRecord, type AdminViewAccess, + type AdminWorkspaceAccess, type AuthenticatedApi, type Overseer, type PresenceParticipant, + type PresenceSubscriber, type WorkpieceId, +} from "@gadgets/workshop-shared/api"; +import { diffFiles } from "@gadgets/workshop-shared/code-change"; +import { MAX_BUFFERED_BLOB_SIZE } from "@gadgets/workshop-shared/gatekeeper"; +import type { TestSession } from "../fixtures/gatekeeper-test/src/test-gatekeeper.js"; +import { + ADMIN_USERNAME, settleRestart, startHarness, TEST_GATEKEEPER_BINDING, TEST_GATEKEEPER_DIR, + TEST_VENDOR_ID, type Harness, +} from "../src/harness.js"; +import { NetworkInterceptor } from "../src/network-interceptor.js"; +import { + connect, listConnectedAccounts, nextUsernames, ObserverConfigRecorder, RpcTarget, signUp, + stubFor, waitFor, WorkpieceRecorder, type ConnectedAccount, +} from "../src/rpc-client.js"; + +// A second deployment admin, so that records can be told apart by admin. +const AUDITOR_USERNAME = "auditor"; +const REASON = "Support ticket 123"; +const NOTES = "Meeting notes: ship the admin audit log.\n"; +const DRAFT_NOTES = `${NOTES}Draft: review the audit filters.\n`; +const ATTACHMENT = "Agenda for the audit review.\n"; +// A text file over MAX_BUFFERED_BLOB_SIZE, which a view answers as a stream. Its first character +// takes two bytes, so that its length in bytes is not its length in characters. +const LARGE_PATH = "large.txt"; +const LARGE_TEXT = `é${"a".repeat(MAX_BUFFERED_BLOB_SIZE)}`; +const OBSERVATION_DESCRIPTION = + "Read the deterministic value exposed by the integration-test gatekeeper."; +const LINK_NOTE = "For the audit reviewers"; +// The kind of the test connection's write, which an auto-approval rule names. +const SET_VALUE = { tag: "set-value", label: "Set value" }; + +// Admin settings and the audit log are deployment-wide, so this file owns its harness and runs +// serially. +const network = new NetworkInterceptor(); +const fileScope = new DisposableStack(); +let harness: Harness; +let admin: RpcStub; +let auditor: RpcStub; + +async function adminApiOf(username: string): Promise> { + const api = fileScope.use(await signUp(fileScope.use(connect(harness.url)), username)); + const stub = await api.getAdminApi(); + if (stub === null) throw new Error(`The admin API was unavailable to ${username}`); + return fileScope.use(stub); +} + +beforeAll(async () => { + network.install(); + harness = await startHarness({ + gatekeepers: [{ binding: TEST_GATEKEEPER_BINDING, dir: TEST_GATEKEEPER_DIR }], + patchWorkshop: config => { + config.vars = { + ...config.vars, + ADMINS: [ADMIN_USERNAME, AUDITOR_USERNAME], + ADMIN_WORKSPACE_ACCESS_MAX: "full", + }; + }, + }); + admin = await adminApiOf(ADMIN_USERNAME); + auditor = await adminApiOf(AUDITOR_USERNAME); +}); + +afterAll(async () => { + try { + fileScope.dispose(); + await harness?.server.close(); + expect(network.getUnmockedCalls()).toEqual([]); + } finally { + network.uninstall(); + } +}); + +/** The code `call` is refused with; throws if it succeeds. */ +async function refusalOf(call: PromiseLike): Promise { + try { + await call; + } catch (error) { + return getAdminAccessErrorCode(error); + } + throw new Error("The call was not refused"); +} + +/** The code `call` is refused with, or null when it succeeds: for polling until a refusal. */ +async function outcomeOf(call: PromiseLike): Promise { + try { + await call; + return null; + } catch (error) { + return getAdminAccessErrorCode(error); + } +} + +/** The newest record of the deployment-wide audit log matching `query`. */ +async function newestRecord(query: Omit = {}) + : Promise { + return (await admin.listAdminAudit({ ...query, limit: 1 })).records[0]; +} + +/** Every record of the deployment-wide audit log about workspace `id`, newest first. */ +async function workspaceRecords(id: string): Promise { + return (await admin.listAdminAudit({ workspace: id, limit: 200 })).records; +} + +/** An access record, narrowed. */ +function accessRecord(record: AdminAuditRecord | undefined): AdminAccessRecord { + if (record?.kind !== "workspace") throw new Error("Expected a workspace access record"); + return record; +} + +const headOf = (workpieces: WorkpieceRecorder, gadgetId: WorkpieceId, after?: string) => + waitFor(`a new head for gadget ${gadgetId}`, async () => { + const summary = workpieces.summaries.get(gadgetId); + return summary?.type === "gadget" && summary.commitId !== undefined && + summary.commitId !== after ? summary.commitId : null; + }); + +type Seeded = { + owner: RpcStub; + ownerName: string; + ws: RpcStub; + id: string; + title: string; +}; + +/** A fresh owner's workspace, retitled so that its records can be recognised. */ +async function newWorkspace(stack: DisposableStack, prefix: string): Promise { + const [ownerName] = nextUsernames(prefix); + const owner = stack.use(await signUp(stack.use(connect(harness.url)), ownerName!)); + const ws = stack.use(await owner.newWorkspace()); + const title = `Workspace of ${ownerName}`; + await ws.setTitle(title); + return { owner, ownerName: ownerName!, ws, id: (await ws.getMetadata()).id, title }; +} + +/** The test account of `api`'s user, which the fake vendor mints with no sign-in. */ +async function testAccount(api: RpcStub): Promise { + await api.provisionAmbientAccount(TEST_VENDOR_ID); + const account = (await listConnectedAccounts(api)) + .find(candidate => candidate.vendorId === TEST_VENDOR_ID); + if (account === undefined) throw new Error("The test account was not provisioned"); + return account; +} + +type Populated = Seeded & { + chatId: number; + gadgetId: WorkpieceId; + /** The test connection's id. */ + connectionId: WorkpieceId; + /** The gadget's head before anything was committed to it. */ + empty: string; + /** The gadget's head once the seed chat was accepted: the commit that holds notes.txt. */ + head: string; + /** A chat whose first message carries one attachment, a.txt, holding ATTACHMENT. */ + attachmentChatId: number; + /** That attachment's id. */ + attachmentId: string; + /** A chat whose changes are not accepted: DRAFT_NOTES in notes.txt, and LARGE_PATH added. */ + draftChatId: number; + /** The test connection's open session: each readValue() records one more observation. */ + session: RpcStub; +}; + +/** + * A workspace holding something of each kind a view reads: an observation through a test + * connection, a chat whose first message is "hello", a chat with an attachment, a gadget whose + * head holds notes.txt, and a chat whose changes to that gadget are not accepted. The gadget is + * seeded without a model: a human-only chat submits the file at the empty head, and accepting it + * commits it. + */ +async function populatedWorkspace(stack: DisposableStack, prefix: string): Promise { + const seeded = await newWorkspace(stack, prefix); + const { owner, ws } = seeded; + const account = await testAccount(owner); + const connection = await ws.newGatekeeper( + account.id, "https://gadgets-test.example/things/inspected"); + if (connection === null) throw new Error("The test connection was not created"); + stack.use(connection); + const session = stack.use(await connection.openSession() as RpcStub); + expect(await session.readValue()).toBe(42); + const chatId = await ws.newChat("hello", null); + + const workpieces = new WorkpieceRecorder(); + const workpiecesStub = stack.use(stubFor(workpieces)); + stack.use(await ws.subscribeToWorkpieces(workpiecesStub)); + await workpieces.loaded; + const gadget = stack.use(ws.createGadget("Notes", undefined, "NOTES")); + const gadgetId = await gadget.getId(); + const empty = await headOf(workpieces, gadgetId); + const seedChatId = await ws.newChat("Seed", null); + await ws.submitCodeChange(seedChatId, { + generation: 0, revision: 0, clientId: "seed", seq: 1, + pins: [{ gadgetId, baseCommit: empty }], + change: diffFiles(new Map([[gadgetId, new Map()]]), + new Map([[gadgetId, new Map([["notes.txt", NOTES]])]])), + }); + expect(await ws.mergeChanges(seedChatId)).toEqual({ outcome: "merged" }); + const head = await headOf(workpieces, gadgetId, empty); + + const oid = await ws.uploadBlob(new TextEncoder().encode(ATTACHMENT)); + const attachmentChatId = await ws.newChat( + "With attachment", null, undefined, [{ oid, mimeType: "text/plain", name: "a.txt" }]); + const sent = (await ws.getChatHistory(attachmentChatId)).messages + .find(message => message.type === "message"); + if (sent?.type !== "message" || !sent.attachments?.length) { + throw new Error("The attachment was not sent"); + } + // The message's reference holds more than a view lists of an attachment: its path in the + // chat's scratch worktree. + expect(sent.attachments[0]).toHaveProperty("workspacePath"); + const attachmentId = sent.attachments[0]!.id; + + const draftChatId = await ws.newChat("Draft", null); + await ws.submitCodeChange(draftChatId, { + generation: 0, revision: 0, clientId: "draft", seq: 1, + pins: [{ gadgetId, baseCommit: head }], + change: diffFiles(new Map([[gadgetId, new Map([["notes.txt", NOTES]])]]), + new Map([[gadgetId, new Map([["notes.txt", DRAFT_NOTES], [LARGE_PATH, LARGE_TEXT]])]])), + }); + return { + ...seeded, chatId, gadgetId, connectionId: await connection.getId(), empty, head, + attachmentChatId, attachmentId, draftChatId, session, + }; +} + +class ActionsRecorder extends RpcTarget implements ActionsSubscriber { + readonly entries: ActionLogEntry[] = []; + entry(record: ActionLogEntry): void { this.entries.push(record); } + ready(): void {} +} + +class PresenceRecorder extends RpcTarget implements PresenceSubscriber { + readonly seen = new Set(); + init(participants: PresenceParticipant[]): void { + for (const participant of participants) this.seen.add(participant.user.id); + } + add(participant: PresenceParticipant): void { this.seen.add(participant.user.id); } + remove(_key: string): void {} +} + +it("refuses every admin method to a non-admin", async () => { + using stack = new DisposableStack(); + const { owner, ownerName, ws, id } = await newWorkspace(stack, "nonadmin"); + expect(await owner.getAdminApi()).toBeNull(); + expect(await owner.amIAdmin()).toBe(false); + + // None of these is on AuthenticatedApi: they are reachable only through an AdminApi. + const loose = owner as unknown as Record Promise>; + const calls: [string, unknown[]][] = [ + ["listUsers", [{}]], + ["listWorkspaces", [{}]], + ["inspectWorkspace", [id, REASON]], + ["listAdminAudit", [{}]], + ["getAdminAuditStats", [{}]], + ["setAdminWorkspaceAccess", ["full"]], + ["setAdminAuditRetentionDays", [120]], + ]; + for (const [method, args] of calls) { + await expect(loose[method]!(...args), method).rejects.toThrow(); + } + + // The bypass is on the Overseer's Durable Object, never on a capability a user holds. + const looseWs = ws as unknown as Record Promise>; + await expect(looseWs.openForAdmin!(ownerName, REASON, "session", { + via: "api", requested: "full", access: "full", expiresAt: new Date(Date.now() + 60_000), + })).rejects.toThrow(); +}); + +it("lists users and workspaces from the deployment directory", async () => { + using stack = new DisposableStack(); + const [ownerName] = nextUsernames("dirowner"); + const owner = stack.use(await signUp(stack.use(connect(harness.url)), ownerName!, "Dir Owner")); + + const listed = await waitFor("the new user to be listed", async () => + (await admin.listUsers({ query: ownerName })).users.find(user => user.id === ownerName) ?? null); + expect(listed).toEqual({ + id: ownerName, name: "Dir Owner", created: expect.any(Date), lastLogin: expect.any(Date), + isAdmin: false, + }); + // Asked through the second admin's capability, which lists the same directory. + for (const username of [ADMIN_USERNAME, AUDITOR_USERNAME]) { + const user = await waitFor(`${username} to be listed`, async () => + (await auditor.listUsers({ query: username })).users.find(entry => entry.id === username) + ?? null); + expect(user.isAdmin).toBe(true); + } + + const first = stack.use(await owner.newWorkspace()); + const second = stack.use(await owner.newWorkspace()); + const firstId = (await first.getMetadata()).id; + const secondId = (await second.getMetadata()).id; + const workspaces = await waitFor("both workspaces to be listed", async () => { + const page = await admin.listWorkspaces({ owner: ownerName }); + return page.workspaces.length === 2 ? page.workspaces : null; + }); + expect(workspaces.map(workspace => workspace.id)).toEqual([firstId, secondId].toSorted()); + expect(workspaces.find(workspace => workspace.id === firstId)).toMatchObject({ + id: firstId, title: (await first.getMetadata()).title, + owner: { id: ownerName, name: "Dir Owner" }, created: expect.any(Date), + }); + + await first.setTitle("Renamed for the directory"); + await waitFor("the rename to be listed", async () => + (await admin.listWorkspaces({ owner: ownerName })).workspaces + .some(workspace => workspace.id === firstId && + workspace.title === "Renamed for the directory") || null); + + const pageOne = await admin.listWorkspaces({ owner: ownerName, limit: 1 }); + expect(pageOne.workspaces).toHaveLength(1); + expect(pageOne.nextCursor).toBeDefined(); + const pageTwo = await admin.listWorkspaces( + { owner: ownerName, limit: 1, cursor: pageOne.nextCursor }); + expect(pageTwo.workspaces).toHaveLength(1); + expect(pageTwo.nextCursor).toBeUndefined(); + expect([...pageOne.workspaces, ...pageTwo.workspaces].map(workspace => workspace.id)) + .toEqual([firstId, secondId].toSorted()); + + // The directory entry is removed inside deleteSelf(), before its critical section, so the + // deletion is listed once it returns. + await first.deleteSelf(); + expect((await admin.listWorkspaces({ owner: ownerName })).workspaces + .map(workspace => workspace.id)).toEqual([secondId]); + + for (const limit of [0, 201, 1.5]) { + await expect(admin.listUsers({ limit }), `listUsers limit ${limit}`).rejects.toThrow(); + await expect(admin.listWorkspaces({ limit }), `listWorkspaces limit ${limit}`) + .rejects.toThrow(); + } + await expect(admin.listUsers({ query: "two\nlines" })).rejects.toThrow(); + await expect(admin.listWorkspaces({ query: "two\nlines" })).rejects.toThrow(); +}); + +it("records an inspection apart from the workspace, where its members never see it", async () => { + using stack = new DisposableStack(); + const { ws, id, title, session, ownerName } = await populatedWorkspace(stack, "inspected"); + + const actions = new ActionsRecorder(); + const presence = new PresenceRecorder(); + const actionsStub = stack.use(stubFor(actions)); + const presenceStub = stack.use(stubFor(presence)); + stack.use(await ws.subscribeToActions(actionsStub)); + stack.use(await ws.subscribeToPresence(presenceStub)); + const actionsBefore = await ws.listActions(); + + { + using view = await admin.inspectWorkspace(id, REASON); + const description = await view.describe(); + expect(description).toMatchObject({ id, title, access: "metadata" }); + await view.listChats(); + await view.listActions(); + // The roster the members see: the owner's session is on it, and the view never is. + const present = (await view.getDiagnostics()).present.map(participant => participant.user.id); + expect(present).toContain(ownerName); + expect(present).not.toContain(ADMIN_USERNAME); + + const logged = await workspaceRecords(id); + expect(logged.map(record => record.method)) + .toEqual(["getDiagnostics", "listActions", "listChats", "describe", "inspectWorkspace"]); + const sessionId = logged[0]!.sessionId; + for (const record of logged) { + expect(record).toMatchObject({ + kind: "workspace", workspaceId: id, workspaceOwner: ownerName, workspaceTitle: title, + admin: ADMIN_USERNAME, reason: REASON, sessionId, via: "api", requested: "metadata", + access: "metadata", exposed: "metadata", + }); + expect(accessRecord(record).refused).toBeUndefined(); + } + + const pages: AdminAuditRecord[][] = []; + let cursor: string | undefined; + do { + const page = await admin.listAdminAudit({ workspace: id, limit: 2, cursor }); + pages.push(page.records); + cursor = page.nextCursor; + } while (cursor !== undefined); + expect(pages.map(page => page.length)).toEqual([2, 2, 1]); + expect(pages.flat()).toEqual(logged); + + expect((await admin.listAdminAudit({ workspace: id, admin: ADMIN_USERNAME })).records) + .toEqual(logged); + expect((await admin.listAdminAudit({ workspace: id, admin: AUDITOR_USERNAME })).records) + .toEqual([]); + const since = logged[2]!.at; + expect((await admin.listAdminAudit({ workspace: id, since })).records) + .toEqual(logged.filter(record => record.at.getTime() >= since.getTime())); + + for (const reason of ["", " ", "x".repeat(501)]) { + await expect(admin.inspectWorkspace(id, reason), `reason of ${reason.length}`) + .rejects.toThrow(); + } + expect(await workspaceRecords(id)).toEqual(logged); + } + + // A view that restarted the workspace would end the owner's sessions here. + await settleRestart(); + expect(await ws.listActions()).toEqual(actionsBefore); + expect(actions.entries).toEqual([]); + expect(presence.seen.size).toBeGreaterThan(0); + expect(presence.seen).not.toContain(ADMIN_USERNAME); + + // The subscription delivers live entries only, so one more observation reaching it shows that it + // was live throughout the view's calls, and that their absence above is not a silent one. + expect(await session.readValue()).toBe(42); + await waitFor("the new observation to reach the subscriber", async () => + actions.entries.length > 0 || null); + expect(actions.entries).toEqual([expect.objectContaining({ + type: "observation", gatekeeperId: expect.any(Number), + description: expect.objectContaining({ description: OBSERVATION_DESCRIPTION }), + })]); +}); + +it("withholds content below full access, and turns inspection off", async () => { + using stack = new DisposableStack(); + const { id, title, chatId, gadgetId } = await populatedWorkspace(stack, "modes"); + + const settings = await admin.getSettings(); + expect(settings.adminWorkspaceAccess) + .toEqual({ effective: "metadata", chosen: "metadata", max: "full" }); + expect(settings.adminAuditRetentionDays).toEqual({ effective: 90, chosen: 90, min: 90 }); + + try { + using metadataView = await admin.inspectWorkspace(id, REASON); + expect(await refusalOf(metadataView.getChat(chatId))) + .toBe(ADMIN_ACCESS_ERROR_CODES.contentWithheld); + expect(await refusalOf(metadataView.readFile(gadgetId, "notes.txt"))) + .toBe(ADMIN_ACCESS_ERROR_CODES.contentWithheld); + // Refused by the level after being recorded, so each refusal is in the log. + const [readRefusal, chatRefusal] = (await workspaceRecords(id)).map(accessRecord); + expect(readRefusal).toMatchObject({ + method: "readFile", args: { gadgetId, path: "notes.txt" }, requested: "metadata", + access: "metadata", exposed: "metadata", refused: ADMIN_ACCESS_ERROR_CODES.contentWithheld, + }); + expect(chatRefusal).toMatchObject({ + method: "getChat", args: { chatId }, requested: "metadata", access: "metadata", + exposed: "metadata", refused: ADMIN_ACCESS_ERROR_CODES.contentWithheld, + }); + + const withheld = (await metadataView.listActions()).entries + .find(entry => entry.type === "observation"); + expect(withheld?.description.title).toBe("Read the test value"); + expect(withheld?.description.description).toBe(""); + expect(withheld?.description).not.toHaveProperty("fields"); + const gadget = (await metadataView.describe()).workpieces + .find(workpiece => workpiece.id === gadgetId); + expect(gadget).toBeDefined(); + expect(gadget).not.toHaveProperty("upstream"); + expect(await metadataView.listFiles(gadgetId)).toEqual([{ name: "notes.txt", kind: "file" }]); + + await admin.setAdminWorkspaceAccess("full"); + using fullView = await admin.inspectWorkspace(id, REASON, { access: "full" }); + expect((await fullView.getChat(chatId)).messages) + .toContainEqual(expect.objectContaining({ type: "message", message: "hello" })); + expect(await fullView.readFile(gadgetId, "notes.txt")).toEqual({ kind: "text", text: NOTES }); + expect((await fullView.listActions()).entries.find(entry => entry.type === "observation") + ?.description.description).toBe(OBSERVATION_DESCRIPTION); + expect((await fullView.describe()).workpieces.find(workpiece => workpiece.id === gadgetId)) + .toMatchObject({ upstream: {} }); + + // Lowering the setting reaches a live view. + await admin.setAdminWorkspaceAccess("metadata"); + await waitFor("the full view to be lowered", async () => + await outcomeOf(fullView.getChat(chatId)) === ADMIN_ACCESS_ERROR_CODES.contentWithheld + || null); + + await admin.setAdminWorkspaceAccess("off"); + const reason = "Off-mode check"; + expect(await refusalOf(admin.inspectWorkspace(id, reason))) + .toBe(ADMIN_ACCESS_ERROR_CODES.accessOff); + expect(await newestRecord({ kind: "workspace" })).toMatchObject({ + kind: "workspace", workspaceId: id, workspaceTitle: title, method: "inspectWorkspace", + admin: ADMIN_USERNAME, reason, access: "off", refused: ADMIN_ACCESS_ERROR_CODES.accessOff, + }); + await waitFor("the live view to be turned off", async () => + await outcomeOf(metadataView.listChats()) === ADMIN_ACCESS_ERROR_CODES.accessOff || null); + expect(await newestRecord({ workspace: id })).toMatchObject({ + method: "listChats", access: "off", refused: ADMIN_ACCESS_ERROR_CODES.accessOff, + reason: REASON, + }); + + await admin.setAdminWorkspaceAccess("metadata"); + + await expect(admin.setAdminWorkspaceAccess("everything" as AdminWorkspaceAccess)) + .rejects.toThrow(); + await expect(admin.setAdminAuditRetentionDays(89)).rejects.toThrow(); + await expect(admin.setAdminAuditRetentionDays(3651)).rejects.toThrow(); + } finally { + await admin.setAdminWorkspaceAccess("metadata"); + await admin.setAdminAuditRetentionDays(90); + } +}); + +it("withholds attachments, a commit's or a draft's files and commit messages below full access", + async () => { + using stack = new DisposableStack(); + const { id, gadgetId, head, attachmentChatId, attachmentId, draftChatId } = + await populatedWorkspace(stack, "withheld"); + + try { + using metadataView = await admin.inspectWorkspace(id, REASON); + // Each reads only content, so the level refuses it once it is recorded. + const refusals: [string, Record, () => PromiseLike][] = [ + ["readChatAttachment", { chatId: attachmentChatId, id: attachmentId }, + () => metadataView.readChatAttachment(attachmentChatId, attachmentId)], + ["readFileAtCommit", { commitId: head, path: "notes.txt" }, + () => metadataView.readFileAtCommit(head, "notes.txt")], + ["readPendingFile", { chatId: draftChatId, gadgetId, path: "notes.txt" }, + () => metadataView.readPendingFile(draftChatId, gadgetId, "notes.txt")], + ]; + for (const [method, args, call] of refusals) { + expect(await refusalOf(call()), method).toBe(ADMIN_ACCESS_ERROR_CODES.contentWithheld); + expect(await newestRecord({ workspace: id }), method).toMatchObject({ + method, args, requested: "metadata", access: "metadata", exposed: "metadata", + refused: ADMIN_ACCESS_ERROR_CODES.contentWithheld, + }); + } + + // A commit's message describes its change, so the history arrives without it. The record + // holds no depth, since the call gave none. + const withheld = await metadataView.getCommitLog(head); + expect(withheld[0]).toMatchObject({ oid: head, message: "" }); + const withheldRecord = accessRecord(await newestRecord({ workspace: id })); + expect(withheldRecord).toMatchObject({ + method: "getCommitLog", access: "metadata", exposed: "metadata", + }); + expect(withheldRecord.args).toEqual({ fromCommit: head }); + expect(withheldRecord.refused).toBeUndefined(); + + // A depth outside the range is refused before anything is recorded. + await expect(metadataView.getCommitLog(head, 0)).rejects.toThrow(TypeError); + expect(await newestRecord({ workspace: id })).toEqual(withheldRecord); + + await admin.setAdminWorkspaceAccess("full"); + using fullView = await admin.inspectWorkspace(id, REASON, { access: "full" }); + const full = await fullView.getCommitLog(head); + expect(full[0]!.message).not.toBe(""); + expect(await newestRecord({ workspace: id })).toMatchObject({ + method: "getCommitLog", args: { fromCommit: head }, access: "full", exposed: "content", + }); + // The message is all that the lower level withheld. + expect(withheld).toEqual(full.map(commit => ({ ...commit, message: "" }))); + } finally { + await admin.setAdminWorkspaceAccess("metadata"); + } +}); + +it("opens a view at the lower of the level asked for and the deployment's", async () => { + using stack = new DisposableStack(); + const { id, chatId } = await populatedWorkspace(stack, "requested"); + const heldReason = "Asked for full, held to metadata"; + const defaultReason = "Asked for nothing"; + const fullReason = "Asked for full, granted it"; + + // The setting is "metadata", so asking for "full" opens the view at "metadata", unrefused. + using held = await admin.inspectWorkspace(id, heldReason, { access: "full" }); + expect((await held.describe()).access).toBe("metadata"); + expect(await refusalOf(held.getChat(chatId))).toBe(ADMIN_ACCESS_ERROR_CODES.contentWithheld); + + try { + await admin.setAdminWorkspaceAccess("full"); + // A view keeps the level it was minted at: raising the setting does not reach it. + expect(await refusalOf(held.getChat(chatId))).toBe(ADMIN_ACCESS_ERROR_CODES.contentWithheld); + expect((await held.describe()).access).toBe("metadata"); + + // An open that asks for nothing gets "metadata", whatever the deployment allows. + { + using view = await admin.inspectWorkspace(id, defaultReason); + expect((await view.describe()).access).toBe("metadata"); + expect(await refusalOf(view.getChat(chatId))).toBe(ADMIN_ACCESS_ERROR_CODES.contentWithheld); + } + + { + using view = await admin.inspectWorkspace(id, fullReason, { access: "full" }); + expect((await view.describe()).access).toBe("full"); + expect((await view.getChat(chatId)).messages) + .toContainEqual(expect.objectContaining({ type: "message", message: "hello" })); + expect(await newestRecord({ workspace: id })).toMatchObject({ + method: "getChat", reason: fullReason, requested: "full", access: "full", + exposed: "content", + }); + } + + const records = (await workspaceRecords(id)).map(accessRecord); + const heldRecords = records.filter(record => record.reason === heldReason); + expect(heldRecords.map(record => record.method).toSorted()) + .toEqual(["describe", "describe", "getChat", "getChat", "inspectWorkspace"]); + for (const record of heldRecords) { + expect(record).toMatchObject({ requested: "full", access: "metadata", exposed: "metadata" }); + } + const defaultRecords = records.filter(record => record.reason === defaultReason); + expect(defaultRecords).toHaveLength(3); + for (const record of defaultRecords) { + expect(record).toMatchObject({ requested: "metadata", access: "metadata" }); + } + + // A level that is not a view level is refused before anything is recorded. The server's + // validator throws, and capnweb delivers the throw as a rejection that keeps its TypeError. + const newest = await newestRecord({ workspace: id }); + await expect(admin.inspectWorkspace(id, REASON, { access: "off" as AdminViewAccess })) + .rejects.toThrow(TypeError); + expect(await newestRecord({ workspace: id })).toEqual(newest); + } finally { + await admin.setAdminWorkspaceAccess("metadata"); + } +}); + +it("records what each call returned, and lists content reads apart", async () => { + using stack = new DisposableStack(); + const { + ws, ownerName, id, chatId, gadgetId, connectionId, empty, head, attachmentChatId, attachmentId, + draftChatId, + } = await populatedWorkspace(stack, "exposed"); + // A share link and an auto-approval rule for the two listings below to show. No write is + // pending, so the rule applies nothing. + const link = await ws.createShareLink("workspace", LINK_NOTE); + await ws.setAutoApprovedActionKind(connectionId, SET_VALUE); + + try { + await admin.setAdminWorkspaceAccess("full"); + using view = await admin.inspectWorkspace(id, REASON, { access: "full" }); + // Each call that checks its answer does so before its record is read. + const calls: [string, () => PromiseLike][] = [ + ["getChat", () => view.getChat(chatId)], + // The answer is absent, so it holds no content. + ["readFile", () => view.readFile(gadgetId, "missing.txt")], + ["readFile", () => view.readFile(gadgetId, "notes.txt")], + ["listFiles", () => view.listFiles(gadgetId)], + // The workspace queues no steer. + ["listChats", () => view.listChats()], + // The observation's description. + ["listActions", () => view.listActions()], + // The test connection binds no hooks. + ["listGatekeepers", () => view.listGatekeepers()], + // The gadget was built from scratch, so its upstream {} names no blueprint. + ["describe", () => view.describe()], + ["getDiagnostics", () => view.getDiagnostics()], + // No call is queued for the chat's agent, so no summary of a call's arguments is kept. + ["getChatState", () => view.getChatState(chatId)], + // An attachment's name, type, size and hash show at every level, as a title does. The + // message's own reference also holds the attachment's path in the chat's scratch worktree, + // which the listing leaves out. + ["listChatAttachments", async () => { + const attachments = await view.listChatAttachments(attachmentChatId); + expect(attachments).toEqual([{ + id: attachmentId, oid: expect.any(String), mimeType: "text/plain", name: "a.txt", + size: new TextEncoder().encode(ATTACHMENT).byteLength, sequence: expect.any(Number), + }]); + expect(Object.keys(attachments[0]!).toSorted()) + .toEqual(["id", "mimeType", "name", "oid", "sequence", "size"]); + }], + // The attachment's bytes. + ["readChatAttachment", async () => expect(new TextDecoder().decode( + await view.readChatAttachment(attachmentChatId, attachmentId))).toBe(ATTACHMENT)], + // The message of the commit that accepted the seed chat. + ["getCommitLog", () => view.getCommitLog(head)], + // Paths are names. The commit's tree, which the draft's added file is not in. + ["listFilesAtCommit", async () => + expect(await view.listFilesAtCommit(head)).toEqual([{ name: "notes.txt", kind: "file" }])], + // The committed text, not the draft's. + ["readFileAtCommit", async () => + expect(await view.readFileAtCommit(head, "notes.txt")) + .toEqual({ kind: "text", text: NOTES })], + // The answer is absent, so it holds no content. + ["readFileAtCommit", () => view.readFileAtCommit(head, "missing.txt")], + // The paths that differ, not the difference: that takes two file reads. + ["listChangedPaths", async () => + expect(await view.listChangedPaths(empty, head)).toEqual(["notes.txt"])], + // The paths the draft adds, modifies and removes, not what it holds at them. + ["listPendingChanges", async () => + expect(await view.listPendingChanges(draftChatId)).toEqual([{ + gadgetId, baseCommit: head, mergedCommit: head, + added: [LARGE_PATH], modified: ["notes.txt"], removed: [], + }])], + // The draft's text. + ["readPendingFile", async () => + expect(await view.readPendingFile(draftChatId, gadgetId, "notes.txt")) + .toEqual({ kind: "text", text: DRAFT_NOTES })], + // A text over MAX_BUFFERED_BLOB_SIZE arrives as a stream of its UTF-8 bytes, recorded as any + // found file is. The stream is read to its end: a caller consumes every stream it receives. + ["readPendingFile", async () => { + const file = await view.readPendingFile(draftChatId, gadgetId, LARGE_PATH); + if (file.kind !== "stream") throw new Error(`Expected a stream, not ${file.kind}`); + const text = await new Response(file.content).text(); + expect(file.byteLength).toBe(new TextEncoder().encode(LARGE_TEXT).byteLength); + expect(file.byteLength).toBeGreaterThan(LARGE_TEXT.length); + // Compared outside the matcher, whose failure would print both texts whole. + expect(text === LARGE_TEXT).toBe(true); + }], + // A holder's profile and the connection, vendor and account ids of their choices are names, + // shown at every level. The list is empty: no collaborator has opened the workspace. + ["listObservers", async () => expect(await view.listObservers()).toEqual([])], + // The test connection's vendor, title and URL, which listGatekeepers() shows too. + ["listObserverRequirements", () => view.listObserverRequirements("workspace")], + // A link's id, note, creator and role are names, as a title is. Its key is in no answer. + ["listShareLinks", async () => { + const links = await view.listShareLinks(); + expect(links).toEqual([{ + linkId: link.linkId, note: LINK_NOTE, created: expect.any(Date), + createdBy: expect.objectContaining({ type: "user", id: ownerName }), role: "build", + }]); + expect(JSON.stringify(links)).not.toContain(link.key); + }], + // A rule's connection, kind of action and enabler are names too. + ["listAutoApprovals", async () => + expect(await view.listAutoApprovals()).toEqual([{ + gatekeeperId: connectionId, actionKind: SET_VALUE, + enabledBy: expect.objectContaining({ type: "user", id: ownerName }), + }])], + // The workspace publishes none. + ["listBlueprints", () => view.listBlueprints()], + ]; + const recorded: { method: string; exposed: string }[] = []; + for (const [method, call] of calls) { + await call(); + const record = accessRecord(await newestRecord({ workspace: id })); + expect(record).toMatchObject({ method, requested: "full", access: "full" }); + expect(record.refused).toBeUndefined(); + recorded.push({ method, exposed: record.exposed }); + } + expect(recorded).toEqual([ + { method: "getChat", exposed: "content" }, + { method: "readFile", exposed: "metadata" }, + { method: "readFile", exposed: "content" }, + { method: "listFiles", exposed: "metadata" }, + { method: "listChats", exposed: "metadata" }, + { method: "listActions", exposed: "content" }, + { method: "listGatekeepers", exposed: "metadata" }, + { method: "describe", exposed: "metadata" }, + { method: "getDiagnostics", exposed: "metadata" }, + { method: "getChatState", exposed: "metadata" }, + { method: "listChatAttachments", exposed: "metadata" }, + { method: "readChatAttachment", exposed: "content" }, + { method: "getCommitLog", exposed: "content" }, + { method: "listFilesAtCommit", exposed: "metadata" }, + { method: "readFileAtCommit", exposed: "content" }, + { method: "readFileAtCommit", exposed: "metadata" }, + { method: "listChangedPaths", exposed: "metadata" }, + { method: "listPendingChanges", exposed: "metadata" }, + { method: "readPendingFile", exposed: "content" }, + { method: "readPendingFile", exposed: "content" }, + { method: "listObservers", exposed: "metadata" }, + { method: "listObserverRequirements", exposed: "metadata" }, + { method: "listShareLinks", exposed: "metadata" }, + { method: "listAutoApprovals", exposed: "metadata" }, + { method: "listBlueprints", exposed: "metadata" }, + ]); + + const logged = (await workspaceRecords(id)).map(accessRecord); + expect(logged.at(-1)).toMatchObject({ method: "inspectWorkspace", exposed: "metadata" }); + const contentReads = (await admin.listAdminAudit({ workspace: id, exposed: "content" })) + .records; + expect(contentReads.map(record => accessRecord(record).method)).toEqual([ + "readPendingFile", "readPendingFile", "readFileAtCommit", "getCommitLog", + "readChatAttachment", "listActions", "readFile", "getChat", + ]); + expect(contentReads).toEqual(logged.filter(record => record.exposed === "content")); + const metadataReads = (await admin.listAdminAudit({ workspace: id, exposed: "metadata" })) + .records; + // The open, and every call above that answered with no content. + expect(metadataReads).toHaveLength(18); + expect(metadataReads).toEqual(logged.filter(record => record.exposed === "metadata")); + expect((await admin.listAdminAudit({ kind: "setting", exposed: "content" })).records) + .toEqual([]); + } finally { + await admin.setAdminWorkspaceAccess("metadata"); + } +}); + +it("names who made each connection, and no one for an ambient one", async () => { + using stack = new DisposableStack(); + const [ownerName] = nextUsernames("creator"); + const owner = stack.use(await signUp(stack.use(connect(harness.url)), ownerName!)); + // The test account is a singleton, which the owner's workspace takes as an ambient connection + // when it is opened, so the account comes first. + const account = await testAccount(owner); + const ws = stack.use(await owner.newWorkspace()); + const id = (await ws.getMetadata()).id; + const connection = await ws.newGatekeeper( + account.id, "https://gadgets-test.example/things/created"); + if (connection === null) throw new Error("The test connection was not created"); + stack.use(connection); + + using view = await admin.inspectWorkspace(id, REASON); + const gatekeepers = await waitFor("the ambient connection to be listed", async () => { + const listed = await view.listGatekeepers(); + if (listed.some(gatekeeper => gatekeeper.creationSpec?.type === "ambient")) return listed; + // Each open provides what is missing. + (await owner.openWorkspace(id))[Symbol.dispose](); + return null; + }); + expect(await newestRecord({ workspace: id })).toMatchObject({ + method: "listGatekeepers", access: "metadata", exposed: "metadata", + }); + + const made = gatekeepers.find(gatekeeper => gatekeeper.creationSpec?.type === "gatekeeper"); + expect(made).toMatchObject({ + id: await connection.getId(), createdBy: { type: "user", id: ownerName }, + }); + // No person creates an ambient connection. + const ambient = gatekeepers.find(gatekeeper => gatekeeper.creationSpec?.type === "ambient"); + expect(ambient).toMatchObject({ creationSpec: { type: "ambient", vendorId: TEST_VENDOR_ID } }); + expect(ambient?.createdBy).toBeUndefined(); +}); + +it("lists who holds an observer record, with what was verified for each", async () => { + using stack = new DisposableStack(); + const [ownerName, admittedName, refusedName] = nextUsernames("observed", "admitted", "refused"); + const signedUp = async (username: string) => + stack.use(await signUp(stack.use(connect(harness.url)), username)); + const owner = await signedUp(ownerName!); + const admitted = await signedUp(admittedName!); + const refused = await signedUp(refusedName!); + // Before the workspace exists, so that its ambient connection is there from its first open: one + // that appeared during a collaborator's open would restart the workspace under them. + const account = await testAccount(owner); + const admittedAccount = await testAccount(admitted); + + const ws = stack.use(await owner.newWorkspace()); + const id = (await ws.getMetadata()).id; + const resourceUrl = "https://gadgets-test.example/things/observed"; + const connection = await ws.newGatekeeper(account.id, resourceUrl); + if (connection === null) throw new Error("The test connection was not created"); + stack.use(connection); + const gatekeeperId = await connection.getId(); + const profileOf = async (username: string) => { + const added = await ws.addCollaborator(username, "workspace"); + if (added === null) throw new Error(`Failed to share the workspace with ${username}`); + return added.profile; + }; + const admittedProfile = await profileOf(admittedName!); + const refusedProfile = await profileOf(refusedName!); + + // The first member answers the one question, about the connection, with an account of their + // own, and is admitted. + const answers = new ObserverConfigRecorder().alwaysChoose(admittedAccount.id, 1); + stack.use(await admitted.openWorkspace(id, undefined, stack.use(stubFor(answers)))); + expect(answers.calls).toEqual([[expect.objectContaining({ gatekeeperId })]]); + // The second has no test account and no way to be asked for one, so their open is refused. + await expect(refused.openWorkspace(id)).rejects.toThrow("no configuration channel was provided"); + + using view = await admin.inspectWorkspace(id, REASON); + const metadataRead = async (method: string) => + expect(await newestRecord({ workspace: id }), method).toMatchObject({ + method, requested: "metadata", access: "metadata", exposed: "metadata", + }); + + // A record is reserved before anything is verified and outlives a refused open, so the listing + // holds both members: who holds a record, not who was admitted. + const observers = await view.listObservers(); + await metadataRead("listObservers"); + expect(observers).toHaveLength(2); + // The keys as they arrived, where a stored record's other fields would have travelled. + for (const observer of observers) { + expect(Object.keys(observer).toSorted()).toEqual(["accountChoices", "profile"]); + for (const choice of observer.accountChoices) { + expect(Object.keys(choice).toSorted()).toEqual(["accountId", "gatekeeperId", "vendorId"]); + } + } + expect(JSON.stringify(observers)).not.toContain("observerId"); + + const verified = observers.find(observer => observer.profile.id === admittedProfile.id); + expect(verified?.profile).toEqual(admittedProfile); + expect(verified?.accountChoices).toContainEqual({ + gatekeeperId, accountId: admittedAccount.id, vendorId: TEST_VENDOR_ID.toLowerCase(), + }); + // Their account is also taken, unasked, for the workspace's ambient connection. + expect(new Set(verified?.accountChoices.map(choice => choice.accountId))) + .toEqual(new Set([admittedAccount.id])); + expect(observers.find(observer => observer.profile.id === refusedProfile.id)) + .toEqual({ profile: refusedProfile, accountChoices: [] }); + + expect(await view.listObserverRequirements("workspace")).toContainEqual( + expect.objectContaining({ gatekeeperId, vendorId: TEST_VENDOR_ID, resourceUrl })); + await metadataRead("listObserverRequirements"); + expect((await view.getDiagnostics()).observers).toBe(2); + await metadataRead("getDiagnostics"); +}); + +it("refuses a workspace that does not exist without creating it", async () => { + using stack = new DisposableStack(); + const { owner, ownerName, ws, id } = await newWorkspace(stack, "deleted"); + + // The Overseer keeps its records in typed storage over the KV API, whose table _cf_KV an + // object's first write creates and deleteAll() drops, so whether the table exists says whether + // anything has been written since the deletion. Miniflare's own tables, which any visit creates, + // are left out, and _cf_KV itself cannot be read (SQLITE_AUTH), hence sqlite_master. + // listDurableObjectIds() cannot tell: it lists every object with a storage file, which any + // visit creates and a deletion leaves. + const tables = async () => { + const storage = await harness.server.getWorker("workshop-backend") + .getDurableObjectStorage("OverseerDurableObject", { id }); + const rows = await storage.exec<{ name: string }>(`SELECT name FROM sqlite_master + WHERE type = 'table' AND name NOT GLOB '__miniflare*' ORDER BY name`); + return rows.map(row => row.name); + }; + expect(await tables()).toContain("_cf_KV"); + + await ws.deleteSelf(); + const deletedAt = new Date(); + await settleRestart(); + const before = await tables(); + expect(before).not.toContain("_cf_KV"); + + for (let attempt = 0; attempt < 2; attempt++) { + expect(await refusalOf(admin.inspectWorkspace(id, REASON))) + .toBe(ADMIN_ACCESS_ERROR_CODES.workspaceNotFound); + } + expect(await tables()).toEqual(before); + expect((await admin.listWorkspaces({ owner: ownerName })).workspaces + .map(workspace => workspace.id)).not.toContain(id); + expect((await admin.listAdminAudit({ workspace: id, since: deletedAt })).records).toEqual([]); + + // open()'s first-open path refuses before it writes, too. + let openError: unknown; + try { + stack.use(await owner.openWorkspace(id)); + } catch (error) { + openError = error; + } + expect(getOpenWorkspaceErrorCode(openError)).toBe(OPEN_WORKSPACE_ERROR_CODES.workspaceNotFound); + expect(await tables()).toEqual(before); + + expect(await refusalOf(admin.inspectWorkspace("not-a-workspace-id", REASON))) + .toBe(ADMIN_ACCESS_ERROR_CODES.workspaceNotFound); +}); + +it("fails a view call whose audit record cannot be written, returning nothing", async () => { + using stack = new DisposableStack(); + const { id, chatId } = await populatedWorkspace(stack, "unaudited"); + // Addresses the deployment-wide log, which is always getByName(""). + const storage = await harness.server.getWorker("workshop-backend") + .getDurableObjectStorage("AdminAuditDurableObject", { name: "" }); + + try { + await admin.setAdminWorkspaceAccess("full"); + using view = await admin.inspectWorkspace(id, REASON, { access: "full" }); + const logged = await workspaceRecords(id); + expect(logged).toHaveLength(1); + + // Dropped before the setting is restored: no setting change can be recorded while it stands. + try { + await storage.exec(`CREATE TRIGGER admin_audit_unavailable BEFORE INSERT ON admin_audit + BEGIN SELECT RAISE(ABORT, 'admin audit unavailable'); END`); + await expect(view.listChats()).rejects.toThrow(); + await expect(view.getChat(chatId)).rejects.toThrow(); + await expect(admin.inspectWorkspace(id, REASON)).rejects.toThrow(); + } finally { + await storage.exec("DROP TRIGGER IF EXISTS admin_audit_unavailable"); + } + expect(await workspaceRecords(id)).toEqual(logged); + } finally { + await admin.setAdminWorkspaceAccess("metadata"); + } +}); + +it("stops a view once it expires, recording nothing more", async () => { + using stack = new DisposableStack(); + const { id } = await newWorkspace(stack, "expiring"); + + // Long enough for the open's hops (two config reads, the record) and describe's own on a slow + // runner, where a one-second view could expire before describe returns. + const lifetimeMs = 5_000; + const before = Date.now(); + using view = await admin.inspectWorkspace(id, REASON, { lifetimeSeconds: lifetimeMs / 1000 }); + const after = Date.now(); + const { expiresAt } = await view.describe(); + expect(expiresAt.getTime()).toBeGreaterThanOrEqual(before + lifetimeMs - 1_000); + expect(expiresAt.getTime()).toBeLessThanOrEqual(after + lifetimeMs + 1_000); + + const newest = await newestRecord({ workspace: id }); + expect(newest).toMatchObject({ method: "describe" }); + await waitFor("the view to expire", async () => + Date.now() > expiresAt.getTime() + 1_000 || null, lifetimeMs * 3); + expect(await refusalOf(view.listChats())).toBe(ADMIN_ACCESS_ERROR_CODES.viewExpired); + expect(await newestRecord({ workspace: id })).toEqual(newest); + + for (const lifetimeSeconds of [0, 3601, 1.5]) { + await expect(admin.inspectWorkspace(id, REASON, { lifetimeSeconds }), + `lifetime ${lifetimeSeconds}`).rejects.toThrow(); + } + expect(await newestRecord({ workspace: id })).toEqual(newest); +}); + +it("bounds long names and titles in the directory and the audit log", async () => { + using stack = new DisposableStack(); + const { owner, ownerName, ws, id } = await newWorkspace(stack, "longtext"); + const name = "N".repeat(5_000); + const title = "T".repeat(5_000); + await owner.setOwnDisplayName(name); + await ws.setTitle(title); + + await waitFor("the long name to be listed", async () => + (await admin.listUsers({ query: ownerName })).users.some(user => + user.id === ownerName && user.name === name.slice(0, MAX_DIRECTORY_TEXT_LENGTH)) || null); + const workspace = await waitFor("the long title to be listed", async () => + (await admin.listWorkspaces({ owner: ownerName })).workspaces.find(entry => + entry.id === id && entry.title === title.slice(0, MAX_DIRECTORY_TEXT_LENGTH) && + entry.owner.name === name.slice(0, MAX_DIRECTORY_TEXT_LENGTH)) ?? null); + expect(workspace.owner.id).toBe(ownerName); + + using view = await admin.inspectWorkspace(id, REASON); + expect((await view.describe()).title).toBe(title); + const cut = `${title.slice(0, MAX_ADMIN_AUDIT_ARG_LENGTH - 1)}…`; + const logged = await workspaceRecords(id); + for (const method of ["inspectWorkspace", "describe"]) { + expect(logged.find(record => record.method === method)).toMatchObject({ workspaceTitle: cut }); + } +}); diff --git a/packages/workshop-backend/AGENTS.md b/packages/workshop-backend/AGENTS.md index de3b07d023..dbbbcc6354 100644 --- a/packages/workshop-backend/AGENTS.md +++ b/packages/workshop-backend/AGENTS.md @@ -79,17 +79,24 @@ them: - Admin operations are exposed as an `AdminApi` capability obtained via `AuthenticatedApi.getAdminApi()` (returns null for non-admins). The admin check (`isDeploymentAdmin()` in `src/auth/config.ts`) runs once when the capability is minted, so the - individual methods don't re-check. -- Every `AdminApi` setter is recorded in the deployment-wide audit log (`AdminAuditDurableObject`, - `src/admin-audit.ts`) before it takes effect: each `AdminSettings` mutator takes a trailing - `AdminSettingActor` and writes through `#mutateAdminConfig()`, which records the diff before - storing it (the site logo and featured blueprints, which live outside `AdminConfig`, record - explicitly first). A change whose record cannot be written is refused. See - [docs/admin-access.md](../../docs/admin-access.md). -- The audit log's minimum retention, `ADMIN_AUDIT_RETENTION_MIN_DAYS`, is an env var beside - `ADMINS`, parsed in `src/admin-access.ts`, so that a compromised admin session cannot shorten it. - An invalid value falls back to 90 days. Admins choose a retention at or above it; - `adminAccessPolicy()` holds the choice there wherever it is enforced. + individual methods don't re-check, except for workspace inspection: `inspectWorkspace()`, + `OverseerDurableObject.openForAdmin()` and every `AdminWorkspaceView` call re-check + `isDeploymentAdmin()` and the view's expiry, and record each call, with what it returned, before + anything it read leaves the server. +- Every `AdminApi` setter is recorded in the deployment-wide audit log before it takes effect: each + `AdminSettings` mutator takes a trailing `AdminSettingActor` and writes through + `#mutateAdminConfig()`, which records the diff before storing it (the site logo and featured + blueprints, which live outside `AdminConfig`, record explicitly first). A change whose record + cannot be written is refused. +- The admin access bounds, `ADMIN_WORKSPACE_ACCESS_MAX` and `ADMIN_AUDIT_RETENTION_MIN_DAYS`, are + env vars beside `ADMINS`, parsed in `src/admin-access.ts`, so that a compromised admin session + cannot widen them. The cap fails closed (any value that is not a level is `off`), and an invalid + floor falls back to 90 days. Admins choose a level and a retention within them; + `adminAccessPolicy()` holds the choice inside them wherever it is enforced. +- Admin investigation rests on the `AdminAuditDurableObject` (`src/admin-audit.ts`, the + deployment-wide audit log) and the `UserDirectoryDurableObject` (`src/user-directory.ts`), which + lists users and workspaces for admins as well as serving user search, fed lazily by user DOs and + Overseers. See [docs/admin-access.md](../../docs/admin-access.md). - `user.ts:#assertResourceEnabled()` is the single core chokepoint where disabled gatekeepers/resources are enforced before a capability is minted. Both mints call it: `getGatekeeperClassFor()` (gadget/agent code can't reach it directly) and diff --git a/packages/workshop-backend/__tests__/admin-access.test.ts b/packages/workshop-backend/__tests__/admin-access.test.ts index 03356fb11c..12c6fe7993 100644 --- a/packages/workshop-backend/__tests__/admin-access.test.ts +++ b/packages/workshop-backend/__tests__/admin-access.test.ts @@ -1,11 +1,22 @@ import { describe, expect, it } from "vitest"; import { - DEFAULT_ADMIN_AUDIT_RETENTION_DAYS, MAX_ADMIN_AUDIT_ARG_LENGTH, MAX_ADMIN_AUDIT_RETENTION_DAYS, - MAX_ADMIN_AUDIT_VALUE_LENGTH, MAX_ANNOUNCEMENT_LENGTH, MAX_INSTANCE_INSTRUCTIONS_LENGTH, + ADMIN_ACCESS_ERROR_CODES, DEFAULT_ADMIN_AUDIT_RETENTION_DAYS, MAX_ADMIN_AUDIT_ARG_LENGTH, + MAX_ADMIN_AUDIT_RETENTION_DAYS, MAX_ADMIN_AUDIT_VALUE_LENGTH, MAX_ANNOUNCEMENT_LENGTH, + MAX_INSTANCE_INSTRUCTIONS_LENGTH, getAdminAccessErrorCode, + type ActionLogEntry, type AdminChatState, type AiChatHistoryPage, type AiChatMetadata, + type BlueprintGadgetSummary, type BoundHookInfo, type CommitInfo, type FileAtCommit, + type WorkpieceSummary, } from "@gadgets/workshop-shared/api"; import { - ADMIN_AUDIT_EXCERPT_CONTEXT, adminAccessPolicy, adminAuditStatsDays, adminAuditValue, - adminListLimit, adminSettingChange, cutAdminAuditText, minAdminAuditRetentionDays, + ADMIN_AUDIT_EXCERPT_CONTEXT, actionHoldsContent, adminAccessPolicy, adminAuditStatsDays, + adminAuditValue, adminCommitLogDepth, adminListLimit, adminSettingChange, adminViewCallAccess, + adminViewLifetimeMs, assertAdminViewLive, blueprintHoldsContent, chatHoldsContent, + chatPageHoldsContent, chatStateHoldsContent, commitHoldsContent, cutAdminAuditText, + fileHoldsContent, hookHoldsContent, lowerAdminWorkspaceAccess, maxAdminWorkspaceAccess, + minAdminAuditRetentionDays, redactAdminAuditArgs, validateAdminAccessReason, + withholdActionContent, withholdBlueprintContent, withholdChatContent, + withholdChatStateContent, withholdCommitContent, withholdHookContent, workpieceHoldsContent, + type AdminViewTerms, } from "../src/admin-access.js"; // The variables these helpers read, as a deployment's environment carries them: a secret or a @@ -14,6 +25,49 @@ function envWith(vars: Record): Cloudflare.Env { return vars as unknown as Cloudflare.Env; } +function codeOf(call: () => unknown): string | undefined { + try { + call(); + } catch (err) { + return getAdminAccessErrorCode(err); + } + throw new Error("expected a refusal"); +} + +const ADMIN = "admin@example.test"; +const adminEnv = envWith({ ADMINS: [ADMIN] }); + +function terms(overrides: Partial = {}): AdminViewTerms { + return { + via: "api", requested: "full", access: "full", expiresAt: new Date(Date.now() + 60_000), + ...overrides, + }; +} + +describe("maxAdminWorkspaceAccess", () => { + it("is metadata while unset or blank", () => { + expect(maxAdminWorkspaceAccess(envWith({}))).toBe("metadata"); + expect(maxAdminWorkspaceAccess(envWith({ ADMIN_WORKSPACE_ACCESS_MAX: "" }))).toBe("metadata"); + expect(maxAdminWorkspaceAccess(envWith({ ADMIN_WORKSPACE_ACCESS_MAX: " " }))).toBe("metadata"); + }); + + it("reads each level in any case, trimmed", () => { + expect(maxAdminWorkspaceAccess(envWith({ ADMIN_WORKSPACE_ACCESS_MAX: " Off " }))).toBe("off"); + expect(maxAdminWorkspaceAccess(envWith({ ADMIN_WORKSPACE_ACCESS_MAX: "METADATA" }))) + .toBe("metadata"); + expect(maxAdminWorkspaceAccess(envWith({ ADMIN_WORKSPACE_ACCESS_MAX: "\tFull\n" }))) + .toBe("full"); + }); + + // A value meant to turn inspection off must never grant the default. + it("fails closed to off for anything that is not a level", () => { + for (let value of ["none", "false", "of", "disabled", "1"]) { + expect(maxAdminWorkspaceAccess(envWith({ ADMIN_WORKSPACE_ACCESS_MAX: value })), value) + .toBe("off"); + } + }); +}); + describe("minAdminAuditRetentionDays", () => { it("defaults while unset or blank, and reads a positive whole number", () => { expect(minAdminAuditRetentionDays(envWith({}))).toBe(DEFAULT_ADMIN_AUDIT_RETENTION_DAYS); @@ -35,18 +89,44 @@ describe("minAdminAuditRetentionDays", () => { }); describe("adminAccessPolicy", () => { - it("holds the admins' choice within the environment's bound", () => { - let chosen = { adminAuditRetentionDays: 30 }; - expect(adminAccessPolicy(chosen, envWith({}))) - .toEqual({ retentionDays: DEFAULT_ADMIN_AUDIT_RETENTION_DAYS }); - expect(adminAccessPolicy(chosen, envWith({ ADMIN_AUDIT_RETENTION_MIN_DAYS: "7" }))) - .toEqual({ retentionDays: 30 }); - expect(adminAccessPolicy({ adminAuditRetentionDays: 400 }, envWith({}))) - .toEqual({ retentionDays: 400 }); + it("orders the levels", () => { + expect(lowerAdminWorkspaceAccess("full", "metadata")).toBe("metadata"); + expect(lowerAdminWorkspaceAccess("off", "full")).toBe("off"); + expect(lowerAdminWorkspaceAccess("metadata", "metadata")).toBe("metadata"); + }); + + it("holds the admins' choices within the environment's bounds", () => { + let full = { adminWorkspaceAccess: "full", adminAuditRetentionDays: 30 } as const; + expect(adminAccessPolicy(full, envWith({}))) + .toEqual({ access: "metadata", retentionDays: DEFAULT_ADMIN_AUDIT_RETENTION_DAYS }); + expect(adminAccessPolicy(full, envWith({ + ADMIN_WORKSPACE_ACCESS_MAX: "full", ADMIN_AUDIT_RETENTION_MIN_DAYS: "7", + }))).toEqual({ access: "full", retentionDays: 30 }); + expect(adminAccessPolicy(full, envWith({ ADMIN_WORKSPACE_ACCESS_MAX: "nope" })).access) + .toBe("off"); + expect(adminAccessPolicy({ adminWorkspaceAccess: "off", adminAuditRetentionDays: 400 }, + envWith({ ADMIN_WORKSPACE_ACCESS_MAX: "full" }))) + .toEqual({ access: "off", retentionDays: 400 }); }); }); describe("validators", () => { + it("validates the reason", () => { + expect(validateAdminAccessReason(" ticket 42 ")).toBe("ticket 42"); + expect(validateAdminAccessReason("x".repeat(500))).toHaveLength(500); + expect(() => validateAdminAccessReason(" ")).toThrow("at most 500 characters"); + expect(() => validateAdminAccessReason("x".repeat(501))).toThrow(TypeError); + }); + + it("validates the view's lifetime", () => { + expect(adminViewLifetimeMs(undefined)).toBe(900_000); + expect(adminViewLifetimeMs(1)).toBe(1000); + expect(adminViewLifetimeMs(3600)).toBe(3_600_000); + for (let bad of [0, 3601, 1.5, Number.NaN]) { + expect(() => adminViewLifetimeMs(bad), String(bad)).toThrow("from 1 to 3600"); + } + }); + it("validates the page size", () => { expect(adminListLimit(undefined)).toBe(50); expect(adminListLimit(200)).toBe(200); @@ -55,6 +135,17 @@ describe("validators", () => { } }); + it("validates the commit log's depth", () => { + expect(adminCommitLogDepth(undefined)).toBe(20); + expect(adminCommitLogDepth(1)).toBe(1); + expect(adminCommitLogDepth(100)).toBe(100); + for (let bad of [0, 101, 1.5, NaN]) { + expect(() => adminCommitLogDepth(bad)).toThrow(TypeError); + expect(() => adminCommitLogDepth(bad)) + .toThrow("The commit log's depth must be a whole number from 1 to 100."); + } + }); + it("validates the statistics window", () => { expect(adminAuditStatsDays(undefined)).toBe(7); expect(adminAuditStatsDays(366)).toBe(366); @@ -62,6 +153,56 @@ describe("validators", () => { }); }); +describe("view calls", () => { + it("refuses an expired view before checking the admin", () => { + let expired = terms({ expiresAt: new Date(1000) }); + expect(codeOf(() => assertAdminViewLive(expired, "someone-else", adminEnv, 1000))) + .toBe(ADMIN_ACCESS_ERROR_CODES.viewExpired); + expect(codeOf(() => assertAdminViewLive(terms(), "someone-else", adminEnv))) + .toBe(ADMIN_ACCESS_ERROR_CODES.notAdmin); + expect(() => assertAdminViewLive(terms(), ADMIN, adminEnv)).not.toThrow(); + }); + + it("runs each call at the view's level lowered to the policy's", () => { + let metadata = { access: "metadata", retentionDays: 90 } as const; + expect(adminViewCallAccess(terms(), "describe", metadata)).toEqual({ access: "metadata" }); + expect(adminViewCallAccess(terms(), "getChat", metadata)) + .toEqual({ access: "metadata", refused: ADMIN_ACCESS_ERROR_CODES.contentWithheld }); + expect(adminViewCallAccess(terms({ access: "metadata" }), "readFile", + { access: "full", retentionDays: 90 })) + .toEqual({ access: "metadata", refused: ADMIN_ACCESS_ERROR_CODES.contentWithheld }); + expect(adminViewCallAccess(terms(), "getChat", { access: "full", retentionDays: 90 })) + .toEqual({ access: "full" }); + expect(adminViewCallAccess(terms(), "readChatAttachment", metadata)) + .toEqual({ access: "metadata", refused: ADMIN_ACCESS_ERROR_CODES.contentWithheld }); + expect(adminViewCallAccess(terms(), "readChatAttachment", + { access: "full", retentionDays: 90 })) + .toEqual({ access: "full" }); + // The chat reads that are not content run at either level. + for (let method of ["getChatState", "listChatAttachments"] as const) { + expect(adminViewCallAccess(terms(), method, metadata)).toEqual({ access: "metadata" }); + } + // A file read at a stored commit, or of a chat's pending content, is content as readFile() is. + for (let method of ["readFileAtCommit", "readPendingFile"] as const) { + expect(adminViewCallAccess(terms(), method, metadata)) + .toEqual({ access: "metadata", refused: ADMIN_ACCESS_ERROR_CODES.contentWithheld }); + expect(adminViewCallAccess(terms(), method, { access: "full", retentionDays: 90 })) + .toEqual({ access: "full" }); + } + // The history and the listings of names run at either level: a commit log withholds its + // messages rather than being refused. + for (let method of [ + "getCommitLog", "listFilesAtCommit", "listChangedPaths", "listPendingChanges", + ] as const) { + expect(adminViewCallAccess(terms(), method, metadata)).toEqual({ access: "metadata" }); + expect(adminViewCallAccess(terms(), method, { access: "full", retentionDays: 90 })) + .toEqual({ access: "full" }); + } + expect(adminViewCallAccess(terms(), "describe", { access: "off", retentionDays: 90 })) + .toEqual({ access: "off", refused: ADMIN_ACCESS_ERROR_CODES.accessOff }); + }); +}); + describe("cutAdminAuditText", () => { it("keeps text at or below the limit whole", () => { expect(cutAdminAuditText("abc", 3)).toBe("abc"); @@ -86,6 +227,219 @@ describe("cutAdminAuditText", () => { expect(cut).toHaveLength(MAX_ADMIN_AUDIT_ARG_LENGTH); expect(cut.endsWith("…")).toBe(true); }); + + it("is the cut redactAdminAuditArgs() applies", () => { + let long = "p".repeat(1000); + expect(redactAdminAuditArgs({ path: long, chatId: 3, beforeSequence: undefined })).toEqual({ + path: cutAdminAuditText(long, MAX_ADMIN_AUDIT_ARG_LENGTH), + chatId: 3, + }); + }); +}); + +describe("withholding content", () => { + it("drops queued steers from a copy", () => { + let meta = { + id: 1, title: "t", started: new Date(0), lastActive: new Date(0), + queuedSteers: [{ text: "secret", author: { id: "u", name: "U" } }], + } as AiChatMetadata; + let withheld = withholdChatContent(meta); + expect(withheld).not.toHaveProperty("queuedSteers"); + expect(withheld.title).toBe("t"); + expect(meta.queuedSteers).toHaveLength(1); + }); + + it("empties descriptions and drops fields from a copy", () => { + let fields = [{ label: "To", value: "x@example.com" }]; + let action = { + id: 1, resourceTitle: "r", createdAt: new Date(0), state: "pending", type: "action", + description: { title: "Send", description: "body", fields }, + } as unknown as ActionLogEntry; + let observation = { + id: 2, resourceTitle: "r", createdAt: new Date(0), state: "approved", type: "observation", + description: { title: "Read", description: "content", fields }, + } as unknown as ActionLogEntry; + let hookEntry = { + id: 3, resourceTitle: "r", createdAt: new Date(0), state: "approved", type: "bindHook", + enabled: true, description: { title: "Hook", description: "what it does" }, + } as unknown as ActionLogEntry; + + for (let entry of [action, observation, hookEntry]) { + let before = structuredClone(entry); + let withheld = withholdActionContent(entry); + expect(withheld.description.title).toBe(entry.description.title); + expect(withheld.description.description).toBe(""); + expect(withheld.description).not.toHaveProperty("fields"); + expect(entry).toEqual(before); + } + + let hook = { + id: 1, gatekeeperId: 2, gadgetId: 3, enabled: true, + description: { title: "Hook", description: "what it does" }, + } as unknown as BoundHookInfo; + expect(withholdHookContent(hook).description).toEqual({ title: "Hook", description: "" }); + expect(hook.description.description).toBe("what it does"); + }); + + it("empties each queued call's argument summary on a copy", () => { + let state = chatState(["first summary", "second summary"]); + let before = structuredClone(state); + expect(withholdChatStateContent(state)).toStrictEqual({ + ...before, + queuedCalls: before.queuedCalls.map(call => ({ ...call, argsSummary: "" })), + }); + expect(state).toStrictEqual(before); + }); + + it("empties a commit's message on a copy", () => { + let commit = commitInfo("Fix the parser\n"); + let before = structuredClone(commit); + expect(withholdCommitContent(commit)).toStrictEqual({ ...before, message: "" }); + expect(commit).toStrictEqual(before); + }); + + it("empties a blueprint's id and drops its screenshot URL on a copy", () => { + let blueprint = blueprintSummary("bp-1"); + let before = structuredClone(blueprint); + let { screenshotUrl: _screenshotUrl, ...rest } = before; + expect(withholdBlueprintContent(blueprint)).toStrictEqual({ ...rest, id: "" }); + expect(blueprint).toStrictEqual(before); + }); +}); + +// A published blueprint with a screenshot, whose URL holds its id. +function blueprintSummary(id: string): BlueprintGadgetSummary { + return { + id, title: "Kit", description: "A kit", version: 2, codeVersionDate: new Date(1700000000_000), + screenshotUrl: `/blueprint-screenshot/${id}?v=1`, dirty: true, unpublishedChanges: true, + }; +} + +// A commit with one parent. +function commitInfo(message: string): CommitInfo { + return { + oid: "1".repeat(40), parents: ["2".repeat(40)], message, + author: { name: "Owner", email: "owner@example.test" }, timestamp: new Date(1700000000_000), + }; +} + +// A chat's state with a recorded turn, a pending action and one queued call per summary. +function chatState(summaries: string[]): AdminChatState { + return { + turn: { + modelId: "m", initiator: { type: "user", id: "u", name: "U" }, callbackInitiated: false, + resumeAttempts: 1, paused: true, + }, + lastMessageAt: new Date(0), + queuedCalls: summaries.map((argsSummary, callId) => + ({ callId, methodName: "run", initiatorModelId: null, argsSummary })), + pendingActionIds: [4], + }; +} + +// Answers as a view returns them, for the *HoldsContent() predicates. +function chatMeta(extra: Partial = {}): AiChatMetadata { + return { id: 1, title: "t", started: new Date(0), lastActive: new Date(0), ...extra } as + AiChatMetadata; +} +function logEntry(type: ActionLogEntry["type"], description: object): ActionLogEntry { + return { + id: 1, resourceTitle: "r", createdAt: new Date(0), state: "approved", type, + ...(type === "bindHook" ? { enabled: true } : {}), description, + } as unknown as ActionLogEntry; +} +function boundHook(description: string): BoundHookInfo { + return { + id: 1, gatekeeperId: 2, gadgetId: 3, enabled: true, + description: { title: "Hook", description }, + } as unknown as BoundHookInfo; +} +function gadgetSummary(extra: Partial = {}): WorkpieceSummary { + return { id: "g", type: "gadget", title: "G", ...extra } as WorkpieceSummary; +} +const STEERS = [{ text: "secret", author: { id: "u", name: "U" } }]; +const FIELDS = [{ label: "To", value: "x@example.com" }]; + +describe("whether an answer holds content", () => { + it("finds a queued steer in a chat", () => { + let full = chatMeta({ queuedSteers: STEERS } as Partial); + expect(chatHoldsContent(full)).toBe(true); + expect(chatHoldsContent(chatMeta({ queuedSteers: [] }))).toBe(false); + expect(chatHoldsContent(chatMeta())).toBe(false); + expect(chatHoldsContent(withholdChatContent(full))).toBe(false); + }); + + it("finds a queued call's argument summary in a chat's state", () => { + let full = chatState(["", "summary"]); + expect(chatStateHoldsContent(full)).toBe(true); + expect(chatStateHoldsContent(chatState([]))).toBe(false); + expect(chatStateHoldsContent(chatState(["", ""]))).toBe(false); + expect(chatStateHoldsContent(withholdChatStateContent(full))).toBe(false); + }); + + it("finds a commit's message", () => { + let full = commitInfo("Fix the parser\n"); + expect(commitHoldsContent(full)).toBe(true); + expect(commitHoldsContent(commitInfo(""))).toBe(false); + expect(commitHoldsContent(withholdCommitContent(full))).toBe(false); + }); + + it("finds a description or a field in an action log entry", () => { + for (let type of ["action", "observation"] as const) { + let full = logEntry(type, { title: "T", description: "body", fields: FIELDS }); + expect(actionHoldsContent(full)).toBe(true); + expect(actionHoldsContent(logEntry(type, { title: "T", description: "", fields: FIELDS }))) + .toBe(true); + expect(actionHoldsContent(logEntry(type, { title: "T", description: "" }))).toBe(false); + expect(actionHoldsContent(logEntry(type, { title: "T", description: "", fields: [] }))) + .toBe(false); + expect(actionHoldsContent(withholdActionContent(full))).toBe(false); + } + let bound = logEntry("bindHook", { title: "Hook", description: "what it does" }); + expect(actionHoldsContent(bound)).toBe(true); + expect(actionHoldsContent(logEntry("bindHook", { title: "Hook", description: "" }))) + .toBe(false); + expect(actionHoldsContent(withholdActionContent(bound))).toBe(false); + }); + + it("finds a blueprint's id", () => { + let full = blueprintSummary("bp-1"); + expect(blueprintHoldsContent(full)).toBe(true); + expect(blueprintHoldsContent(blueprintSummary(""))).toBe(false); + expect(blueprintHoldsContent(withholdBlueprintContent(full))).toBe(false); + }); + + it("finds a hook's description", () => { + expect(hookHoldsContent(boundHook("what it does"))).toBe(true); + expect(hookHoldsContent(boundHook(""))).toBe(false); + expect(hookHoldsContent(withholdHookContent(boundHook("what it does")))).toBe(false); + }); + + it("finds the blueprint a gadget follows", () => { + let upstream = { blueprintId: "bp-1", commitId: "c" }; + expect(workpieceHoldsContent(gadgetSummary({ upstream }))).toBe(true); + expect(workpieceHoldsContent(gadgetSummary({ upstream: {} }))).toBe(false); + expect(workpieceHoldsContent(gadgetSummary())).toBe(false); + expect(workpieceHoldsContent({ id: "w", type: "worktree", title: "W" } as WorkpieceSummary)) + .toBe(false); + }); + + it("finds messages or a compaction summary in a chat page", () => { + let message = { sequence: 1 } as unknown as AiChatHistoryPage["messages"][number]; + let compacted = { to: 1, summary: "s" } as AiChatHistoryPage["compacted"]; + expect(chatPageHoldsContent({ messages: [message] })).toBe(true); + expect(chatPageHoldsContent({ messages: [], compacted })).toBe(true); + expect(chatPageHoldsContent({ messages: [] })).toBe(false); + expect(chatPageHoldsContent({ messages: [], compacted: undefined })).toBe(false); + }); + + it("finds any file at the path, an unreadable one too", () => { + let files: FileAtCommit[] = [ + { kind: "text", text: "x" }, { kind: "unreadable", message: "a is a symlink to b" }, + { kind: "absent" }, + ]; + expect(files.map(fileHoldsContent)).toEqual([true, true, false]); + }); }); describe("adminAuditValue", () => { diff --git a/packages/workshop-backend/__tests__/admin-api-impl.test.ts b/packages/workshop-backend/__tests__/admin-api-impl.test.ts index 3b6944b29b..446b3f5d0f 100644 --- a/packages/workshop-backend/__tests__/admin-api-impl.test.ts +++ b/packages/workshop-backend/__tests__/admin-api-impl.test.ts @@ -1,17 +1,30 @@ import { describe, expect, it } from "vitest"; -import { type AdminVia } from "@gadgets/workshop-shared/api"; +import { env } from "cloudflare:workers"; +import { + ADMIN_ACCESS_ERROR_CODES, DEFAULT_ADMIN_VIEW_LIFETIME_SECONDS, MAX_ADMIN_AUDIT_ARG_LENGTH, + createAdminAccessError, getAdminAccessErrorCode, + type AdminAccessRecord, type AdminVia, type AdminViewAccess, +} from "@gadgets/workshop-shared/api"; import { AdminApiImpl } from "../src/admin-settings.js"; +import type { OverseerDurableObject } from "../src/overseer.js"; import { DEFAULT_ADMIN_CONFIG, type AdminConfig, } from "../src/storage-schema/admin-settings-storage.js"; +declare module "cloudflare:workers" { + interface ProvidedEnv { + TEST_OVERSEER: DurableObjectNamespace; + } +} + const ADMIN = "admin@example.test"; +const REASON = "Investigating a support ticket"; type Call = { target: string; method: string; args: unknown[] }; type Handler = (...args: any[]) => unknown; /** - * An AdminApiImpl over fake namespaces, with `vars` as its environment. `calls` + * An AdminApiImpl over fake namespaces, with ADMINS listing ADMIN and `vars` beside it. `calls` * lists every namespace and stub call made after the capability was minted, in order; a stub * method answers from `handlers[target][method]`, else with undefined. The AdminSettings stub * answers committedAdminConfig() with `config` over the defaults. @@ -43,9 +56,20 @@ function deployment(options: { }); const ctx = { exports: { AdminSettings: namespace("admin"), + UserDirectoryDurableObject: namespace("directory"), AdminAuditDurableObject: namespace("audit"), + OverseerDurableObject: { + idFromString: (id: string) => { + calls.push({ target: "overseers", method: "idFromString", args: [id] }); + return env.TEST_OVERSEER.idFromString(id); + }, + get: (id: DurableObjectId) => { + calls.push({ target: "overseers", method: "get", args: [id.toString()] }); + return stub("overseer"); + }, + }, } } as unknown as ExecutionContext; - const deploymentEnv = { ...options.vars } as unknown as Cloudflare.Env; + const deploymentEnv = { ADMINS: [ADMIN], ...options.vars } as unknown as Cloudflare.Env; const mint = (adminUserId = ADMIN, via: AdminVia = "cli") => { const api = new AdminApiImpl(ctx, deploymentEnv, adminUserId, via); calls.length = 0; @@ -65,6 +89,191 @@ async function rejection(call: Promise): Promise { throw new Error("Expected the call to reject."); } +const workspaceId = () => env.TEST_OVERSEER.newUniqueId().toString(); + +describe("AdminApiImpl.inspectWorkspace", () => { + it("rejects a bad reason or lifetime before touching anything", async () => { + const { mint, calls } = deployment(); + const api = mint(); + const id = workspaceId(); + for (let reason of ["", " ", "r".repeat(501)]) { + expect(await rejection(api.inspectWorkspace(id, reason))).toBeInstanceOf(TypeError); + } + for (let lifetimeSeconds of [0, -1, 1.5, 3601]) { + expect(await rejection(api.inspectWorkspace(id, REASON, { lifetimeSeconds }))) + .toBeInstanceOf(TypeError); + } + expect(calls).toEqual([]); + }); + + it("never runs with a level that is not \"metadata\" or \"full\"", () => { + // The validator refuses it before the method's body runs: synchronously, on a direct call. + const { mint, calls } = deployment(); + const api = mint(); + const id = workspaceId(); + expect(() => api.inspectWorkspace(id, REASON, { access: "off" as any })).toThrow(TypeError); + expect(() => api.inspectWorkspace(id, REASON, { access: "everything" as any })) + .toThrow(TypeError); + expect(calls).toEqual([]); + }); + + it("refuses a caller no longer an admin before touching anything", async () => { + const { mint, calls } = deployment(); + const error = await rejection( + mint("former@example.test").inspectWorkspace(workspaceId(), REASON)); + expect(getAdminAccessErrorCode(error)).toBe(ADMIN_ACCESS_ERROR_CODES.notAdmin); + expect(calls).toEqual([]); + }); + + it("refuses a malformed id as not found, recording nothing", async () => { + const { mint, calls } = deployment({ config: { adminWorkspaceAccess: "full" } }); + const error = await rejection(mint().inspectWorkspace("not-a-workspace", REASON)); + expect(getAdminAccessErrorCode(error)).toBe(ADMIN_ACCESS_ERROR_CODES.workspaceNotFound); + expect(calls.map(call => `${call.target}.${call.method}`)).toEqual(["overseers.idFromString"]); + }); + + describe.each([ + ["the authoritative config", { config: { adminWorkspaceAccess: "off" as const } }], + ["ADMIN_WORKSPACE_ACCESS_MAX off", { vars: { ADMIN_WORKSPACE_ACCESS_MAX: "off" } }], + ["ADMIN_WORKSPACE_ACCESS_MAX not a level", { vars: { ADMIN_WORKSPACE_ACCESS_MAX: "none" } }], + ])("while access is off by %s", (_label, setup) => { + const offDeployment = (listing: Handler, append: Handler = () => {}) => deployment({ + ...setup, + handlers: { directory: { workspaceListing: listing }, audit: { append } }, + }); + + it("records the refused open in the deployment-wide log only, then refuses", async () => { + const { mint, callsTo } = offDeployment(() => ({ title: "Quarterly plan", ownerId: "ada" })); + const api = mint(); + const id = workspaceId(); + const before = Date.now(); + const error = await rejection(api.inspectWorkspace(id, ` ${REASON} `, { + lifetimeSeconds: 60, + })); + const after = Date.now(); + expect(getAdminAccessErrorCode(error)).toBe(ADMIN_ACCESS_ERROR_CODES.accessOff); + + const appends = callsTo("audit", "append"); + expect(appends).toHaveLength(1); + const [record, retentionDays] = appends[0]!.args as [AdminAccessRecord, number]; + expect(record).toMatchObject({ + kind: "workspace", admin: ADMIN, via: "cli", workspaceId: id, workspaceOwner: "ada", + workspaceTitle: "Quarterly plan", method: "inspectWorkspace", reason: REASON, + requested: "metadata", access: "off", exposed: "metadata", + refused: ADMIN_ACCESS_ERROR_CODES.accessOff, + }); + expect(Object.keys(record.args)).toEqual(["expiresAt"]); + const expiresAt = Date.parse(record.args.expiresAt as string); + expect(expiresAt).toBeGreaterThanOrEqual(before + 60_000); + expect(expiresAt).toBeLessThanOrEqual(after + 60_000); + // No retention: the directory read awaits, so the one read before it may be stale. + expect(retentionDays).toBeUndefined(); + expect(callsTo("overseers", "get")).toEqual([]); + + // The same capability's records share a session id. + await rejection(api.inspectWorkspace(id, REASON)); + const [second] = callsTo("audit", "append")[1]!.args as [AdminAccessRecord]; + expect(second.sessionId).toBe(record.sessionId); + + // A level asked for is recorded too, refused alike. + await rejection(api.inspectWorkspace(id, REASON, { access: "full" })); + const [third] = callsTo("audit", "append")[2]!.args as [AdminAccessRecord]; + expect(third).toMatchObject({ requested: "full", access: "off" }); + }); + + it("cuts a long title as the audit log cuts any", async () => { + const { mint, callsTo } = offDeployment(() => ({ title: "t".repeat(1000), ownerId: "ada" })); + await rejection(mint().inspectWorkspace(workspaceId(), REASON)); + const [record] = callsTo("audit", "append")[0]!.args as [AdminAccessRecord]; + expect(record.workspaceTitle).toBe(`${"t".repeat(MAX_ADMIN_AUDIT_ARG_LENGTH - 1)}…`); + }); + + it.each([ + ["lists none", () => null], + ["cannot be asked", () => { throw new Error("directory unavailable"); }], + ])("records an empty title and owner when the directory %s", async (_case, listing) => { + const { mint, callsTo } = offDeployment(listing); + const error = await rejection(mint().inspectWorkspace(workspaceId(), REASON)); + expect(getAdminAccessErrorCode(error)).toBe(ADMIN_ACCESS_ERROR_CODES.accessOff); + const [record] = callsTo("audit", "append")[0]!.args as [AdminAccessRecord]; + expect(record.workspaceTitle).toBe(""); + expect(record.workspaceOwner).toBe(""); + }); + + it("fails with the audit log's own error when the refusal cannot be recorded", async () => { + const { mint, callsTo } = offDeployment(() => ({ title: "Plan", ownerId: "ada" }), () => { + throw new Error("audit log unavailable"); + }); + const error = await rejection(mint().inspectWorkspace(workspaceId(), REASON)); + expect(String(error)).toContain("audit log unavailable"); + expect(callsTo("overseers", "get")).toEqual([]); + }); + }); + + it.each([ + ["full", "full", undefined, "metadata"], + ["full", "full", "full", "full"], + ["full", undefined, "full", "metadata"], + ["metadata", "full", "full", "metadata"], + ["metadata", "full", "metadata", "metadata"], + ] as const)("opens a view at the lowest of %s, the cap %s and the level asked for, %s", + async (chosen, max, requested, access) => { + // The view comes back wrapped in its validator, so it is told apart by what it answers. + const view = { describe: async () => "the minted view" }; + const { mint, callsTo } = deployment({ + config: { adminWorkspaceAccess: chosen }, + vars: max === undefined ? {} : { ADMIN_WORKSPACE_ACCESS_MAX: max }, + handlers: { overseer: { openForAdmin: () => view } }, + }); + const id = workspaceId(); + const before = Date.now(); + const minted = await mint(ADMIN, "api").inspectWorkspace( + id, ` ${REASON} `, requested === undefined ? undefined : { access: requested }); + expect(await minted.describe()).toBe("the minted view"); + const after = Date.now(); + + expect(callsTo("overseers", "get").map(call => call.args)).toEqual([[id]]); + const opens = callsTo("overseer", "openForAdmin"); + expect(opens).toHaveLength(1); + const [adminId, reason, sessionId, terms] = opens[0]!.args as + [string, string, string, + { via: string; requested: AdminViewAccess; access: AdminViewAccess; expiresAt: Date }]; + expect([adminId, reason, typeof sessionId]).toEqual([ADMIN, REASON, "string"]); + expect(terms.via).toBe("api"); + expect(terms.requested).toBe(requested ?? "metadata"); + expect(terms.access).toBe(access); + const lifetime = DEFAULT_ADMIN_VIEW_LIFETIME_SECONDS * 1000; + expect(terms.expiresAt.getTime()).toBeGreaterThanOrEqual(before + lifetime); + expect(terms.expiresAt.getTime()).toBeLessThanOrEqual(after + lifetime); + expect(callsTo("audit", "append")).toEqual([]); + }); + + it.each([ + ["succeeds", () => {}], + ["fails", () => { throw new Error("directory unavailable"); }], + ])("drops a workspace found missing from the directory, if that %s", async (_case, remove) => { + const { mint, callsTo } = deployment({ handlers: { + overseer: { openForAdmin: () => { + throw createAdminAccessError(ADMIN_ACCESS_ERROR_CODES.workspaceNotFound); + } }, + directory: { removeWorkspace: remove }, + } }); + const id = workspaceId(); + const error = await rejection(mint().inspectWorkspace(id, REASON)); + expect(getAdminAccessErrorCode(error)).toBe(ADMIN_ACCESS_ERROR_CODES.workspaceNotFound); + expect(callsTo("directory", "removeWorkspace").map(call => call.args)).toEqual([[id]]); + }); + + it("passes an expired view's refusal through, leaving the directory alone", async () => { + const { mint, calls } = deployment({ handlers: { overseer: { openForAdmin: () => { + throw createAdminAccessError(ADMIN_ACCESS_ERROR_CODES.viewExpired); + } } } }); + const error = await rejection(mint().inspectWorkspace(workspaceId(), REASON)); + expect(getAdminAccessErrorCode(error)).toBe(ADMIN_ACCESS_ERROR_CODES.viewExpired); + expect(calls.filter(call => call.target.startsWith("director"))).toEqual([]); + }); +}); + describe("AdminApiImpl settings", () => { it("says who changes a setting, through which method and session", async () => { const { mint, callsTo } = deployment(); @@ -90,6 +299,50 @@ describe("AdminApiImpl settings", () => { }); }); +describe("AdminApiImpl directory listings", () => { + it("lists users with whether each is an admin, forwarding only the query's fields", async () => { + const cursor = "c".repeat(300_000); + const { mint, callsTo } = deployment({ handlers: { directory: { listUsers: () => ({ + users: [{ id: ADMIN, name: "Admin" }, { id: "user", name: "User" }], nextCursor: "user", + }) } } }); + const query = { query: "a", cursor, limit: 2, smuggled: true }; + const page = await mint().listUsers(query as Parameters[0]); + expect(page).toEqual({ + users: [ + { id: ADMIN, name: "Admin", isAdmin: true }, + { id: "user", name: "User", isAdmin: false }, + ], + nextCursor: "user", + }); + const [forwarded] = callsTo("directory", "listUsers")[0]!.args as [Record]; + expect(forwarded).toStrictEqual({ query: "a", cursor, limit: 2 }); + }); + + it("lists workspaces, forwarding only the query's fields", async () => { + const owner = "o".repeat(300_000); + const cursor = "c".repeat(300_000); + const listed = { workspaces: [] }; + const { mint, callsTo } = deployment( + { handlers: { directory: { listWorkspaces: () => listed } } }); + expect(await mint().listWorkspaces( + { owner, cursor, smuggled: 1 } as Parameters[0])) + .toBe(listed); + const [query] = callsTo("directory", "listWorkspaces")[0]!.args as [Record]; + expect(query).toStrictEqual({ owner, query: undefined, cursor, limit: 50 }); + }); + + it("rejects a page size out of range before asking the directory", async () => { + const { mint, calls } = deployment(); + const api = mint(); + for (let limit of [0, 201, 1.5]) { + expect(await rejection(api.listUsers({ limit }))).toBeInstanceOf(TypeError); + expect(await rejection(api.listWorkspaces({ limit }))).toBeInstanceOf(TypeError); + expect(await rejection(api.listAdminAudit({ limit }))).toBeInstanceOf(TypeError); + } + expect(calls).toEqual([]); + }); +}); + describe("AdminApiImpl audit log", () => { it("forwards a listing's filters, with the retention the environment holds up", async () => { const page = { records: [] }; @@ -100,23 +353,18 @@ describe("AdminApiImpl audit log", () => { }); const since = new Date("2026-01-01T00:00:00Z"); expect(await mint().listAdminAudit({ - kind: "setting", admin: ADMIN, method: "setSiteName", since, cursor: "next", extra: true, + kind: "workspace", admin: ADMIN, method: "getChat", workspace: "w", owner: "ada", + exposed: "content", since, cursor: "next", extra: true, } as Parameters[0])).toBe(page); expect(callsTo("audit", "list")[0]!.args).toStrictEqual([ - { kind: "setting", admin: ADMIN, method: "setSiteName", since, cursor: "next", limit: 50 }, + { + kind: "workspace", admin: ADMIN, method: "getChat", workspace: "w", owner: "ada", + exposed: "content", since, cursor: "next", limit: 50, + }, 120, ]); }); - it("rejects a page size out of range before asking the log", async () => { - const { mint, calls } = deployment(); - const api = mint(); - for (let limit of [0, 201, 1.5]) { - expect(await rejection(api.listAdminAudit({ limit }))).toBeInstanceOf(TypeError); - } - expect(calls).toEqual([]); - }); - it("forwards a statistics window, 7 days by default", async () => { const stats = { days: 7 }; const { mint, callsTo } = deployment({ diff --git a/packages/workshop-backend/__tests__/admin-audit.test.ts b/packages/workshop-backend/__tests__/admin-audit.test.ts index 8d2798c8ab..69872c1ded 100644 --- a/packages/workshop-backend/__tests__/admin-audit.test.ts +++ b/packages/workshop-backend/__tests__/admin-audit.test.ts @@ -1,7 +1,10 @@ import { env } from "cloudflare:workers"; import { runInDurableObject } from "cloudflare:test"; import { describe, expect, it } from "vitest"; -import { type AdminAuditRecord, type AdminSettingRecord } from "@gadgets/workshop-shared/api"; +import { + MAX_ADMIN_AUDIT_ARG_LENGTH, + type AdminAccessRecord, type AdminAuditRecord, type AdminSettingRecord, +} from "@gadgets/workshop-shared/api"; import type { AdminAuditDurableObject } from "../src/admin-audit.js"; import { utcDayKey } from "../src/ai-gateway-billing/limits/config.js"; // Load the whole backend (the pool's `main`) up front, so its slow load does not count against the @@ -26,6 +29,15 @@ function daysAgo(days: number): Date { return new Date(Date.now() - days * DAY_MS); } +function access(overrides: Partial = {}): AdminAccessRecord { + return { + kind: "workspace", at: new Date(), admin: "ada", sessionId: "session-1", via: "api", + workspaceId: "w1", workspaceOwner: "owner-1", workspaceTitle: "Plans", method: "describe", + args: {}, reason: "ticket 1", + requested: "metadata", access: "metadata", exposed: "metadata", ...overrides, + }; +} + function setting(overrides: Partial = {}): AdminSettingRecord { return { kind: "setting", at: new Date(), admin: "ada", sessionId: "session-1", via: "cli", @@ -52,14 +64,6 @@ function sql>(stub: Stub, query: strin state.storage.sql.exec(query, ...params).toArray()); } -// A workspace access row written as of now, as the stacked inspection change appends them, so that -// a kind filter has another kind to leave out. -function insertAccessRow(stub: Stub, admin: string): Promise { - return sql(stub, `INSERT INTO admin_audit - (at, kind, admin, workspace_id, workspace_title, exposed, record) - VALUES (?, 'workspace', ?, 'w1', 'Plans', 'metadata', '{}')`, Date.now(), admin); -} - // A raw row, past append()'s shaping, to test the table's own CHECKs. function insertRow(stub: Stub, kind: string, workspaceId: string | null, title: string | null, exposed: string | null): Promise { @@ -71,14 +75,12 @@ function insertRow(stub: Stub, kind: string, workspaceId: string | null, title: describe("AdminAuditDurableObject.list", () => { it("lists newest first, and returns each record as appended", async () => { let log = auditLog(); - let first = setting({ at: daysAgo(2) }); - let second = setting({ at: daysAgo(1), method: "setSignupsEnabled", - changes: [{ setting: "signupsEnabled", before: "true", after: "false" }] }); - let third = setting({ method: "setBanner", changes: [ - { setting: "banner", key: "message", after: "Maintenance tonight" }, - { setting: "banner", key: "color", before: "blue", after: "red" }, - ] }); - let fourth = setting({ via: "api", sessionId: "session-2" }); + let first = access({ at: daysAgo(2), args: { chatId: 4 }, refused: "ADMIN_CONTENT_WITHHELD" }); + let second = setting({ at: daysAgo(1) }); + let third = access({ workspaceId: "w2", method: "inspectWorkspace" }); + let fourth = access({ + method: "getChat", args: { chatId: 4 }, requested: "full", access: "full", exposed: "content", + }); for (let record of [first, second, third, fourth]) await log.append(record, RETENTION); let page = await log.list({ limit: 10 }, RETENTION); @@ -88,38 +90,69 @@ describe("AdminAuditDurableObject.list", () => { it("stores each field once", async () => { let log = auditLog(); + await log.append(access(), RETENTION); await log.append(setting(), RETENTION); let rows = await sql< { kind: string; workspace_id: string | null; exposed: string | null; record: string }>(log, "SELECT kind, workspace_id, exposed, record FROM admin_audit ORDER BY seq"); expect(rows.map(row => [row.kind, row.workspace_id, row.exposed])) - .toEqual([["setting", null, null]]); - let json = JSON.parse(rows[0].record); - for (let field of ["kind", "at", "admin"]) expect(json, field).not.toHaveProperty(field); + .toEqual([["workspace", "w1", "metadata"], ["setting", null, null]]); + for (let row of rows) { + let json = JSON.parse(row.record); + for (let field of ["kind", "at", "admin", "workspaceId", "workspaceTitle", "exposed"]) { + expect(json, field).not.toHaveProperty(field); + } + } + }); + + it("combines the kind, admin, workspace and since filters", async () => { + let log = auditLog(); + let records: AdminAuditRecord[] = [ + access({ admin: "ada", workspaceId: "w1", at: daysAgo(3) }), + access({ admin: "ada", workspaceId: "w2", at: daysAgo(1) }), + access({ admin: "grace", workspaceId: "w1", at: daysAgo(1) }), + setting({ admin: "ada", at: daysAgo(1) }), + ]; + for (let record of records) await log.append(record, RETENTION); + + let list = async (query: object) => + (await log.list({ limit: 50, ...query }, RETENTION)).records; + expect(await list({ kind: "setting" })).toEqual([records[3]]); + expect(await list({ kind: "workspace", admin: "ada" })).toEqual([records[1], records[0]]); + expect(await list({ workspace: "w1" })).toEqual([records[2], records[0]]); + expect(await list({ workspace: "w1", admin: "ada", since: daysAgo(2) })).toEqual([]); + expect(await list({ admin: "ada", since: daysAgo(2) })).toEqual([records[3], records[1]]); }); - it("combines the kind, admin and since filters", async () => { + it("filters on what a call exposed, alone and with the other filters", async () => { let log = auditLog(); let records: AdminAuditRecord[] = [ - setting({ admin: "ada", at: daysAgo(3) }), - setting({ admin: "grace", at: daysAgo(1) }), + access({ admin: "ada", workspaceId: "w1", exposed: "content", at: daysAgo(3) }), + access({ admin: "ada", workspaceId: "w2", exposed: "content", at: daysAgo(1) }), + access({ admin: "grace", workspaceId: "w1", exposed: "content", at: daysAgo(1) }), + access({ admin: "ada", workspaceId: "w1", at: daysAgo(1) }), setting({ admin: "ada", at: daysAgo(1) }), ]; for (let record of records) await log.append(record, RETENTION); let list = async (query: object) => (await log.list({ limit: 50, ...query }, RETENTION)).records; - await insertAccessRow(log, "ada"); - expect(await list({ kind: "setting", admin: "ada" })).toEqual([records[2], records[0]]); - expect((await list({ admin: "ada", since: daysAgo(2) })).map(record => record.kind)) - .toEqual(["workspace", "setting"]); - expect(await list({ admin: "linus" })).toEqual([]); + expect(await list({ exposed: "content" })).toEqual([records[2], records[1], records[0]]); + expect(await list({ exposed: "content", kind: "workspace", admin: "ada" })) + .toEqual([records[1], records[0]]); + expect(await list({ exposed: "content", workspace: "w1" })).toEqual([records[2], records[0]]); + expect(await list({ exposed: "content", admin: "ada", workspace: "w1", since: daysAgo(2) })) + .toEqual([]); + expect(await list({ exposed: "content", since: daysAgo(2) })) + .toEqual([records[2], records[1]]); + expect(await list({ exposed: "metadata" })).toEqual([records[3]]); + expect(await list({ kind: "setting", exposed: "metadata" })).toEqual([]); + expect(await list({ kind: "setting", exposed: "content" })).toEqual([]); }); it("pages with a cursor, and rejects a malformed cursor or an over-long filter", async () => { let log = auditLog(); - let records = Array.from({ length: 5 }, - (_, i) => setting({ changes: [{ setting: "siteName", after: `Site ${i}` }] })); + let records = Array.from({ length: 5 }, (_, i) => access({ args: { i } })); for (let record of records) await log.append(record, RETENTION); let first = await log.list({ limit: 2 }, RETENTION); @@ -135,6 +168,8 @@ describe("AdminAuditDurableObject.list", () => { } await expectRejection(log.list({ limit: 2, admin: "a".repeat(1001) }, RETENTION), "Audit filters must be at most 1000 characters."); + await expectRejection(log.list({ limit: 2, workspace: "w".repeat(1001) }, RETENTION), + "Audit filters must be at most 1000 characters."); }); it("filters by method through its index, paging within the filter", async () => { @@ -161,22 +196,51 @@ describe("AdminAuditDurableObject.list", () => { WHERE json_extract(record, '$.method') = 'setSiteName' ORDER BY seq DESC`); expect(plan.map(step => step.detail).join("\n")).toContain("admin_audit_method"); }); + + it("filters by workspace owner through its index", async () => { + let log = auditLog(); + let adas = access({ workspaceOwner: "ada" }); + let graces = [access({ workspaceOwner: "grace", workspaceId: "w2" }), + access({ workspaceOwner: "grace", workspaceId: "w3", exposed: "content" })]; + for (let record of [adas, ...graces, setting()]) await log.append(record, RETENTION); + + expect((await log.list({ limit: 50, owner: "grace" }, RETENTION)).records) + .toEqual([graces[1], graces[0]]); + expect((await log.list({ limit: 50, owner: "grace", exposed: "content" }, RETENTION)).records) + .toEqual([graces[1]]); + expect((await log.list({ limit: 50, owner: "linus" }, RETENTION)).records).toEqual([]); + await expectRejection(log.list({ limit: 2, owner: "a".repeat(1001) }, RETENTION), + "Audit filters must be at most 1000 characters."); + + let plan = await sql<{ detail: string }>(log, `EXPLAIN QUERY PLAN SELECT seq FROM admin_audit + WHERE json_extract(record, '$.workspaceOwner') = 'grace' ORDER BY seq DESC`); + expect(plan.map(step => step.detail).join("\n")).toContain("admin_audit_owner"); + }); }); describe("AdminAuditDurableObject retention", () => { it("hides and deletes records older than the retention", async () => { let log = auditLog(); - await log.append(setting({ at: daysAgo(RETENTION + 10) }), RETENTION); - let kept = setting({ at: daysAgo(RETENTION - 10) }); + await log.append(access({ at: daysAgo(RETENTION + 10) }), RETENTION); + let kept = access({ at: daysAgo(RETENTION - 10) }); await log.append(kept, RETENTION); expect((await log.list({ limit: 10 }, RETENTION)).records).toEqual([kept]); expect(await sql(log, "SELECT seq FROM admin_audit")).toHaveLength(1); }); + it("stores a record given no retention without pruning", async () => { + let log = auditLog(); + let old = access({ at: daysAgo(RETENTION + 10) }); + await log.append(old, RETENTION + 30); + await log.append(access(), undefined); + expect(await sql(log, "SELECT seq FROM admin_audit")).toHaveLength(2); + expect((await log.list({ limit: 10 }, RETENTION + 30)).records).toHaveLength(2); + }); + it("keeps records inside the deployment's floor whatever retention it is passed", async () => { let log = auditLog(); - let record = setting({ at: daysAgo(30) }); + let record = access({ at: daysAgo(30) }); await log.append(record, 1); expect((await log.list({ limit: 10 }, 1)).records).toEqual([record]); await expectRejection(log.list({ limit: 10 }, 0), "at least 1"); @@ -223,16 +287,54 @@ describe("AdminAuditDurableObject schema", () => { "CHECK constraint failed"); await expectRejection(insertRow(log, "workspace", "w1", "t", "bogus"), "CHECK constraint failed"); + let { exposed: _exposed, ...unexposed } = access(); + await expectRejection(log.append(unexposed as AdminAccessRecord, RETENTION), + "CHECK constraint failed"); expect(await sql(log, "SELECT seq FROM admin_audit")).toHaveLength(0); }); + it("keeps a long owner short, apart from another sharing its start, and finds it by its whole id", + async () => { + let log = auditLog(); + let ada = `${"o".repeat(600)}a`; + let grace = `${"o".repeat(600)}g`; + let huge = "o".repeat(800_000); + for (let owner of [ada, grace, huge]) await log.append(access({ workspaceOwner: owner }), RETENTION); + + let kept = (await log.list({ limit: 50 }, RETENTION)).records + .map(r => (r as AdminAccessRecord).workspaceOwner); + expect(kept.every(owner => owner.length === MAX_ADMIN_AUDIT_ARG_LENGTH)).toBe(true); + expect(new Set(kept).size).toBe(3); + expect(kept[2]).toMatch(/^o+…#[0-9a-f]{20}$/); + for (let [owner, stored] of [[ada, kept[2]], [grace, kept[1]]]) { + let byWhole = (await log.list({ limit: 50, owner }, RETENTION)).records; + expect(byWhole.map(r => (r as AdminAccessRecord).workspaceOwner)).toEqual([stored]); + expect((await log.list({ limit: 50, owner: stored }, RETENTION)).records).toEqual(byWhole); + } + let [{ size }] = await sql<{ size: number }>(log, + "SELECT max(length(record)) AS size FROM admin_audit"); + expect(size).toBeLessThan(10_000); + }); + + it("stores an over-long title cut, once, whatever the caller sent", async () => { + let log = auditLog(); + let title = "t".repeat(3 * 2 ** 20); + await log.append(access({ workspaceTitle: title }), RETENTION); + + let [record] = (await log.list({ limit: 1 }, RETENTION)).records as AdminAccessRecord[]; + expect(record.workspaceTitle).toHaveLength(MAX_ADMIN_AUDIT_ARG_LENGTH); + expect(record.workspaceTitle.endsWith("…")).toBe(true); + let [{ size }] = await sql<{ size: number }>(log, + "SELECT length(record) + length(workspace_title) AS size FROM admin_audit"); + expect(size).toBeLessThan(10_000); + }); }); describe("AdminAuditDurableObject.stats", () => { it("counts every UTC day of the window, oldest first, empty days included", async () => { let log = auditLog(); let recent = [new Date(), new Date(), daysAgo(2)]; - for (let at of recent) await log.append(setting({ at }), RETENTION); + for (let at of recent) await log.append(access({ at }), RETENTION); await log.append(setting({ at: daysAgo(20) }), RETENTION); let stats = await log.stats({ days: 7 }, RETENTION); @@ -251,13 +353,13 @@ describe("AdminAuditDurableObject.stats", () => { expect((await log.stats({ days: 30 }, RETENTION)).totalEvents).toBe(4); }); - it("orders the top admins by count, then by id", async () => { + it("orders the top admins by count, then by id, counting both kinds of record", async () => { let log = auditLog(); let records: AdminAuditRecord[] = [ - setting({ admin: "grace" }), - setting({ admin: "grace" }), - setting({ admin: "ada" }), - setting({ admin: "linus" }), + access({ admin: "grace" }), + access({ admin: "grace" }), + access({ admin: "ada" }), + access({ admin: "linus" }), setting({ admin: "ada" }), ]; for (let record of records) await log.append(record, RETENTION); @@ -265,30 +367,106 @@ describe("AdminAuditDurableObject.stats", () => { let stats = await log.stats({ days: 7 }, RETENTION); expect(stats.totalEvents).toBe(5); expect(stats.topAdmins).toEqual([ - { admin: "ada", count: 2 }, { admin: "grace", count: 2 }, { admin: "linus", count: 1 }, + { admin: "ada", count: 2, contentReads: 0 }, { admin: "grace", count: 2, contentReads: 0 }, + { admin: "linus", count: 1, contentReads: 0 }, ]); }); it("filters by kind", async () => { let log = auditLog(); + await log.append(access({ admin: "ada" }), RETENTION); await log.append(setting({ admin: "grace" }), RETENTION); await log.append(setting({ admin: "grace" }), RETENTION); - await insertAccessRow(log, "ada"); let settings = await log.stats({ days: 7, kind: "setting" }, RETENTION); expect(settings.totalEvents).toBe(2); - expect(settings.topAdmins).toEqual([{ admin: "grace", count: 2 }]); - expect((await log.stats({ days: 7 }, RETENTION)).totalEvents).toBe(3); + expect(settings.topAdmins).toEqual([{ admin: "grace", count: 2, contentReads: 0 }]); + + let workspaces = await log.stats({ days: 7, kind: "workspace" }, RETENTION); + expect(workspaces.totalEvents).toBe(1); + expect(workspaces.topAdmins).toEqual([{ admin: "ada", count: 1, contentReads: 0 }]); }); it("counts nothing past the retention, however long the window", async () => { let log = auditLog(); - await log.append(setting({ at: daysAgo(RETENTION + 5) }), RETENTION); - await log.append(setting({ at: daysAgo(5) }), RETENTION); - await log.append(setting({ at: daysAgo(5), admin: "grace" }), RETENTION); + await log.append(access({ at: daysAgo(RETENTION + 5), exposed: "content" }), RETENTION); + await log.append(access({ at: daysAgo(5) }), RETENTION); + await log.append(access({ at: daysAgo(5), admin: "grace", exposed: "content" }), RETENTION); let stats = await log.stats({ days: 366 }, RETENTION); expect(stats.totalEvents).toBe(2); expect(stats.byDay).toHaveLength(366); + expect(stats.contentReads).toBe(1); + expect(stats.topAdmins).toEqual([ + { admin: "ada", count: 1, contentReads: 0 }, { admin: "grace", count: 1, contentReads: 1 }, + ]); }); + it("counts the content reads in the window", async () => { + let log = auditLog(); + await log.append(access({ exposed: "content" }), RETENTION); + await log.append(access({ exposed: "content", at: daysAgo(2) }), RETENTION); + await log.append(access({ exposed: "content", at: daysAgo(20) }), RETENTION); + await log.append(access(), RETENTION); + // A day holding only a setting record counts no content read. + await log.append(setting({ at: daysAgo(4) }), RETENTION); + + let stats = await log.stats({ days: 7 }, RETENTION); + expect(stats.totalEvents).toBe(4); + expect(stats.contentReads).toBe(2); + expect((await log.stats({ days: 30 }, RETENTION)).contentReads).toBe(3); + }); + + it("counts each top admin's content reads, ranking by every record", async () => { + let log = auditLog(); + let records: AdminAuditRecord[] = [ + access({ admin: "grace", workspaceId: "w2", exposed: "content" }), + access({ admin: "grace", workspaceId: "w2", exposed: "content" }), + access({ admin: "ada", workspaceId: "w3", exposed: "content" }), + access({ admin: "linus", workspaceId: "w1", exposed: "content" }), + // Many calls exposing nothing rank first, with no content read. + ...Array.from({ length: 5 }, () => access({ admin: "zed", workspaceId: "w9" })), + setting({ admin: "ada" }), + ]; + for (let record of records) await log.append(record, RETENTION); + + let stats = await log.stats({ days: 7 }, RETENTION); + expect(stats.contentReads).toBe(4); + expect(stats.topAdmins).toEqual([ + { admin: "zed", count: 5, contentReads: 0 }, { admin: "ada", count: 2, contentReads: 1 }, + { admin: "grace", count: 2, contentReads: 2 }, { admin: "linus", count: 1, contentReads: 1 }, + ]); + + let many = auditLog(); + for (let i = 0; i < 12; i++) { + let id = String(i).padStart(2, "0"); + await many.append( + access({ admin: `a${id}`, workspaceId: `w${id}`, exposed: "content" }), RETENTION); + } + let capped = await many.stats({ days: 7 }, RETENTION); + expect(capped.contentReads).toBe(12); + expect(capped.topAdmins.map(a => a.admin)) + .toEqual(Array.from({ length: 10 }, (_, i) => `a${String(i).padStart(2, "0")}`)); + }); + + it("filters the content reads by kind", async () => { + let log = auditLog(); + await log.append(access({ admin: "ada", exposed: "content" }), RETENTION); + await log.append(access({ admin: "grace", workspaceId: "w2" }), RETENTION); + await log.append(setting({ admin: "grace" }), RETENTION); + + let settings = await log.stats({ days: 7, kind: "setting" }, RETENTION); + expect(settings.contentReads).toBe(0); + expect(settings.topAdmins).toEqual([{ admin: "grace", count: 1, contentReads: 0 }]); + + let all = await log.stats({ days: 7 }, RETENTION); + let workspaces = await log.stats({ days: 7, kind: "workspace" }, RETENTION); + expect(all.contentReads).toBe(1); + expect(all.topAdmins).toEqual([ + { admin: "grace", count: 2, contentReads: 0 }, { admin: "ada", count: 1, contentReads: 1 }, + ]); + expect(workspaces.contentReads).toBe(all.contentReads); + expect(workspaces.topAdmins).toEqual([ + { admin: "ada", count: 1, contentReads: 1 }, { admin: "grace", count: 1, contentReads: 0 }, + ]); + }); }); diff --git a/packages/workshop-backend/__tests__/admin-config.test.ts b/packages/workshop-backend/__tests__/admin-config.test.ts index 8e3758b316..93adefbe50 100644 --- a/packages/workshop-backend/__tests__/admin-config.test.ts +++ b/packages/workshop-backend/__tests__/admin-config.test.ts @@ -1,5 +1,5 @@ import { describe, expect, it } from "vitest"; -import { DEFAULT_UPDATE_CHECK_INTERVAL_MINUTES, DEFAULT_UPDATE_MINIMUM_AGE_HOURS, DEFAULT_UPDATE_NOTICE_SNOOZE_HOURS, MAX_UPDATE_CHECK_INTERVAL_MINUTES, MAX_UPDATE_HOURS, type GatewayModel, DEFAULT_ADMIN_AUDIT_RETENTION_DAYS, MAX_ADMIN_AUDIT_ARG_LENGTH, MAX_ADMIN_AUDIT_RETENTION_DAYS } from "@gadgets/workshop-shared/api"; +import { DEFAULT_UPDATE_CHECK_INTERVAL_MINUTES, DEFAULT_UPDATE_MINIMUM_AGE_HOURS, DEFAULT_UPDATE_NOTICE_SNOOZE_HOURS, MAX_UPDATE_CHECK_INTERVAL_MINUTES, MAX_UPDATE_HOURS, type GatewayModel, DEFAULT_ADMIN_AUDIT_RETENTION_DAYS, MAX_ADMIN_AUDIT_ARG_LENGTH, MAX_ADMIN_AUDIT_RETENTION_DAYS, DEFAULT_ADMIN_WORKSPACE_ACCESS } from "@gadgets/workshop-shared/api"; import { adminConfigSeed, changesModelSettings, defaultOutputFormatId, effectiveAdminConfig, isUpdateCheckInterval, isUpdateHours, normalizeAdminConfig, parseAdminConfig, parseAdminConfigSeed, readAdminConfig, parseRestrictedWebFetchHosts, reorderFormats, resolveFormatOutput, sanitizeAddedModel, sanitizeModelSettings, sanitizeOutputOverrides, serializeAdminConfig, diffAdminConfig } from "../src/admin-config.js"; import { DEFAULT_ADMIN_CONFIG, type AdminConfig, type FormatCuration } from "../src/storage-schema/admin-settings-storage.js"; @@ -611,16 +611,21 @@ describe("ADMIN_CONFIG_SEED", () => { }); }); -describe("admin config audit retention", () => { - it("defaults the audit retention", () => { +describe("admin config admin access", () => { + it("defaults the access level and the audit retention", () => { let config = parseAdminConfig(null); + expect(config.adminWorkspaceAccess).toBe(DEFAULT_ADMIN_WORKSPACE_ACCESS); expect(config.adminAuditRetentionDays).toBe(DEFAULT_ADMIN_AUDIT_RETENTION_DAYS); + expect(DEFAULT_ADMIN_CONFIG.adminWorkspaceAccess).toBe("metadata"); expect(DEFAULT_ADMIN_CONFIG.adminAuditRetentionDays).toBe(90); }); - // The environment's bound is applied where the choice is enforced, never here, so a stored - // choice outside today's bound is kept for when it changes. - it("keeps a stored choice whatever the environment's bound", () => { + // The environment's bounds are applied where the choice is enforced, never here, so a stored + // choice outside today's bounds is kept for when they change. + it("keeps a stored choice whatever the environment's bounds", () => { + for (let level of ["off", "metadata", "full"] as const) { + expect(normalizeAdminConfig({ adminWorkspaceAccess: level }).adminWorkspaceAccess).toBe(level); + } for (let days of [1, 7, 365, MAX_ADMIN_AUDIT_RETENTION_DAYS]) { expect(normalizeAdminConfig({ adminAuditRetentionDays: days }).adminAuditRetentionDays) .toBe(days); @@ -628,6 +633,10 @@ describe("admin config audit retention", () => { }); it("reads a malformed stored choice as the default", () => { + for (let stored of ["FULL", "none", "", null, 1]) { + expect(normalizeAdminConfig({ adminWorkspaceAccess: stored as "full" }) + .adminWorkspaceAccess, String(stored)).toBe("metadata"); + } for (let stored of [0, -1, 1.5, MAX_ADMIN_AUDIT_RETENTION_DAYS + 1, NaN, "30", null]) { expect(normalizeAdminConfig({ adminAuditRetentionDays: stored as number }) .adminAuditRetentionDays, String(stored)).toBe(90); diff --git a/packages/workshop-backend/__tests__/admin-settings-audit.test.ts b/packages/workshop-backend/__tests__/admin-settings-audit.test.ts index c5f0ee1398..4403ab7f6e 100644 --- a/packages/workshop-backend/__tests__/admin-settings-audit.test.ts +++ b/packages/workshop-backend/__tests__/admin-settings-audit.test.ts @@ -409,7 +409,7 @@ describe("AdminSettings promotes bundled formats", () => { }); }); -describe("AdminSettings audit retention bound", () => { +describe("AdminSettings admin access bounds", () => { it("records a logo with the retention committed, not one still in flight", async () => { const { inDo, seed, put, appended } = adminSettings(); await seed({ adminAuditRetentionDays: 365 }); @@ -459,6 +459,30 @@ describe("AdminSettings audit retention bound", () => { }); }); + it("refuses an access level above ADMIN_WORKSPACE_ACCESS_MAX", async () => { + const { inDo, appended, config, records } = adminSettings(); + await expect(inDo(admin => admin.setAdminWorkspaceAccess("full", ACTOR))) + .rejects.toThrow("ADMIN_WORKSPACE_ACCESS_MAX"); + expect(appended).toEqual([]); + + await inDo(admin => admin.setAdminWorkspaceAccess("off", ACTOR)); + expect(records()[0]!.changes).toStrictEqual( + [{ setting: "adminWorkspaceAccess", before: "metadata", after: "off" }]); + + await inDo(admin => admin.setAdminWorkspaceAccess("full", ACTOR), + { more: { ADMIN_WORKSPACE_ACCESS_MAX: "full" } }); + expect((await config()).adminWorkspaceAccess).toBe("full"); + }); + + it("allows nothing while ADMIN_WORKSPACE_ACCESS_MAX is not a level", async () => { + const { inDo, appended } = adminSettings({ ADMIN_WORKSPACE_ACCESS_MAX: "none" }); + await expect(inDo(admin => admin.setAdminWorkspaceAccess("metadata", ACTOR))) + .rejects.toThrow("ADMIN_WORKSPACE_ACCESS_MAX"); + expect(appended).toEqual([]); + expect((await inDo(admin => admin.getSettings("admin"))).adminWorkspaceAccess) + .toStrictEqual({ effective: "off", chosen: "metadata", max: "off" }); + }); + it("refuses a retention below ADMIN_AUDIT_RETENTION_MIN_DAYS", async () => { const { inDo, appended, config } = adminSettings(); for (let days of [30, 89, 90.5, 3651]) { @@ -476,17 +500,21 @@ describe("AdminSettings audit retention bound", () => { expect((await config()).adminAuditRetentionDays).toBe(400); }); - it("reports the retention enforced, chosen and bounded", async () => { + it("reports the level and retention enforced, chosen and bounded", async () => { const { inDo, appended } = adminSettings(); + const full = { more: { ADMIN_WORKSPACE_ACCESS_MAX: "full" } }; + await inDo(admin => admin.setAdminWorkspaceAccess("full", ACTOR), full); await inDo(admin => admin.setAdminAuditRetentionDays(100, ACTOR)); const settings = (more: object) => inDo(admin => admin.getSettings("admin"), { more }); - expect(await settings({})).toMatchObject({ + expect(await settings({ ADMIN_WORKSPACE_ACCESS_MAX: "full" })).toMatchObject({ + adminWorkspaceAccess: { effective: "full", chosen: "full", max: "full" }, adminAuditRetentionDays: { effective: 100, chosen: 100, min: 90 }, }); - // Raising the bound keeps the admins' choice, but enforces the bound. + // Lowering the bounds keeps the admins' choices, but enforces the bounds. expect(await settings({ ADMIN_AUDIT_RETENTION_MIN_DAYS: "180" })).toMatchObject({ + adminWorkspaceAccess: { effective: "metadata", chosen: "full", max: "metadata" }, adminAuditRetentionDays: { effective: 180, chosen: 100, min: 180 }, }); diff --git a/packages/workshop-backend/__tests__/admin-workspace-view.test.ts b/packages/workshop-backend/__tests__/admin-workspace-view.test.ts new file mode 100644 index 0000000000..8a3b5850f0 --- /dev/null +++ b/packages/workshop-backend/__tests__/admin-workspace-view.test.ts @@ -0,0 +1,1997 @@ +// A deployment admin's read-only view of a workspace (OverseerDurableObject.openForAdmin() and +// AdminWorkspaceView): what it refuses before recording anything, what it records and where, what +// a level below "full" withholds, how live views follow the deployment's setting, and that nothing +// it does reaches the workspace's members or its connections. +// +// Runs against a real OverseerDurableObject (the TEST_OVERSEER binding) whose impl.env is replaced +// per test: the admin path must read the environment only through it. The deployment-wide log is +// the real AdminAuditDurableObject the Overseer reaches through ctx.exports, read back through the +// TEST_ADMIN_AUDIT binding. + +import { describe, expect, it } from "vitest"; +import { env } from "cloudflare:workers"; +import { runInDurableObject } from "cloudflare:test"; +import { RpcTarget } from "capnweb"; +import { + ADMIN_ACCESS_ERROR_CODES, ADMIN_WORKSPACE_METHODS, DEFAULT_ADMIN_COMMIT_LOG_DEPTH, + MAX_ADMIN_ACCESS_REASON_LENGTH, MAX_ADMIN_AUDIT_ARG_LENGTH, MAX_ADMIN_COMMIT_LOG_DEPTH, + MAX_ADMIN_VIEW_LIFETIME_SECONDS, getAdminAccessErrorCode, + type AdminAccessRecord, type AdminAutoApproval, type AdminChatAttachment, type AdminChatState, + type AdminObserver, type AdminPendingChange, type AdminWorkspaceAccess, type AiChatAuthorInfo, + type BlueprintGadgetSummary, type ChatGadgetPin, type FileAtCommit, type ObserverBindingNeed, + type ShareLinkInfo, +} from "@gadgets/workshop-shared/api"; +import { diffFiles, type CodeChange, type CodeContent } from "@gadgets/workshop-shared/code-change"; +import { MAX_BUFFERED_BLOB_SIZE } from "@gadgets/workshop-shared/gatekeeper"; +import type { OverseerDurableObject } from "../src/overseer.js"; +import type { AdminAuditDurableObject } from "../src/admin-audit.js"; +import type { AdminViewTerms } from "../src/admin-access.js"; +import { encodeGitTree, gitObjectOid } from "../src/git-codec.js"; +import { ADMIN_CONFIG_KEY } from "../src/storage-schema/blueprints-kv.js"; +import { putAction } from "./fixtures.js"; +// Load the whole backend (the pool's `main`) up front, so its slow load does not count against the +// first test's timeout. +import "./test-worker.js"; + +declare module "cloudflare:workers" { + interface ProvidedEnv { + TEST_OVERSEER: DurableObjectNamespace; + TEST_ADMIN_AUDIT: DurableObjectNamespace; + } +} + +// The pool's ADMINS. +const ADMIN = "deployment-admin@example.test"; +const OWNER_USER_ID = "owner-user-do"; +// A view that outlives its open by a margin a slow runner keeps (the open reads the config twice +// and awaits its record), then expires within the test. +const SHORT_LIFETIME_MS = 1000; +const OWNER: AiChatAuthorInfo = { type: "user", id: "owner@example.test", name: "Owner" }; +// The seed's collaborator, added by the owner, who holds an observer record. +const BOB: AiChatAuthorInfo = { type: "user", id: "bob@example.test", name: "Bob" }; +// An id the workspace keeps no profile for: neither its owner's nor a collaborator's. +const STRANGER_ID = "stranger@example.test"; +const AGENT: AiChatAuthorInfo = { type: "agent", id: "some-model", name: "Agent" }; +// Who the seed's commits are by. +const COMMITTER = { name: "Owner", email: "owner@example.test" }; +const REASON = "Support ticket 42"; +// The pool sets no ADMIN_AUDIT_RETENTION_MIN_DAYS, so retention is at least 90 days. +const RETENTION = 90; + +const GATEKEEPER_ID = 0; +// The seed's second connection, an ambient one: no person created it, and no gadget binds it. +const AMBIENT_GATEKEEPER_ID = 1; +const CHAT_ID = 1; +// A chat with no message, turn, queued call or action, which the tests that need one add. +const IDLE_CHAT_ID = 2; +// The seeded chat that holds changes to the seeded gadget which are not accepted: notes.txt +// modified, in a "changes" message, and added.txt added and todo.txt removed, in a row no message +// holds yet. +const DRAFT_CHAT_ID = 3; +// A chat with a worktree, which addWorktree() adds. +const WORKTREE_CHAT_ID = 4; +// The parent the seed's orphan commit names. The workspace stores no such object. +const MISSING_PARENT = "ab".repeat(20); + +// The seeded chat's attachments: one stored as a blob, one image kept as bytes on its record, and +// one whose record names only a file of a commit. +const BLOB_ATTACHMENT_ID = "11111111-1111-4111-8111-111111111111"; +const INLINE_ATTACHMENT_ID = "22222222-2222-4222-8222-222222222222"; +const LEGACY_ATTACHMENT_ID = "33333333-3333-4333-8333-333333333333"; +const BLOB_ATTACHMENT_BYTES = new Uint8Array([0x25, 0x50, 0x44, 0x46, 0x00, 0xff, 0x01]); +const INLINE_ATTACHMENT_BYTES = new Uint8Array([0x89, 0x50, 0x4e, 0x47, 0x00, 0xfe]); +const LEGACY_ATTACHMENT_BYTES = new Uint8Array([0x25, 0x50, 0x44, 0x46, 0x00, 0xfd, 0x02, 0x03]); + +// The seed's share links, by who created each: the owner, the collaborator, and STRANGER_ID. +const OWNER_LINK_ID = "a1".repeat(32); +const BOB_LINK_ID = "b2".repeat(32); +const STRANGER_LINK_ID = "c3".repeat(32); +// The blueprint the seed publishes from its gadget. +const BLUEPRINT_ID = "published-blueprint"; + +// The call the seed queues for the chat's agent, and the turn seedTurn() records. +const QUEUED_CALL_ID = 7; +const ARGS_SUMMARY = '[0]: "secret argument"'; +const MODEL_ID = "test-model"; +const INITIATOR_USER_ID = "0123456789abcdef"; + +type Workspace = { + impl: any; + instance: OverseerDurableObject; + state: DurableObjectState; + id: string; + // The deployment's stored settings, as the fake KV serves them. + config: { adminWorkspaceAccess: AdminWorkspaceAccess; adminAuditRetentionDays: number }; + // Replaces the KV read when set, so a test can hold a call at its config read. + kvGet?: (key: string) => Promise; + gadgetId: number; + // The seeded gadget's head, and the head's parent, whose files differ. + commitId: string; + parentCommitId: string; + // A commit whose parent, MISSING_PARENT, is not stored. + orphanCommitId: string; + // The blob behind the seeded chat's blob-backed attachment. + attachmentOid: string; + // The commit that holds its legacy attachment's file. + legacyCommitId: string; + // The message the attachments are on, and when it was written: the chat's newest row. + attachmentSequence: number; + lastMessageAt: Date; +}; + +function serveConfig(ws: Workspace, key: string): string | null { + return key === ADMIN_CONFIG_KEY ? JSON.stringify(ws.config) : null; +} + +// Sets impl.env to the pool's env plus a fake BLUEPRINTS serving `ws.config`, and `vars`. +function setEnv(ws: Workspace, vars: Record = {}): void { + ws.impl.env = { + ...env, + BLUEPRINTS: { get: (key: string) => ws.kvGet ? ws.kvGet(key) : serveConfig(ws, key) }, + ...vars, + }; +} + +async function withWorkspace(fn: (ws: Workspace) => Promise, + opts: { seed?: boolean } = {}): Promise { + let stub = env.TEST_OVERSEER.getByName(`admin-view-${crypto.randomUUID()}`); + await runInDurableObject(stub, async (instance: OverseerDurableObject, state) => { + let impl = (instance as unknown as { impl: any }).impl; + let ws: Workspace = { + impl, instance, state, id: state.id.toString(), + config: { adminWorkspaceAccess: "metadata", adminAuditRetentionDays: RETENTION }, + gadgetId: -1, commitId: "", parentCommitId: "", orphanCommitId: "", attachmentOid: "", + legacyCommitId: "", attachmentSequence: -1, lastMessageAt: new Date(NaN), + }; + setEnv(ws); + if (opts.seed ?? true) await seed(ws); + await fn(ws); + }); +} + +async function seed(ws: Workspace): Promise { + let impl = ws.impl; + impl.directories = { + getByName: () => ({ syncWorkspace: async () => {}, removeWorkspace: async () => {} }), + }; + impl.users = { + idFromString: (id: string) => id, + get: () => ({ + whoami: async () => OWNER, getGadget: async () => null, + // What the workspace tells its owner's account of members' activity, taken and dropped. + messageArrived: async () => {}, setWorkspaceLastActive: async () => {}, + getWorkspace: async () => null, syncWorkspaceOutputs: async () => {}, + }), + }; + impl.ownerId = OWNER_USER_ID; + impl.ownerProfileId = OWNER.id; + impl.storage.ownerId.put(OWNER_USER_ID); + impl.storage.title.put("Quarterly plan"); + + impl.storage.chatMeta.put({ + id: CHAT_ID, title: "Plans", started: new Date(0), lastActive: new Date(0), + queuedSteers: [{ text: "steer text", author: OWNER }], + }); + impl.addChatMessages(CHAT_ID, OWNER, [{ type: "message", message: "secret message" }]); + await seedAttachments(ws); + // Every field a queued call's row stores, the ones no answer may carry included. + impl.storage.pendingAgentCalls.put({ + chatId: CHAT_ID, callId: QUEUED_CALL_ID, methodName: "summarize", args: ["secret argument"], + argsSummary: ARGS_SUMMARY, initiatorUserId: INITIATOR_USER_ID, initiatorModelId: MODEL_ID, + }); + + impl.storage.nextGatekeeperId.put(AMBIENT_GATEKEEPER_ID + 1); + impl.storage.gatekeepers.put({ + id: GATEKEEPER_ID, resourceTitle: "Repo", resourceUrl: "https://example.com/repo", + class: {} as any, + creationSpec: { + type: "gatekeeper", vendorId: "TestVendor", resourceUrl: "https://example.com/repo", + typeUrlPattern: "https://*", + }, + createdBy: OWNER, + }); + impl.storage.gatekeepers.put({ + id: AMBIENT_GATEKEEPER_ID, resourceTitle: "Library", class: {} as any, + creationSpec: { type: "ambient", vendorId: "library", accountId: 3 }, + }); + putAction(impl.storage, 0, { type: "observation", gatekeeperId: GATEKEEPER_ID }); + putAction(impl.storage, 1, { type: "action", gatekeeperId: GATEKEEPER_ID }); + putAction(impl.storage, 2, { type: "bindHook", gatekeeperId: GATEKEEPER_ID }); + for (let id of [0, 1]) { + let record = impl.storage.actions.get(id); + record.description.fields = [{ label: "Body", kind: "inline", value: "secret field" }]; + impl.storage.actions.put(record); + } + let hook = impl.storage.actions.get(2); + hook.hookId = 0; + impl.storage.actions.put(hook); + impl.storage.boundHooks.put({ + id: 0, actionId: 2, gatekeeperId: GATEKEEPER_ID, controller: {} as any, callback: {} as any, + description: { title: "On push", description: "secret hook description" }, enabled: true, + }); + + // The gadget's history: its head, whose parent held another notes.txt and a file since removed. + ws.parentCommitId = await commitFiles(ws, + { "notes.txt": "first notes\n", "old.txt": "dropped\n", "todo.txt": "todo\n" }, + { message: "first notes", timestamp: new Date(1699990000_000) }); + ws.commitId = await commitFiles(ws, { "notes.txt": "private notes\n", "todo.txt": "todo\n" }, + { parents: [ws.parentCommitId], message: "notes" }); + ws.orphanCommitId = await commitFiles(ws, { "checkout.txt": "checked out\n" }, + { parents: [MISSING_PARENT], message: "checked out" }); + let gadget = impl.createGadget("Notes", "NOTES", undefined, undefined, ws.commitId); + ws.gadgetId = gadget.id; + impl.storage.gadgets.put({ + ...gadget, + upstream: { blueprintId: "bp-1" }, + bindings: { REPO: { target: GATEKEEPER_ID } }, + }); + + // lastActive differs by chat: chatMeta.byLastActive is a unique index. + impl.storage.chatMeta.put( + { id: DRAFT_CHAT_ID, title: "Draft", started: new Date(0), lastActive: new Date(1000) }); + await submit(ws, DRAFT_CHAT_ID, { [ws.gadgetId]: [["notes.txt", { set: "draft notes\n" }]] }, + [{ gadgetId: ws.gadgetId, baseCommit: ws.commitId }]); + impl.materializeChatChanges(DRAFT_CHAT_ID); + await submit(ws, DRAFT_CHAT_ID, { + [ws.gadgetId]: [["added.txt", { set: "added\n" }], ["todo.txt", { remove: true }]], + }); + seedSharing(ws); +} + +// Shares the workspace: a collaborator who holds an observer record, three share links, an +// auto-approval rule and a published blueprint. +function seedSharing(ws: Workspace): void { + let storage = ws.impl.storage; + storage.collaborators.put({ + profile: BOB, + addedBy: [{ type: "user", sharer: OWNER.id, created: new Date(0), role: "build" }], + }); + // Every field an observer's record stores, the ones no answer may carry included. + storage.observers.put({ + profileId: BOB.id, observerId: "observer-bob", accountChoices: { [GATEKEEPER_ID]: 7 }, + }); + + storage.shareKeys.put({ + id: OWNER_LINK_ID, note: "For the team", created: new Date(1000), createdBy: OWNER.id, + role: "build", + }); + storage.shareKeys.put({ + id: BOB_LINK_ID, note: "Demo", created: new Date(2000), createdBy: BOB.id, role: "use", + gadgetId: ws.gadgetId, + }); + // A link from before links had roles, by someone the workspace keeps no profile for. + storage.shareKeys.put({ id: STRANGER_LINK_ID, created: new Date(3000), createdBy: STRANGER_ID }); + + // A rule for a kind no seeded action is of, so it applies to none of them. + storage.autoApproveTags.put({ + gatekeeperId: GATEKEEPER_ID, actionKind: { tag: "publish", label: "Publishes" }, + enabledBy: OWNER, + }); + + storage.blueprints.put({ + id: BLUEPRINT_ID, gadgetId: ws.gadgetId, commitId: ws.commitId, + metadata: { + title: "Notes kit", description: "Keeps notes", author: OWNER, created: new Date(0), + version: 2, lastUpdated: new Date(5000), screenshot: true, bindings: {}, + }, + }); +} + +// The seed's observer as a view lists them at any level: an entry's two fields and a choice's +// three, and nothing else of the stored record. +function seededObservers(): AdminObserver[] { + return [{ + profile: BOB, + accountChoices: [{ gatekeeperId: GATEKEEPER_ID, accountId: 7, vendorId: "testvendor" }], + }]; +} + +// The seed's share links as a view lists them at any level, each with its creator resolved. +function seededShareLinks(ws: Workspace): ShareLinkInfo[] { + return [ + { + linkId: OWNER_LINK_ID, note: "For the team", created: new Date(1000), createdBy: OWNER, + role: "build", + }, + { + linkId: BOB_LINK_ID, note: "Demo", created: new Date(2000), createdBy: BOB, role: "use", + gadgetId: ws.gadgetId, + }, + { + linkId: STRANGER_LINK_ID, note: undefined, created: new Date(3000), + createdBy: { type: "user", id: STRANGER_ID, name: STRANGER_ID }, role: "build", + }, + ]; +} + +// The seed's auto-approval rule as a view lists it at any level. +function seededAutoApprovals(): AdminAutoApproval[] { + return [{ + gatekeeperId: GATEKEEPER_ID, actionKind: { tag: "publish", label: "Publishes" }, + enabledBy: OWNER, + }]; +} + +// The seed's blueprint as a view lists it at "full". Below, its id is "" and it has no +// `screenshotUrl`. +function seededBlueprint(): BlueprintGadgetSummary { + return { + id: BLUEPRINT_ID, title: "Notes kit", description: "Keeps notes", version: 2, + codeVersionDate: new Date(1700000000_000), + screenshotUrl: `/blueprint-screenshot/${BLUEPRINT_ID}?v=5000`, dirty: undefined, + }; +} + +// What a collaborator must pass for each of the seed's connections, with the vendor as the +// connection's creationSpec spells it. +const REPO_REQUIREMENT: ObserverBindingNeed = { + gatekeeperId: GATEKEEPER_ID, vendorId: "TestVendor", resourceTitle: "Repo", + resourceUrl: "https://example.com/repo", +}; +const LIBRARY_REQUIREMENT: ObserverBindingNeed = { + gatekeeperId: AMBIENT_GATEKEEPER_ID, vendorId: "library", resourceTitle: "Library", + resourceUrl: undefined, +}; + +// Writes `files` as a commit by COMMITTER. +async function commitFiles(ws: Workspace, files: Record, options: { + parents?: string[], message?: string, timestamp?: Date, +} = {}): Promise { + return await ws.impl.gitCache.writeFilesAsCommit(new Map(Object.entries(files)), { + parents: options.parents ?? [], + author: COMMITTER, + message: options.message ?? "test commit", + timestamp: options.timestamp ?? new Date(1700000000_000), + }); +} + +// A member's submission of `change` at the chat's current stream position, declaring `pins`, +// each from a client of its own. +async function submit(ws: Workspace, chatId: number, change: CodeChange, + pins?: ChatGadgetPin[]): Promise { + let codeBase = ws.impl.storage.chatMeta.get(chatId).codeBase ?? { generation: 0, revision: 0 }; + await ws.impl.submitCodeChange(chatId, { + generation: codeBase.generation, revision: codeBase.revision, + clientId: crypto.randomUUID(), seq: 1, change, ...(pins ? { pins } : {}), + }, OWNER, OWNER_USER_ID); +} + +// Lets what a member's change starts in the background (telling the owner's account) finish. +const settled = () => new Promise(resolve => setTimeout(resolve, 0)); + +// An `edit` of one file, built the way clients build theirs. +function editChange(id: number, path: string, before: string, after: string): CodeChange { + let content = (text: string): CodeContent => new Map([[id, new Map([[path, text]])]]); + return diffFiles(content(before), content(after)); +} + +// One agent step that changes code, as the end of a turn commits it. +async function agentStep(ws: Workspace, chatId: number, change: CodeChange): Promise { + await ws.impl.commitAgentStep(chatId, AGENT, [{ type: "message", message: "step" }], { + changes: [{ change }], createdGadgets: [], createdWorktrees: [], addedBindings: [], + worktreeCommits: [], + }); +} + +// Adds a chat with a worktree at a commit of `files`, none of which the chat has touched. +async function addWorktree(ws: Workspace, files: Record) + : Promise<{ id: number, baseCommit: string }> { + let impl = ws.impl; + impl.storage.chatMeta.put({ + id: WORKTREE_CHAT_ID, title: "Repo work", started: new Date(0), lastActive: new Date(2000), + }); + let baseCommit = await commitFiles(ws, files); + let { id, title } = await impl.createWorktree("Repo", WORKTREE_CHAT_ID, baseCommit); + await impl.commitAgentStep(WORKTREE_CHAT_ID, AGENT, [{ type: "message", message: "step" }], { + changes: [], createdGadgets: [], + createdWorktrees: [{ worktreeId: id, title, bindingName: "TREE" }], + addedBindings: [], worktreeCommits: [], + }); + return { id, baseCommit }; +} + +// Adds a chat with a worktree at a commit of four files, changed every way a listing tells +// apart: a.txt edited by a step's message, which pins the worktree, and then, in rows no message +// holds yet, new.txt added, gone.txt removed, and same.txt edited and edited back to its base +// text. src/untouched.txt is never touched. +async function addWorktreeChat(ws: Workspace): Promise<{ id: number, baseCommit: string }> { + let worktree = await addWorktree(ws, { + "a.txt": "one\n", "gone.txt": "bye\n", "same.txt": "same\n", "src/untouched.txt": "kept\n", + }); + let { id } = worktree; + await agentStep(ws, WORKTREE_CHAT_ID, editChange(id, "a.txt", "one\n", "one!\n")); + await submit(ws, WORKTREE_CHAT_ID, + { [id]: [["new.txt", { set: "fresh\n" }], ["gone.txt", { remove: true }]] }); + await submit(ws, WORKTREE_CHAT_ID, editChange(id, "same.txt", "same\n", "changed\n")); + await submit(ws, WORKTREE_CHAT_ID, editChange(id, "same.txt", "changed\n", "same\n")); + return worktree; +} + +// The seeded draft chat's pending change, as a view lists it at any level. +function seededPendingChange(ws: Workspace): AdminPendingChange { + return { + gadgetId: ws.gadgetId, baseCommit: ws.commitId, mergedCommit: ws.commitId, + added: ["added.txt"], modified: ["notes.txt"], removed: ["todo.txt"], + }; +} + +// A `stream` answer's bytes, read to the end. +async function streamBytes(file: FileAtCommit): Promise { + if (file.kind !== "stream") throw new Error(`Expected a stream, got ${file.kind}.`); + return new Uint8Array(await new Response(file.content).arrayBuffer()); +} + +// Adds a message with three attachments to the seeded chat. The stored references carry what a +// listing must drop: the blob-backed one its path in the scratch worktree, the image its bytes. +// The image has no name, and neither it nor the legacy one has an oid. +async function seedAttachments(ws: Workspace): Promise { + let impl = ws.impl; + ws.attachmentOid = impl.gitCache.writeBlob(BLOB_ATTACHMENT_BYTES).oid; + ws.legacyCommitId = await impl.gitCache.writeFilesAsCommit( + new Map([["legacy.pdf", impl.gitCache.writeBlob(LEGACY_ATTACHMENT_BYTES)]]), { + parents: [], + author: { name: "Owner", email: "owner@example.test" }, + message: "attachments", + timestamp: new Date(1700000000_000), + }); + impl.storage.chatAttachmentContent.put({ + fileId: BLOB_ATTACHMENT_ID, + state: { + type: "committed", chatId: CHAT_ID, oid: ws.attachmentOid, workspacePath: "report.pdf", + }, + }); + impl.storage.chatAttachmentContent.put({ + fileId: INLINE_ATTACHMENT_ID, data: INLINE_ATTACHMENT_BYTES, + state: { type: "committed", chatId: CHAT_ID }, + }); + impl.storage.chatAttachmentContent.put({ + fileId: LEGACY_ATTACHMENT_ID, + state: { + type: "committed", chatId: CHAT_ID, workspaceCommitId: ws.legacyCommitId, + workspacePath: "legacy.pdf", + }, + }); + impl.addChatMessages(CHAT_ID, OWNER, [{ + type: "message", message: "see attached", + attachments: [ + { + id: BLOB_ATTACHMENT_ID, oid: ws.attachmentOid, mimeType: "application/pdf", + name: "report.pdf", size: BLOB_ATTACHMENT_BYTES.byteLength, workspacePath: "report.pdf", + }, + { + id: INLINE_ATTACHMENT_ID, mimeType: "image/png", + size: INLINE_ATTACHMENT_BYTES.byteLength, content: INLINE_ATTACHMENT_BYTES, + }, + { + id: LEGACY_ATTACHMENT_ID, mimeType: "application/pdf", name: "legacy.pdf", + size: LEGACY_ATTACHMENT_BYTES.byteLength, + }, + ], + }]); + let newest = [...impl.storage.chats.list()].at(-1); + ws.attachmentSequence = newest.sequence; + ws.lastMessageAt = newest.timestamp; +} + +// Records an agent turn for the seeded chat, with every field a turn's record stores. The chat +// names no agent as responding, so the turn is a paused one. +function seedTurn(ws: Workspace): void { + ws.impl.storage.activeAgents.put({ + chatId: CHAT_ID, initiatorUserId: INITIATOR_USER_ID, modelId: MODEL_ID, initiator: OWNER, + callbackInitiated: false, + }); +} + +function addIdleChat(ws: Workspace): void { + ws.impl.storage.chatMeta.put( + { id: IDLE_CHAT_ID, title: "Idle", started: new Date(0), lastActive: new Date(0) }); +} + +// The seeded chat's state once seedTurn() has run, as a view reports it: `argsSummary` is the +// queued call's summary at "full" and "" below. +function seededChatState(ws: Workspace, argsSummary: string): AdminChatState { + return { + turn: { + modelId: MODEL_ID, initiator: OWNER, callbackInitiated: false, resumeAttempts: 0, + paused: true, + }, + lastMessageAt: ws.lastMessageAt, + queuedCalls: [{ + callId: QUEUED_CALL_ID, methodName: "summarize", initiatorModelId: MODEL_ID, argsSummary, + }], + pendingActionIds: [0, 1, 2], + }; +} + +// The seeded chat's attachments as a view lists them at any level. +function seededAttachments(ws: Workspace): AdminChatAttachment[] { + let sequence = ws.attachmentSequence; + return [ + { + id: BLOB_ATTACHMENT_ID, oid: ws.attachmentOid, mimeType: "application/pdf", + name: "report.pdf", size: BLOB_ATTACHMENT_BYTES.byteLength, sequence, + }, + { + id: INLINE_ATTACHMENT_ID, oid: undefined, mimeType: "image/png", name: undefined, + size: INLINE_ATTACHMENT_BYTES.byteLength, sequence, + }, + { + id: LEGACY_ATTACHMENT_ID, oid: undefined, mimeType: "application/pdf", name: "legacy.pdf", + size: LEGACY_ATTACHMENT_BYTES.byteLength, sequence, + }, + ]; +} + +// A view asks for the level it is minted at unless `overrides` says otherwise. +function terms(overrides: Partial = {}): AdminViewTerms { + return { + via: "api", access: "metadata", expiresAt: new Date(Date.now() + 600_000), ...overrides, + requested: overrides.requested ?? overrides.access ?? "metadata", + }; +} + +function open(ws: Workspace, overrides: Partial = {}, + admin = ADMIN, reason = REASON) { + return ws.instance.openForAdmin(admin, reason, "session-1", terms(overrides)); +} + +// The code a refusal carries, failing the test if `promise` resolves. +async function codeOf(promise: Promise): Promise { + try { + await promise; + } catch (err) { + return getAdminAccessErrorCode(err) ?? `uncoded: ${String(err)}`; + } + throw new Error("Expected a refusal."); +} + +// The deployment-wide log's records for this workspace, oldest first. +async function deploymentRecords(ws: Workspace): Promise { + let page = await env.TEST_ADMIN_AUDIT.getByName("") + .list({ workspace: ws.id, limit: 200 }, RETENTION); + return (page.records as AdminAccessRecord[]).toReversed(); +} + +function storageKeys(ws: Workspace): string[] { + return [...ws.state.storage.kv.list()].map(([key]) => key); +} + +// What a member could notice of the workspace: everything it stores (keys and values, git +// objects included), who is present, the sessions open and the alarm. +async function memberVisibleState(ws: Workspace) { + return { + storage: [...ws.state.storage.kv.list()], + present: ws.impl.presenceSnapshot(), + sessions: ws.impl.liveSessionCounts(), + alarm: await ws.state.storage.getAlarm(), + }; +} + +// What a holder of `view` can reach over RPC: every string-named member of its class, and of +// any class between it and RpcTarget. Cap'n Web exposes prototype members, never own ones. +function rpcSurface(view: object): Set { + let names = new Set(); + for (let proto = Object.getPrototypeOf(view); proto !== RpcTarget.prototype; + proto = Object.getPrototypeOf(proto)) { + for (let name of Object.getOwnPropertyNames(proto)) names.add(name); + } + names.delete("constructor"); + return names; +} + +describe("opening an admin view", () => { + it("refuses an uninitialised workspace without initialising it", () => withWorkspace(async ws => { + expect(await codeOf(open(ws))).toBe(ADMIN_ACCESS_ERROR_CODES.workspaceNotFound); + expect(storageKeys(ws)).toStrictEqual([]); + expect(await deploymentRecords(ws)).toStrictEqual([]); + }, { seed: false })); + + it("refuses, unrecorded, a caller who is not an admin and malformed terms", + () => withWorkspace(async ws => { + expect(await codeOf(open(ws, {}, "owner@example.test"))) + .toBe(ADMIN_ACCESS_ERROR_CODES.notAdmin); + await expect(open(ws, {}, ADMIN, " ")).rejects.toThrow(TypeError); + await expect(open(ws, {}, ADMIN, "x".repeat(MAX_ADMIN_ACCESS_REASON_LENGTH + 1))) + .rejects.toThrow(TypeError); + await expect(open(ws, { via: "gatekeeper" as any })).rejects.toThrow(TypeError); + await expect(open(ws, { access: "off" as any })).rejects.toThrow(TypeError); + await expect(open(ws, { requested: "off" as any })).rejects.toThrow(TypeError); + await expect(open(ws, { requested: "everything" as any })).rejects.toThrow(TypeError); + await expect(open(ws, { requested: "full", access: "off" as any })).rejects.toThrow(TypeError); + await expect(open(ws, { requested: "metadata", access: "full" })).rejects.toThrow(TypeError); + // Past the helper, which fills in a missing level. + await expect(ws.instance.openForAdmin(ADMIN, REASON, "session-1", + { ...terms(), requested: undefined } as unknown as AdminViewTerms)) + .rejects.toThrow(TypeError); + await expect(open(ws, { expiresAt: new Date(NaN) })).rejects.toThrow(TypeError); + await expect(open(ws, { expiresAt: "tomorrow" as any })).rejects.toThrow(TypeError); + await expect(open(ws, { + expiresAt: new Date(Date.now() + (MAX_ADMIN_VIEW_LIFETIME_SECONDS + 120) * 1000), + })).rejects.toThrow(TypeError); + // Already past is not malformed: it is an expired view. + expect(await codeOf(open(ws, { expiresAt: new Date(Date.now() - 1000) }))) + .toBe(ADMIN_ACCESS_ERROR_CODES.viewExpired); + + expect(await deploymentRecords(ws)).toStrictEqual([]); + })); + + it("records the open before returning the view", () => withWorkspace(async ws => { + let expiresAt = new Date(Date.now() + 600_000); + await open(ws, { via: "cli", expiresAt }); + let [record, ...rest] = await deploymentRecords(ws); + expect(rest).toStrictEqual([]); + expect(record).toStrictEqual({ + kind: "workspace", workspaceId: ws.id, workspaceOwner: OWNER.id, at: expect.any(Date), + admin: ADMIN, sessionId: "session-1", via: "cli", workspaceTitle: "Quarterly plan", + method: "inspectWorkspace", args: { expiresAt: expiresAt.toISOString() }, reason: REASON, + requested: "metadata", access: "metadata", exposed: "metadata", + }); + })); + + it("records an open while the setting is off as refused", + () => withWorkspace(async ws => { + ws.config.adminWorkspaceAccess = "off"; + expect(await codeOf(open(ws))).toBe(ADMIN_ACCESS_ERROR_CODES.accessOff); + let records = await deploymentRecords(ws); + expect(records.map(r => [r.method, r.access, r.refused])) + .toStrictEqual([["inspectWorkspace", "off", ADMIN_ACCESS_ERROR_CODES.accessOff]]); + })); + + it("fails when the record cannot be written", () => withWorkspace(async ws => { + let live = await open(ws); + ws.impl.adminAudits = { + getByName: () => ({ append: async () => { throw new Error("audit store down"); } }), + }; + await expect(open(ws)).rejects.toThrow("audit store down"); + await expect(live.listChats()).rejects.toThrow("audit store down"); + expect((await deploymentRecords(ws)).map(r => r.method)).toStrictEqual(["inspectWorkspace"]); + })); +}); + +describe("a metadata view", () => { + it("records every call, content refusals included", + () => withWorkspace(async ws => { + let view = await open(ws); + await view.describe(); + await view.listChats(); + expect(await codeOf(view.getChat(CHAT_ID))).toBe(ADMIN_ACCESS_ERROR_CODES.contentWithheld); + await view.getChatState(CHAT_ID); + await view.listChatAttachments(CHAT_ID); + expect(await codeOf(view.readChatAttachment(CHAT_ID, BLOB_ATTACHMENT_ID))) + .toBe(ADMIN_ACCESS_ERROR_CODES.contentWithheld); + await view.listActions({ filter: "all" }); + await view.listFiles(ws.gadgetId); + expect(await codeOf(view.readFile(ws.gadgetId, "notes.txt"))) + .toBe(ADMIN_ACCESS_ERROR_CODES.contentWithheld); + await view.getCommitLog(ws.commitId); + await view.getCommitLog(ws.commitId, 1); + await view.listFilesAtCommit(ws.parentCommitId); + expect(await codeOf(view.readFileAtCommit(ws.parentCommitId, "notes.txt"))) + .toBe(ADMIN_ACCESS_ERROR_CODES.contentWithheld); + await view.listChangedPaths(ws.parentCommitId, ws.commitId); + await view.listPendingChanges(DRAFT_CHAT_ID); + expect(await codeOf(view.readPendingFile(DRAFT_CHAT_ID, ws.gadgetId, "notes.txt"))) + .toBe(ADMIN_ACCESS_ERROR_CODES.contentWithheld); + await view.listGatekeepers(); + await view.listObservers(); + await view.listObserverRequirements("workspace"); + await view.listObserverRequirements({ gadgetId: ws.gadgetId }); + await view.listShareLinks(); + await view.listAutoApprovals(); + await view.listBlueprints(); + await view.getDiagnostics(); + + let records = await deploymentRecords(ws); + expect(records.map(r => [r.method, r.args, r.access, r.refused])).toStrictEqual([ + ["inspectWorkspace", { expiresAt: expect.any(String) }, "metadata", undefined], + ["describe", {}, "metadata", undefined], + ["listChats", {}, "metadata", undefined], + ["getChat", { chatId: CHAT_ID }, "metadata", ADMIN_ACCESS_ERROR_CODES.contentWithheld], + ["getChatState", { chatId: CHAT_ID }, "metadata", undefined], + ["listChatAttachments", { chatId: CHAT_ID }, "metadata", undefined], + ["readChatAttachment", { chatId: CHAT_ID, id: BLOB_ATTACHMENT_ID }, "metadata", + ADMIN_ACCESS_ERROR_CODES.contentWithheld], + ["listActions", { filter: "all" }, "metadata", undefined], + ["listFiles", { gadgetId: ws.gadgetId }, "metadata", undefined], + ["readFile", { gadgetId: ws.gadgetId, path: "notes.txt" }, "metadata", + ADMIN_ACCESS_ERROR_CODES.contentWithheld], + ["getCommitLog", { fromCommit: ws.commitId }, "metadata", undefined], + ["getCommitLog", { fromCommit: ws.commitId, depth: 1 }, "metadata", undefined], + ["listFilesAtCommit", { commitId: ws.parentCommitId }, "metadata", undefined], + ["readFileAtCommit", { commitId: ws.parentCommitId, path: "notes.txt" }, "metadata", + ADMIN_ACCESS_ERROR_CODES.contentWithheld], + ["listChangedPaths", { fromCommit: ws.parentCommitId, toCommit: ws.commitId }, "metadata", + undefined], + ["listPendingChanges", { chatId: DRAFT_CHAT_ID }, "metadata", undefined], + ["readPendingFile", { chatId: DRAFT_CHAT_ID, gadgetId: ws.gadgetId, path: "notes.txt" }, + "metadata", ADMIN_ACCESS_ERROR_CODES.contentWithheld], + ["listGatekeepers", {}, "metadata", undefined], + ["listObservers", {}, "metadata", undefined], + ["listObserverRequirements", { grant: "workspace" }, "metadata", undefined], + ["listObserverRequirements", { gadgetId: ws.gadgetId }, "metadata", undefined], + ["listShareLinks", {}, "metadata", undefined], + ["listAutoApprovals", {}, "metadata", undefined], + ["listBlueprints", {}, "metadata", undefined], + ["getDiagnostics", {}, "metadata", undefined], + ]); + // The list above is kept by hand: a method the audit names fails here until it is called + // above, and its record asserted. + expect(new Set(records.map(r => r.method))).toStrictEqual(new Set(ADMIN_WORKSPACE_METHODS)); + expect(new Set(records.map(r => r.reason))).toStrictEqual(new Set([REASON])); + expect(new Set(records.map(r => r.requested))).toStrictEqual(new Set(["metadata"])); + expect(new Set(records.map(r => r.exposed))).toStrictEqual(new Set(["metadata"])); + })); + + it("exposes no method but the ones the audit names", () => withWorkspace(async ws => { + let view = await open(ws); + expect(view).toBeInstanceOf(RpcTarget); + expect(rpcSurface(view)).toStrictEqual( + new Set(ADMIN_WORKSPACE_METHODS.filter(method => method !== "inspectWorkspace"))); + })); + + it("withholds content but keeps structure", () => withWorkspace(async ws => { + seedTurn(ws); + let view = await open(ws); + + let chats = await view.listChats(); + expect(chats.map(chat => chat.title)).toStrictEqual(["Draft", "Plans"]); + expect(chats.find(chat => chat.id === CHAT_ID)).not.toHaveProperty("queuedSteers"); + + // Whole answers: a stored record's other fields (a queued call's arguments, an attachment's + // path and bytes) would pass a looser check. + expect(await view.getChatState(CHAT_ID)).toStrictEqual(seededChatState(ws, "")); + expect(await view.listChatAttachments(CHAT_ID)).toStrictEqual(seededAttachments(ws)); + + let { entries } = await view.listActions(); + expect(entries.map(entry => [entry.type, entry.description.title])).toStrictEqual([ + ["bindHook", "Action 2"], ["action", "Action 1"], ["observation", "Action 0"], + ]); + for (let entry of entries) { + expect(entry.description.description).toBe(""); + expect(entry.description).not.toHaveProperty("fields"); + } + + let description = await view.describe(); + expect(description).toMatchObject({ + id: ws.id, title: "Quarterly plan", owner: OWNER, access: "metadata", + collaborators: [{ + profile: BOB, access: { role: "build" }, + addedBy: [{ type: "user", sharer: OWNER.id, created: new Date(0), role: "build" }], + }], + }); + let gadget = description.workpieces.find(w => w.id === ws.gadgetId)!; + expect(gadget).toMatchObject({ type: "gadget", title: "Notes", commitId: ws.commitId }); + expect(gadget).not.toHaveProperty("upstream"); + + let [gatekeeper, ambient, ...others] = await view.listGatekeepers(); + expect(gatekeeper).toMatchObject({ + id: GATEKEEPER_ID, resourceTitle: "Repo", vendorId: "testvendor", createdBy: OWNER, + bindings: [{ gadgetId: ws.gadgetId, name: "REPO" }], + }); + expect(gatekeeper.hooks.map(hook => [hook.description.title, hook.description.description])) + .toStrictEqual([["On push", ""]]); + // The ambient connection, whole: no person created it. + expect(ambient).toStrictEqual({ + id: AMBIENT_GATEKEEPER_ID, resourceTitle: "Library", resourceUrl: undefined, + vendorId: "library", creationSpec: { type: "ambient", vendorId: "library", accountId: 3 }, + createdBy: undefined, bindings: [], hooks: [], + }); + expect(others).toStrictEqual([]); + + // Whole answers again: an observer's stored record also holds the id connections know them + // by. These three hold nothing the level withholds. + expect(await view.listObservers()).toStrictEqual(seededObservers()); + expect(await view.listShareLinks()).toStrictEqual(seededShareLinks(ws)); + expect(await view.listAutoApprovals()).toStrictEqual(seededAutoApprovals()); + // A blueprint keeps its title, description, version and date, without the id that is a link + // to its code, or the screenshot URL that holds it. + let { screenshotUrl: _screenshotUrl, ...blueprint } = seededBlueprint(); + expect(await view.listBlueprints()).toStrictEqual([{ ...blueprint, id: "" }]); + + expect(await view.listFiles(ws.gadgetId)).toStrictEqual( + [{ name: "notes.txt", kind: "file" }, { name: "todo.txt", kind: "file" }]); + + // A commit keeps its id, parents, author and time, without its message. + expect(await view.getCommitLog(ws.commitId)).toStrictEqual([ + { + oid: ws.commitId, parents: [ws.parentCommitId], message: "", author: COMMITTER, + timestamp: new Date(1700000000_000), + }, + { + oid: ws.parentCommitId, parents: [], message: "", author: COMMITTER, + timestamp: new Date(1699990000_000), + }, + ]); + // Paths are names, so these answer in full. + expect(await view.listFilesAtCommit(ws.parentCommitId)).toStrictEqual([ + { name: "notes.txt", kind: "file" }, { name: "old.txt", kind: "file" }, + { name: "todo.txt", kind: "file" }, + ]); + expect(await view.listChangedPaths(ws.parentCommitId, ws.commitId)) + .toStrictEqual(["notes.txt", "old.txt"]); + expect(await view.listPendingChanges(DRAFT_CHAT_ID)).toStrictEqual([seededPendingChange(ws)]); + })); +}); + +describe("an inspection", () => { + it("changes nothing a member sees and joins no session", () => withWorkspace(async ws => { + let impl = ws.impl; + let actionsBefore = [...impl.storage.actions.list()]; + let notified: string[] = []; + impl.storage.actions.subscribe({ + add: () => notified.push("add"), + update: () => notified.push("update"), + remove: () => notified.push("remove"), + }); + // A chat with a worktree, whose content the view is the one to fold: the fold that reads its + // bases from stored objects alone. + let worktree = await addWorktreeChat(ws); + await settled(); + impl.invalidateChatContent(WORKTREE_CHAT_ID); + impl.invalidateChatContent(DRAFT_CHAT_ID); + let before = await memberVisibleState(ws); + + // Every view method, at "metadata", where the content reads are refused, and then at "full", + // where each runs. + let view = await open(ws); + await view.describe(); + await view.listChats(); + await view.getChat(CHAT_ID).catch(() => {}); + await view.getChatState(CHAT_ID); + await view.listChatAttachments(CHAT_ID); + await view.readChatAttachment(CHAT_ID, BLOB_ATTACHMENT_ID).catch(() => {}); + await view.listActions(); + await view.listFiles(ws.gadgetId); + await view.readFile(ws.gadgetId, "notes.txt").catch(() => {}); + await view.getCommitLog(ws.commitId); + await view.listFilesAtCommit(ws.parentCommitId); + await view.readFileAtCommit(ws.parentCommitId, "notes.txt").catch(() => {}); + await view.listChangedPaths(ws.parentCommitId, ws.commitId); + await view.listPendingChanges(DRAFT_CHAT_ID); + await view.readPendingFile(DRAFT_CHAT_ID, ws.gadgetId, "notes.txt").catch(() => {}); + await view.listGatekeepers(); + await view.listObservers(); + await view.listObserverRequirements("workspace"); + await view.listObserverRequirements({ gadgetId: ws.gadgetId }); + await view.listShareLinks(); + await view.listAutoApprovals(); + await view.listBlueprints(); + await view.getDiagnostics(); + + setEnv(ws, { ADMIN_WORKSPACE_ACCESS_MAX: "full" }); + ws.config.adminWorkspaceAccess = "full"; + let full = await open(ws, { access: "full" }); + await full.describe(); + await full.listChats(); + await full.getChat(CHAT_ID); + await full.getChatState(CHAT_ID); + await full.listChatAttachments(CHAT_ID); + for (let id of [BLOB_ATTACHMENT_ID, INLINE_ATTACHMENT_ID, LEGACY_ATTACHMENT_ID]) { + await full.readChatAttachment(CHAT_ID, id); + } + await full.listActions(); + await full.listFiles(ws.gadgetId); + await full.readFile(ws.gadgetId, "notes.txt"); + await full.getCommitLog(ws.commitId); + await full.listFilesAtCommit(ws.parentCommitId); + await full.readFileAtCommit(ws.parentCommitId, "notes.txt"); + await full.listChangedPaths(ws.parentCommitId, ws.commitId); + await full.listPendingChanges(DRAFT_CHAT_ID); + await full.readPendingFile(DRAFT_CHAT_ID, ws.gadgetId, "notes.txt"); + expect(await full.listPendingChanges(WORKTREE_CHAT_ID)).toHaveLength(1); + expect(await full.readPendingFile(WORKTREE_CHAT_ID, worktree.id, "a.txt")) + .toStrictEqual({ kind: "text", text: "one!\n" }); + await full.listGatekeepers(); + await full.listObservers(); + await full.listObserverRequirements("workspace"); + await full.listObserverRequirements({ gadgetId: ws.gadgetId }); + // Asks the owner's account, since the owner created a link, and still writes nothing. + expect(await full.listShareLinks()).toHaveLength(3); + await full.listAutoApprovals(); + await full.listBlueprints(); + await full.getDiagnostics(); + + expect(await memberVisibleState(ws)).toStrictEqual(before); + expect(notified).toStrictEqual([]); + expect([...impl.storage.actions.list()]).toStrictEqual(actionsBefore); + expect(impl.liveSessionCounts()).toStrictEqual({ owner: 0, build: 0, use: 0 }); + expect((await view.getDiagnostics()).sessions).toStrictEqual({ owner: 0, build: 0, use: 0 }); + })); +}); + +describe("a full view", () => { + it("reads content when both the cap and the setting allow it, through impl.env alone", + () => withWorkspace(async ws => { + // The pool's own env caps at "metadata": only impl.env says "full". + setEnv(ws, { ADMIN_WORKSPACE_ACCESS_MAX: "full" }); + ws.config.adminWorkspaceAccess = "full"; + let view = await open(ws, { access: "full" }); + + let history = await view.getChat(CHAT_ID); + expect(history.messages.map(message => message.type === "message" && message.message)) + .toStrictEqual(["secret message", "see attached"]); + expect(await view.readFile(ws.gadgetId, "notes.txt")) + .toStrictEqual({ kind: "text", text: "private notes\n" }); + expect((await view.listChats()).find(chat => chat.id === CHAT_ID)!.queuedSteers) + .toStrictEqual([{ text: "steer text", author: OWNER }]); + let { workpieces } = await view.describe(); + expect(workpieces.find(w => w.id === ws.gadgetId)!.upstream) + .toStrictEqual({ blueprintId: "bp-1" }); + let [gatekeeper] = await view.listGatekeepers(); + expect(gatekeeper.hooks[0].description.description).toBe("secret hook description"); + let { entries } = await view.listActions(); + expect(entries.find(entry => entry.type === "action")!.description) + .toMatchObject({ description: "Action 1 description", fields: [expect.anything()] }); + expect((await deploymentRecords(ws)).every(r => r.access === "full")).toBe(true); + })); + + it("reads a chat's turn, queued calls and attachments", () => withWorkspace(async ws => { + setEnv(ws, { ADMIN_WORKSPACE_ACCESS_MAX: "full" }); + ws.config.adminWorkspaceAccess = "full"; + let view = await open(ws, { access: "full" }); + + // No turn is recorded yet. + let idle = await view.getChatState(CHAT_ID); + expect(idle.turn).toBeUndefined(); + expect((await view.getDiagnostics()).activeAgentChatIds).toStrictEqual([]); + + // Whole answers, as at "metadata": only the queued call's summary differs. An action of the + // chat that was decided is not pending, so the answer leaves it out. + seedTurn(ws); + putAction(ws.impl.storage, 3, { state: "approved", gatekeeperId: GATEKEEPER_ID }); + expect(await view.getChatState(CHAT_ID)).toStrictEqual(seededChatState(ws, ARGS_SUMMARY)); + expect(await view.listChatAttachments(CHAT_ID)).toStrictEqual(seededAttachments(ws)); + expect((await view.getDiagnostics()).activeAgentChatIds).toStrictEqual([CHAT_ID]); + + // The turn runs once the chat names its agent, and its attempts are the record's. + let impl = ws.impl; + impl.storage.chatMeta.put({ + ...impl.storage.chatMeta.get(CHAT_ID), + activeAgent: { type: "agent", id: MODEL_ID, name: "Test model" }, + }); + impl.storage.activeAgents.put({ ...impl.storage.activeAgents.get(CHAT_ID), resumeAttempts: 2 }); + expect((await view.getChatState(CHAT_ID)).turn).toStrictEqual({ + modelId: MODEL_ID, initiator: OWNER, callbackInitiated: false, resumeAttempts: 2, + paused: false, + }); + expect((await view.getDiagnostics()).activeAgentChatIds).toStrictEqual([CHAT_ID]); + + expect(await view.readChatAttachment(CHAT_ID, BLOB_ATTACHMENT_ID)) + .toStrictEqual(BLOB_ATTACHMENT_BYTES); + expect(await view.readChatAttachment(CHAT_ID, INLINE_ATTACHMENT_ID)) + .toStrictEqual(INLINE_ATTACHMENT_BYTES); + expect(await view.readChatAttachment(CHAT_ID, LEGACY_ATTACHMENT_ID)) + .toStrictEqual(LEGACY_ATTACHMENT_BYTES); + })); + + it("records what each call's answer exposed", () => withWorkspace(async ws => { + addIdleChat(ws); + setEnv(ws, { ADMIN_WORKSPACE_ACCESS_MAX: "full" }); + ws.config.adminWorkspaceAccess = "full"; + let view = await open(ws, { access: "full" }); + + await view.getChat(CHAT_ID); + await view.readFile(ws.gadgetId, "notes.txt"); + await view.listChats(); + await view.describe(); + await view.listGatekeepers(); + await view.listActions(); + await view.listFiles(ws.gadgetId); + await view.getDiagnostics(); + expect(await view.readFile(ws.gadgetId, "missing.txt")).toStrictEqual({ kind: "absent" }); + expect((await view.getChat(999)).messages).toStrictEqual([]); + expect((await view.listActions({ beforeId: 0 })).entries).toStrictEqual([]); + await view.getChatState(CHAT_ID); + await view.getChatState(IDLE_CHAT_ID); + await view.listChatAttachments(CHAT_ID); + await view.readChatAttachment(CHAT_ID, BLOB_ATTACHMENT_ID); + await view.getCommitLog(ws.commitId); + await view.readFileAtCommit(ws.parentCommitId, "old.txt"); + await view.readPendingFile(DRAFT_CHAT_ID, ws.gadgetId, "added.txt"); + await view.listFilesAtCommit(ws.commitId); + await view.listChangedPaths(ws.parentCommitId, ws.commitId); + await view.listPendingChanges(DRAFT_CHAT_ID); + expect(await view.readFileAtCommit(ws.commitId, "old.txt")).toStrictEqual({ kind: "absent" }); + expect(await view.readPendingFile(DRAFT_CHAT_ID, ws.gadgetId, "todo.txt")) + .toStrictEqual({ kind: "absent" }); + await view.listObservers(); + await view.listObserverRequirements("workspace"); + await view.listShareLinks(); + await view.listAutoApprovals(); + await view.listBlueprints(); + // With no blueprint published, the listing holds no id. + ws.impl.storage.blueprints.delete(BLUEPRINT_ID); + expect(await view.listBlueprints()).toStrictEqual([]); + + let records = await deploymentRecords(ws); + expect(records.map(r => [r.method, r.exposed])).toStrictEqual([ + ["inspectWorkspace", "metadata"], + ["getChat", "content"], + ["readFile", "content"], + ["listChats", "content"], + ["describe", "content"], + ["listGatekeepers", "content"], + ["listActions", "content"], + ["listFiles", "metadata"], + ["getDiagnostics", "metadata"], + ["readFile", "metadata"], + ["getChat", "metadata"], + ["listActions", "metadata"], + ["getChatState", "content"], + ["getChatState", "metadata"], + ["listChatAttachments", "metadata"], + ["readChatAttachment", "content"], + ["getCommitLog", "content"], + ["readFileAtCommit", "content"], + ["readPendingFile", "content"], + ["listFilesAtCommit", "metadata"], + ["listChangedPaths", "metadata"], + ["listPendingChanges", "metadata"], + ["readFileAtCommit", "metadata"], + ["readPendingFile", "metadata"], + ["listObservers", "metadata"], + ["listObserverRequirements", "metadata"], + ["listShareLinks", "metadata"], + ["listAutoApprovals", "metadata"], + ["listBlueprints", "content"], + ["listBlueprints", "metadata"], + ]); + expect(records.every(r => r.requested === "full" && r.access === "full")).toBe(true); + })); + + it("returns nothing when the deployment-wide record cannot be written", + () => withWorkspace(async ws => { + setEnv(ws, { ADMIN_WORKSPACE_ACCESS_MAX: "full" }); + ws.config.adminWorkspaceAccess = "full"; + let view = await open(ws, { access: "full" }); + ws.impl.adminAudits = { + getByName: () => ({ append: async () => { throw new Error("audit store down"); } }), + }; + await expect(view.getChat(CHAT_ID)).rejects.toThrow("audit store down"); + expect((await deploymentRecords(ws)).map(r => r.method)).toStrictEqual(["inspectWorkspace"]); + })); + + it("is held to the environment's cap whatever the setting says", () => withWorkspace(async ws => { + ws.config.adminWorkspaceAccess = "full"; + let view = await open(ws, { access: "full" }); + expect(await codeOf(view.getChat(CHAT_ID))).toBe(ADMIN_ACCESS_ERROR_CODES.contentWithheld); + expect((await deploymentRecords(ws)).map(r => r.access)).toStrictEqual(["metadata", "metadata"]); + })); + + it("opens at the deployment's level when it asks for more, and is never raised", + () => withWorkspace(async ws => { + // As AdminApiImpl mints a view that asked for "full" while the deployment allows "metadata". + let view = await open(ws, { requested: "full", access: "metadata" }); + expect(await codeOf(view.getChat(CHAT_ID))).toBe(ADMIN_ACCESS_ERROR_CODES.contentWithheld); + expect((await deploymentRecords(ws)).map(r => [r.method, r.requested, r.access, r.exposed])) + .toStrictEqual([ + ["inspectWorkspace", "full", "metadata", "metadata"], + ["getChat", "full", "metadata", "metadata"], + ]); + + setEnv(ws, { ADMIN_WORKSPACE_ACCESS_MAX: "full" }); + ws.config.adminWorkspaceAccess = "full"; + expect(await codeOf(view.getChat(CHAT_ID))).toBe(ADMIN_ACCESS_ERROR_CODES.contentWithheld); + expect((await view.describe()).access).toBe("metadata"); + })); + + it("withholds from its next call once the setting drops to metadata", + () => withWorkspace(async ws => { + setEnv(ws, { ADMIN_WORKSPACE_ACCESS_MAX: "full" }); + ws.config.adminWorkspaceAccess = "full"; + let view = await open(ws, { access: "full" }); + let seededChat = async () => (await view.listChats()).find(chat => chat.id === CHAT_ID)!; + expect((await seededChat()).queuedSteers).toBeDefined(); + + ws.config.adminWorkspaceAccess = "metadata"; + expect(await seededChat()).not.toHaveProperty("queuedSteers"); + expect(await codeOf(view.readFile(ws.gadgetId, "notes.txt"))) + .toBe(ADMIN_ACCESS_ERROR_CODES.contentWithheld); + })); +}); + +describe("a live view's checks", () => { + it("refuses, recorded, a call once the setting is off", () => withWorkspace(async ws => { + let view = await open(ws); + ws.config.adminWorkspaceAccess = "off"; + expect(await codeOf(view.describe())).toBe(ADMIN_ACCESS_ERROR_CODES.accessOff); + let records = await deploymentRecords(ws); + expect(records.at(-1)).toMatchObject( + { method: "describe", access: "off", refused: ADMIN_ACCESS_ERROR_CODES.accessOff }); + })); + + it("refuses, unrecorded, a call after expiry or demotion, or on a deleted workspace", + () => withWorkspace(async ws => { + let shortLived = await open(ws, { expiresAt: new Date(Date.now() + SHORT_LIFETIME_MS) }); + let view = await open(ws); + await scheduler.wait(SHORT_LIFETIME_MS + 100); + expect(await codeOf(shortLived.listChats())).toBe(ADMIN_ACCESS_ERROR_CODES.viewExpired); + + setEnv(ws, { ADMINS: ["someone-else@example.test"] }); + expect(await codeOf(view.listChats())).toBe(ADMIN_ACCESS_ERROR_CODES.notAdmin); + setEnv(ws); + + ws.impl.ownerId = undefined; + expect(await codeOf(view.listChats())).toBe(ADMIN_ACCESS_ERROR_CODES.workspaceNotFound); + + expect((await deploymentRecords(ws)).map(r => r.method)) + .toStrictEqual(["inspectWorkspace", "inspectWorkspace"]); + })); + + it("writes nothing for a call that the workspace's deletion overtakes", + () => withWorkspace(async ws => { + let view = await open(ws); + let reached!: () => void; + let atRead = new Promise(resolve => { reached = resolve; }); + let release!: () => void; + let released = new Promise(resolve => { release = resolve; }); + ws.kvGet = async key => { + reached(); + await released; + return serveConfig(ws, key); + }; + + let call = view.listChats(); + await atRead; + await ws.state.storage.deleteAll(); + ws.impl.ownerId = undefined; + release(); + + expect(await codeOf(call)).toBe(ADMIN_ACCESS_ERROR_CODES.workspaceNotFound); + expect(storageKeys(ws)).toStrictEqual([]); + expect((await deploymentRecords(ws)).map(r => r.method)).toStrictEqual(["inspectWorkspace"]); + })); + + it("writes nothing for a call that expires while it reads the setting", + () => withWorkspace(async ws => { + let view = await open(ws, { expiresAt: new Date(Date.now() + SHORT_LIFETIME_MS) }); + let reached!: () => void; + let atRead = new Promise(resolve => { reached = resolve; }); + let release!: () => void; + let released = new Promise(resolve => { release = resolve; }); + ws.kvGet = async key => { + reached(); + await released; + return serveConfig(ws, key); + }; + + let call = view.listChats(); + await atRead; + await scheduler.wait(SHORT_LIFETIME_MS + 100); + release(); + + expect(await codeOf(call)).toBe(ADMIN_ACCESS_ERROR_CODES.viewExpired); + expect((await deploymentRecords(ws)).map(r => r.method)).toStrictEqual(["inspectWorkspace"]); + })); +}); + +// Holds describe() at its read of the owner's profile until `release`. +function holdDescribe(ws: Workspace): { reached: Promise, release: () => void } { + let reach!: () => void; + let reached = new Promise(resolve => { reach = resolve; }); + let release!: () => void; + let released = new Promise(resolve => { release = resolve; }); + ws.impl.users = { + idFromString: (id: string) => id, + get: () => ({ whoami: async () => { reach(); await released; return OWNER; } }), + }; + return { reached, release }; +} + +describe("a call overtaken while it reads its answer", () => { + it("writes nothing when the workspace is deleted", () => withWorkspace(async ws => { + let view = await open(ws); + let hold = holdDescribe(ws); + + let call = view.describe(); + await hold.reached; + await ws.state.storage.deleteAll(); + ws.impl.ownerId = undefined; + hold.release(); + + expect(await codeOf(call)).toBe(ADMIN_ACCESS_ERROR_CODES.workspaceNotFound); + expect(storageKeys(ws)).toStrictEqual([]); + expect((await deploymentRecords(ws)).map(r => r.method)).toStrictEqual(["inspectWorkspace"]); + })); + + it("writes nothing when the view expires", () => withWorkspace(async ws => { + let view = await open(ws, { expiresAt: new Date(Date.now() + SHORT_LIFETIME_MS) }); + let hold = holdDescribe(ws); + + let call = view.describe(); + await hold.reached; + await scheduler.wait(SHORT_LIFETIME_MS + 100); + hold.release(); + + expect(await codeOf(call)).toBe(ADMIN_ACCESS_ERROR_CODES.viewExpired); + expect((await deploymentRecords(ws)).map(r => r.method)).toStrictEqual(["inspectWorkspace"]); + })); +}); + +describe("the records' bounds", () => { + it("cut a long title, while describe() returns it whole", + () => withWorkspace(async ws => { + let title = "t".repeat(2_000_000); + ws.impl.storage.title.put(title); + let view = await open(ws); + expect((await view.describe()).title).toBe(title); + + let cut = `${"t".repeat(MAX_ADMIN_AUDIT_ARG_LENGTH - 1)}…`; + let records = await deploymentRecords(ws); + expect(records.map(r => r.workspaceTitle)).toStrictEqual([cut, cut]); + })); + + it("report the workspace's operational state in the diagnostics", () => withWorkspace(async ws => { + let view = await open(ws); + let diagnostics = await view.getDiagnostics(); + expect(diagnostics).toMatchObject({ + chats: 2, pendingActions: 3, hooks: { total: 1, enabled: 1 }, observers: 1, + ownerRegistrationPending: false, activeAgentChatIds: [], pendingAgentCalls: 1, + }); + expect(diagnostics.storageBytes).toBeGreaterThan(0); + // No member has the workspace open, and the admin's own view is not presence. + expect(diagnostics.present).toStrictEqual([]); + + let leave = ws.impl.joinPresence(BOB.id, BOB, "build"); + expect((await view.getDiagnostics()).present) + .toStrictEqual([{ key: expect.any(String), user: BOB, role: "build" }]); + leave(); + expect((await view.getDiagnostics()).present).toStrictEqual([]); + })); +}); + +// Deletes a stored object that a connection could serve again, and makes reaching any connection's +// facet visible to the test. +function forbidPulls(ws: Workspace, oid: string, type: "blob" | "tree" | "commit") + : { pulled: boolean } { + let state = { pulled: false }; + ws.impl.storage.gitObjectMetadata.put( + { oid, type, onRemote: [GATEKEEPER_ID], pullableFrom: [], pendingPush: [] }); + ws.impl.storage.gitObjects.delete(oid); + ws.impl.getGatekeeperFacet = () => { + state.pulled = true; + throw new Error("an admin read reached a connection"); + }; + return state; +} + +describe("file reads", () => { + it("never pull a missing blob through a connection", () => withWorkspace(async ws => { + setEnv(ws, { ADMIN_WORKSPACE_ACCESS_MAX: "full" }); + ws.config.adminWorkspaceAccess = "full"; + let view = await open(ws, { access: "full" }); + let entry = await ws.impl.gitCache.pathEntryAtCommit(ws.commitId, "notes.txt"); + let pulls = forbidPulls(ws, entry.oid, "blob"); + + await expect(view.readFile(ws.gadgetId, "notes.txt")) + .rejects.toThrow(`git object ${entry.oid} is not stored in this workspace`); + expect(pulls.pulled).toBe(false); + // Recorded as an attempt that exposed nothing. + let records = await deploymentRecords(ws); + expect(records.at(-1)) + .toMatchObject({ method: "readFile", access: "full", exposed: "metadata" }); + expect(records.at(-1)).not.toHaveProperty("refused"); + })); + + it("never pull a missing tree through a connection", () => withWorkspace(async ws => { + let view = await open(ws); + let root = await ws.impl.gitCache.pathEntryAtCommit(ws.commitId, ""); + let pulls = forbidPulls(ws, root.oid, "tree"); + + await expect(view.listFiles(ws.gadgetId)) + .rejects.toThrow(`git object ${root.oid} is not stored in this workspace`); + expect(pulls.pulled).toBe(false); + })); + + it("read only a permanent gadget's head", () => withWorkspace(async ws => { + let view = await open(ws); + await expect(view.listFiles(GATEKEEPER_ID)) + .rejects.toThrow(`No gadget with committed code has id ${GATEKEEPER_ID}.`); + await expect(view.listFiles(999)).rejects.toThrow("No gadget with committed code has id 999."); + })); +}); + +describe("chat reads", () => { + it("report a chat with nothing recorded or waiting as such", () => withWorkspace(async ws => { + addIdleChat(ws); + let view = await open(ws); + let state = await view.getChatState(IDLE_CHAT_ID); + expect(state.turn).toBeUndefined(); + expect(state.lastMessageAt).toBeUndefined(); + expect(state.queuedCalls).toStrictEqual([]); + expect(state.pendingActionIds).toStrictEqual([]); + expect(await view.listChatAttachments(IDLE_CHAT_ID)).toStrictEqual([]); + // As for an id that names no chat, which only the listing answers. + expect(await view.listChatAttachments(999)).toStrictEqual([]); + })); + + it("record a read that fails as an attempt that exposed nothing", () => withWorkspace(async ws => { + setEnv(ws, { ADMIN_WORKSPACE_ACCESS_MAX: "full" }); + ws.config.adminWorkspaceAccess = "full"; + let view = await open(ws, { access: "full" }); + let elsewhere = "44444444-4444-4444-8444-444444444444"; + ws.impl.storage.chatAttachmentContent.put({ + fileId: elsewhere, data: new Uint8Array([1]), + state: { type: "committed", chatId: IDLE_CHAT_ID }, + }); + + await expect(view.getChatState(999)).rejects.toThrow("No such chatId: 999"); + await expect(view.readChatAttachment(CHAT_ID, "nope")) + .rejects.toThrow("Invalid chat attachment ID."); + await expect(view.readChatAttachment(CHAT_ID, elsewhere)) + .rejects.toThrow("Chat attachment not found."); + + let records = (await deploymentRecords(ws)).slice(1); + expect(records.map(r => [r.method, r.args, r.access, r.exposed])).toStrictEqual([ + ["getChatState", { chatId: 999 }, "full", "metadata"], + ["readChatAttachment", { chatId: CHAT_ID, id: "nope" }, "full", "metadata"], + ["readChatAttachment", { chatId: CHAT_ID, id: elsewhere }, "full", "metadata"], + ]); + for (let record of records) expect(record).not.toHaveProperty("refused"); + })); + + it("report a legacy attachment whose path names no file as missing from Git", + () => withWorkspace(async ws => { + setEnv(ws, { ADMIN_WORKSPACE_ACCESS_MAX: "full" }); + ws.config.adminWorkspaceAccess = "full"; + let view = await open(ws, { access: "full" }); + let gitCache = ws.impl.gitCache; + let commit = { + parents: [], + author: { name: "Owner", email: "owner@example.test" }, + message: "attachments", + timestamp: new Date(1700000000_000), + }; + // A commit whose tree holds a directory and a submodule. Writing files produces no submodule + // entry, so the tree is encoded here. + let withDir = await gitCache.writeFilesAsCommit( + new Map([["dir/inner.bin", gitCache.writeBlob(new Uint8Array([0x00, 0x01]))]]), commit); + let tree = encodeGitTree([ + { mode: "40000", name: "dir", oid: gitCache.localPathEntryAtCommit(withDir, "dir").oid }, + { mode: "160000", name: "vendored", oid: "1".repeat(40) }, + ]); + gitCache.importObjects([{ type: "tree", payload: tree }]); + let commitId = gitCache.writeCommitForTree(gitObjectOid("tree", tree), commit); + // Legacy records whose paths name nothing, the directory and the submodule. + let paths = new Map([ + ["55555555-5555-4555-8555-555555555555", "absent.bin"], + ["66666666-6666-4666-8666-666666666666", "dir"], + ["77777777-7777-4777-8777-777777777777", "vendored"], + ]); + for (let [fileId, workspacePath] of paths) { + ws.impl.storage.chatAttachmentContent.put({ + fileId, + state: { type: "committed", chatId: CHAT_ID, workspaceCommitId: commitId, workspacePath }, + }); + await expect(view.readChatAttachment(CHAT_ID, fileId)) + .rejects.toThrow("Chat attachment content is missing from Git."); + } + + let records = (await deploymentRecords(ws)).slice(1); + expect(records.map(r => [r.method, r.args, r.access, r.exposed])).toStrictEqual( + [...paths.keys()].map( + id => ["readChatAttachment", { chatId: CHAT_ID, id }, "full", "metadata"])); + for (let record of records) expect(record).not.toHaveProperty("refused"); + })); + + it("never pull an attachment through a connection", () => withWorkspace(async ws => { + setEnv(ws, { ADMIN_WORKSPACE_ACCESS_MAX: "full" }); + ws.config.adminWorkspaceAccess = "full"; + let view = await open(ws, { access: "full" }); + let gitCache = ws.impl.gitCache; + + let pulls = forbidPulls(ws, ws.attachmentOid, "blob"); + await expect(view.readChatAttachment(CHAT_ID, BLOB_ATTACHMENT_ID)) + .rejects.toThrow("Unknown Git blob."); + expect(pulls.pulled).toBe(false); + + // The legacy attachment's file, then the tree that names it. + let file = await gitCache.pathEntryAtCommit(ws.legacyCommitId, "legacy.pdf"); + let root = await gitCache.pathEntryAtCommit(ws.legacyCommitId, ""); + pulls = forbidPulls(ws, file.oid, "blob"); + await expect(view.readChatAttachment(CHAT_ID, LEGACY_ATTACHMENT_ID)) + .rejects.toThrow("Unknown Git blob."); + expect(pulls.pulled).toBe(false); + + pulls = forbidPulls(ws, root.oid, "tree"); + await expect(view.readChatAttachment(CHAT_ID, LEGACY_ATTACHMENT_ID)) + .rejects.toThrow(`git object ${root.oid} is not stored in this workspace`); + expect(pulls.pulled).toBe(false); + })); +}); + +// A view at "full", which the environment's cap and the setting both allow. +async function openFull(ws: Workspace) { + setEnv(ws, { ADMIN_WORKSPACE_ACCESS_MAX: "full" }); + ws.config.adminWorkspaceAccess = "full"; + return await open(ws, { access: "full" }); +} + +// What a read throws for an id that names no stored commit, and for an object that is missing. +const noStoredCommit = (oid: string) => `No commit ${oid} is stored in this workspace.`; +const missingObject = (oid: string) => `git object ${oid} is not stored in this workspace`; + +describe("history and stored commits", () => { + it("read a commit's history, its files and what changed since its parent", + () => withWorkspace(async ws => { + let view = await openFull(ws); + let head = { + oid: ws.commitId, parents: [ws.parentCommitId], message: "notes\n", author: COMMITTER, + timestamp: new Date(1700000000_000), + }; + let parent = { + oid: ws.parentCommitId, parents: [], message: "first notes\n", author: COMMITTER, + timestamp: new Date(1699990000_000), + }; + expect(await view.getCommitLog(ws.commitId)).toStrictEqual([head, parent]); + expect(await view.getCommitLog(ws.commitId, 1)).toStrictEqual([head]); + // A commit whose parent is not stored lists alone: the walk stops there. + expect((await view.getCommitLog(ws.orphanCommitId)).map(commit => [commit.oid, commit.parents])) + .toStrictEqual([[ws.orphanCommitId, [MISSING_PARENT]]]); + + // The parent is no gadget's head, and reads as one does. + expect(await view.listFilesAtCommit(ws.parentCommitId)).toStrictEqual([ + { name: "notes.txt", kind: "file" }, { name: "old.txt", kind: "file" }, + { name: "todo.txt", kind: "file" }, + ]); + expect(await view.readFileAtCommit(ws.parentCommitId, "notes.txt")) + .toStrictEqual({ kind: "text", text: "first notes\n" }); + expect(await view.readFileAtCommit(ws.parentCommitId, "old.txt")) + .toStrictEqual({ kind: "text", text: "dropped\n" }); + expect(await view.readFileAtCommit(ws.commitId, "old.txt")).toStrictEqual({ kind: "absent" }); + + expect(await view.listChangedPaths(ws.parentCommitId, ws.commitId)) + .toStrictEqual(["notes.txt", "old.txt"]); + expect(await view.listChangedPaths(ws.commitId, ws.parentCommitId)) + .toStrictEqual(["notes.txt", "old.txt"]); + expect(await view.listChangedPaths(ws.commitId, ws.commitId)).toStrictEqual([]); + })); + + it("list 20 commits of a longer history when no depth is given", () => withWorkspace(async ws => { + // A line of 22 commits, a second apart. Any stored commit can start a log, so it needs no + // gadget. + let oids: string[] = []; + for (let step = 1; step <= 22; step++) { + oids.push(await commitFiles(ws, { "step.txt": `${step}\n` }, { + parents: oids.slice(-1), message: `step ${step}`, + timestamp: new Date(1700000000_000 + step * 1000), + })); + } + let tip = oids.at(-1)!; + let newestFirst = oids.toReversed(); + + let view = await openFull(ws); + let log = await view.getCommitLog(tip); + expect(log).toHaveLength(DEFAULT_ADMIN_COMMIT_LOG_DEPTH); + expect(log.map(commit => commit.oid)).toStrictEqual(newestFirst.slice(0, 20)); + expect(log.at(-1)!.message).toBe("step 3\n"); + expect((await view.getCommitLog(tip, 21)).map(commit => commit.oid)) + .toStrictEqual(newestFirst.slice(0, 21)); + let whole = await view.getCommitLog(tip, MAX_ADMIN_COMMIT_LOG_DEPTH); + expect(whole.map(commit => commit.oid)).toStrictEqual(newestFirst); + expect(whole.at(-1)!.message).toBe("step 1\n"); + + let withheld = await (await open(ws)).getCommitLog(tip); + expect(withheld.map(commit => [commit.oid, commit.message])) + .toStrictEqual(newestFirst.slice(0, 20).map(oid => [oid, ""])); + + // A record keeps the depth as its caller gave it, not the one the read applied. + let records = (await deploymentRecords(ws)).filter(r => r.method === "getCommitLog"); + expect(records.map(r => [r.args, r.access, r.exposed])).toStrictEqual([ + [{ fromCommit: tip }, "full", "content"], + [{ fromCommit: tip, depth: 21 }, "full", "content"], + [{ fromCommit: tip, depth: MAX_ADMIN_COMMIT_LOG_DEPTH }, "full", "content"], + [{ fromCommit: tip }, "metadata", "metadata"], + ]); + })); + + it("refuse, unrecorded, a depth outside the log's bounds", () => withWorkspace(async ws => { + let view = await openFull(ws); + for (let depth of [0, MAX_ADMIN_COMMIT_LOG_DEPTH + 1, 1.5]) { + await expect(view.getCommitLog(ws.commitId, depth)).rejects.toThrow(TypeError); + await expect(view.getCommitLog(ws.commitId, depth)).rejects.toThrow( + `The commit log's depth must be a whole number from 1 to ${MAX_ADMIN_COMMIT_LOG_DEPTH}.`); + } + expect((await deploymentRecords(ws)).map(r => r.method)).toStrictEqual(["inspectWorkspace"]); + })); + + it("record an id that names no stored commit as a read that failed", + () => withWorkspace(async ws => { + let view = await openFull(ws); + let tree = ws.impl.gitCache.commitTree(ws.commitId); + let blob = ws.impl.gitCache.localPathEntryAtCommit(ws.commitId, "notes.txt").oid; + + await expect(view.getCommitLog("not-an-oid")).rejects.toThrow("Invalid commit id."); + await expect(view.listFilesAtCommit(MISSING_PARENT)) + .rejects.toThrow(noStoredCommit(MISSING_PARENT)); + await expect(view.readFileAtCommit(tree, "notes.txt")).rejects.toThrow(noStoredCommit(tree)); + await expect(view.listChangedPaths(ws.commitId, blob)).rejects.toThrow(noStoredCommit(blob)); + await expect(view.listChangedPaths(MISSING_PARENT, ws.commitId)) + .rejects.toThrow(noStoredCommit(MISSING_PARENT)); + await expect(view.getCommitLog(tree)).rejects.toThrow(noStoredCommit(tree)); + + let records = (await deploymentRecords(ws)).slice(1); + expect(records.map(r => [r.method, r.args, r.access, r.exposed])).toStrictEqual([ + ["getCommitLog", { fromCommit: "not-an-oid" }, "full", "metadata"], + ["listFilesAtCommit", { commitId: MISSING_PARENT }, "full", "metadata"], + ["readFileAtCommit", { commitId: tree, path: "notes.txt" }, "full", "metadata"], + ["listChangedPaths", { fromCommit: ws.commitId, toCommit: blob }, "full", "metadata"], + ["listChangedPaths", { fromCommit: MISSING_PARENT, toCommit: ws.commitId }, "full", + "metadata"], + ["getCommitLog", { fromCommit: tree }, "full", "metadata"], + ]); + for (let record of records) expect(record).not.toHaveProperty("refused"); + })); + + it("never pull a missing object through a connection", () => withWorkspace(async ws => { + let view = await openFull(ws); + let gitCache = ws.impl.gitCache; + + // A parent a connection could serve ends the walk, unasked. + let pulls = forbidPulls(ws, MISSING_PARENT, "commit"); + expect((await view.getCommitLog(ws.orphanCommitId)).map(commit => commit.oid)) + .toStrictEqual([ws.orphanCommitId]); + expect(pulls.pulled).toBe(false); + + let blob = gitCache.localPathEntryAtCommit(ws.parentCommitId, "old.txt").oid; + pulls = forbidPulls(ws, blob, "blob"); + await expect(view.readFileAtCommit(ws.parentCommitId, "old.txt")) + .rejects.toThrow(missingObject(blob)); + expect(pulls.pulled).toBe(false); + + let root = gitCache.commitTree(ws.parentCommitId); + pulls = forbidPulls(ws, root, "tree"); + await expect(view.listFilesAtCommit(ws.parentCommitId)).rejects.toThrow(missingObject(root)); + await expect(view.listChangedPaths(ws.parentCommitId, ws.commitId)) + .rejects.toThrow(missingObject(root)); + expect(pulls.pulled).toBe(false); + + // The head's own parent, once it is gone: the head lists alone. + pulls = forbidPulls(ws, ws.parentCommitId, "commit"); + expect((await view.getCommitLog(ws.commitId)).map(commit => commit.oid)) + .toStrictEqual([ws.commitId]); + expect(pulls.pulled).toBe(false); + })); +}); + +describe("a chat's pending changes", () => { + it("list what the chat changed against its pin's base, and read its files", + () => withWorkspace(async ws => { + let view = await openFull(ws); + expect(await view.listPendingChanges(DRAFT_CHAT_ID)).toStrictEqual([seededPendingChange(ws)]); + // A modified and an added path hold the chat's text, and a removed one nothing. + expect(await view.readPendingFile(DRAFT_CHAT_ID, ws.gadgetId, "notes.txt")) + .toStrictEqual({ kind: "text", text: "draft notes\n" }); + expect(await view.readPendingFile(DRAFT_CHAT_ID, ws.gadgetId, "added.txt")) + .toStrictEqual({ kind: "text", text: "added\n" }); + expect(await view.readPendingFile(DRAFT_CHAT_ID, ws.gadgetId, "todo.txt")) + .toStrictEqual({ kind: "absent" }); + // A chat that changed nothing holds no changes. + expect(await view.listPendingChanges(CHAT_ID)).toStrictEqual([]); + + // A gadget the chat created, not accepted yet: no pin, so no base, and every file is added. + let created = ws.impl.createGadget("Sketch", "SKETCH", DRAFT_CHAT_ID); + await agentStep(ws, DRAFT_CHAT_ID, + { [created.id]: [["main.js", { set: "code\n" }], ["lib/util.js", { set: "util\n" }]] }); + expect(await view.listPendingChanges(DRAFT_CHAT_ID)).toStrictEqual([ + seededPendingChange(ws), + { gadgetId: created.id, added: ["lib/util.js", "main.js"], modified: [], removed: [] }, + ]); + expect(await view.readPendingFile(DRAFT_CHAT_ID, created.id, "main.js")) + .toStrictEqual({ kind: "text", text: "code\n" }); + })); + + it("list one side of an accept whole, never the chat's files against the pins it cleared", + async () => { + // An accept clears the chat's pins in a synchronous tail, which can run in the microtask + // between the fold settling and the listing going on. The accept is started first and the + // listing a growing number of microtasks later, so that some offset lands in that gap: the + // listing then folds a second time, which is counted so that the sweep cannot miss the gap + // unnoticed. + let refolded = false; + for (let ticks = 0; ticks <= 16; ticks++) { + await withWorkspace(async ws => { + let view = await openFull(ws); + // Only the view passes a base reader, so these are its folds and not the accept's. + let folds = 0; + let fold = ws.impl.getCurrentChatContent.bind(ws.impl); + ws.impl.getCurrentChatContent = (...args: unknown[]) => { + if (args[2] !== undefined) folds++; + return fold(...args); + }; + let merge = ws.impl.mergeChanges(DRAFT_CHAT_ID, { profile: OWNER }, OWNER_USER_ID); + for (let i = 0; i < ticks; i++) await Promise.resolve(); + let listing = await view.listPendingChanges(DRAFT_CHAT_ID); + expect((await merge).outcome).toBe("merged"); + expect([[seededPendingChange(ws)], []], `${ticks} microtasks after the accept began`) + .toContainEqual(listing); + refolded ||= folds > 1; + }); + } + expect(refolded).toBe(true); + }); + + it("record a chat or a workpiece that holds none as a read that failed", + () => withWorkspace(async ws => { + let view = await openFull(ws); + await expect(view.readPendingFile(DRAFT_CHAT_ID, 999, "x")) + .rejects.toThrow(`Chat ${DRAFT_CHAT_ID} holds no changes to workpiece 999.`); + await expect(view.listPendingChanges(999)).rejects.toThrow("No such chatId: 999"); + await expect(view.readPendingFile(999, ws.gadgetId, "x")) + .rejects.toThrow("No such chatId: 999"); + + let records = (await deploymentRecords(ws)).slice(1); + expect(records.map(r => [r.method, r.args, r.access, r.exposed])).toStrictEqual([ + ["readPendingFile", { chatId: DRAFT_CHAT_ID, gadgetId: 999, path: "x" }, "full", "metadata"], + ["listPendingChanges", { chatId: 999 }, "full", "metadata"], + ["readPendingFile", { chatId: 999, gadgetId: ws.gadgetId, path: "x" }, "full", "metadata"], + ]); + for (let record of records) expect(record).not.toHaveProperty("refused"); + })); + + it("fold a worktree's changes from stored objects alone", () => withWorkspace(async ws => { + let view = await openFull(ws); + let { id, baseCommit } = await addWorktreeChat(ws); + + // same.txt, edited back to its base text, is in no list. + expect(await view.listPendingChanges(WORKTREE_CHAT_ID)).toStrictEqual([{ + gadgetId: id, baseCommit, mergedCommit: baseCommit, + added: ["new.txt"], modified: ["a.txt"], removed: ["gone.txt"], + }]); + expect(await view.readPendingFile(WORKTREE_CHAT_ID, id, "a.txt")) + .toStrictEqual({ kind: "text", text: "one!\n" }); + expect(await view.readPendingFile(WORKTREE_CHAT_ID, id, "gone.txt")) + .toStrictEqual({ kind: "absent" }); + // A file the chat never touched is not in its content: it reads at the base. + expect(await view.readPendingFile(WORKTREE_CHAT_ID, id, "src/untouched.txt")) + .toStrictEqual({ kind: "absent" }); + expect(await view.readFileAtCommit(baseCommit, "src/untouched.txt")) + .toStrictEqual({ kind: "text", text: "kept\n" }); + + // Without the edited file's base text the fold fails, and asks no connection for it. + let base = ws.impl.gitCache.localPathEntryAtCommit(baseCommit, "a.txt").oid; + let pulls = forbidPulls(ws, base, "blob"); + ws.impl.invalidateChatContent(WORKTREE_CHAT_ID); + await expect(view.listPendingChanges(WORKTREE_CHAT_ID)) + .rejects.toThrow(missingObject(base)); + await expect(view.readPendingFile(WORKTREE_CHAT_ID, id, "a.txt")) + .rejects.toThrow(missingObject(base)); + expect(pulls.pulled).toBe(false); + let records = (await deploymentRecords(ws)).slice(-2); + expect(records.map(r => [r.method, r.access, r.exposed])).toStrictEqual([ + ["listPendingChanges", "full", "metadata"], ["readPendingFile", "full", "metadata"], + ]); + })); + + it("never pull a worktree's base to tell its paths apart", () => withWorkspace(async ws => { + let view = await openFull(ws); + let { baseCommit } = await addWorktreeChat(ws); + // With the chat's content folded, a listing still reads the base's tree, which says what + // each path the chat touched was there. + await view.listPendingChanges(WORKTREE_CHAT_ID); + let root = ws.impl.gitCache.commitTree(baseCommit); + let pulls = forbidPulls(ws, root, "tree"); + await expect(view.listPendingChanges(WORKTREE_CHAT_ID)).rejects.toThrow(missingObject(root)); + expect(pulls.pulled).toBe(false); + })); + + it("stream a file the chat holds as bytes", () => withWorkspace(async ws => { + let view = await openFull(ws); + let { id, baseCommit } = await addWorktreeChat(ws); + let bytes = new Uint8Array([0x00, 0xff, 0x10, 0x80, 0x7f]); + // Bytes also replace two of the base's files: same.txt with other bytes, and + // src/untouched.txt with the very bytes it holds there, which changes nothing. + let set = (content: Uint8Array) => ({ setBytes: ws.impl.gitCache.writeBlob(content) }); + await agentStep(ws, WORKTREE_CHAT_ID, { [id]: [ + ["data.bin", set(bytes)], ["same.txt", set(bytes)], + ["src/untouched.txt", set(new TextEncoder().encode("kept\n"))], + ] }); + + let file = await view.readPendingFile(WORKTREE_CHAT_ID, id, "data.bin"); + expect(file).toMatchObject({ kind: "stream", byteLength: bytes.byteLength }); + expect(await streamBytes(file)).toStrictEqual(bytes); + expect(await view.listPendingChanges(WORKTREE_CHAT_ID)).toStrictEqual([{ + gadgetId: id, baseCommit, mergedCommit: baseCommit, + added: ["data.bin", "new.txt"], modified: ["a.txt", "same.txt"], removed: ["gone.txt"], + }]); + expect((await deploymentRecords(ws)).at(-2)) + .toMatchObject({ method: "readPendingFile", exposed: "content" }); + })); + + it("list workpieces by id and paths by name, in whatever order the chat changed them", + () => withWorkspace(async ws => { + let view = await openFull(ws); + // The chat changes its worktree before the seeded gadget, whose id is the lower one, and the + // worktree's paths in descending order: two modified, two removed and two added. A scratch + // file the base lacks, written and then removed, is in no list. + let { id, baseCommit } = await addWorktree(ws, + { "a.txt": "a\n", "b.txt": "b\n", "c.txt": "c\n", "d.txt": "d\n" }); + await agentStep(ws, WORKTREE_CHAT_ID, editChange(id, "d.txt", "d\n", "d!\n")); + await submit(ws, WORKTREE_CHAT_ID, { [id]: [ + ["c.txt", { set: "c!\n" }], ["b.txt", { remove: true }], ["a.txt", { remove: true }], + ["z.txt", { set: "z\n" }], ["y.txt", { set: "y\n" }], ["tmp.txt", { set: "t\n" }], + ] }); + await submit(ws, WORKTREE_CHAT_ID, { [id]: [["tmp.txt", { remove: true }]] }); + await submit(ws, WORKTREE_CHAT_ID, { [ws.gadgetId]: [["notes.txt", { set: "notes!\n" }]] }, + [{ gadgetId: ws.gadgetId, baseCommit: ws.commitId }]); + + expect(await view.listPendingChanges(WORKTREE_CHAT_ID)).toStrictEqual([ + { + gadgetId: ws.gadgetId, baseCommit: ws.commitId, mergedCommit: ws.commitId, + added: [], modified: ["notes.txt"], removed: [], + }, + { + gadgetId: id, baseCommit, mergedCommit: baseCommit, + added: ["y.txt", "z.txt"], modified: ["c.txt", "d.txt"], removed: ["a.txt", "b.txt"], + }, + ]); + })); + + it("leave out what a revert took back, and list what the chat changed after it", + () => withWorkspace(async ws => { + let view = await openFull(ws); + // The second of two steps is reverted, so what it touched is no longer the chat's: b.txt, + // which it removed, is in no list. A revert also starts the chat's stream of changes over, + // and the removal of c.txt is a row of the stream that follows it. + let { id, baseCommit } = await addWorktree(ws, + { "a.txt": "a\n", "b.txt": "b\n", "c.txt": "c\n" }); + await agentStep(ws, WORKTREE_CHAT_ID, editChange(id, "a.txt", "a\n", "a!\n")); + await agentStep(ws, WORKTREE_CHAT_ID, + { [id]: [["b.txt", { remove: true }], ["n.txt", { set: "n\n" }]] }); + let reverted = [...ws.impl.storage.chats.list()] + .findLast(msg => msg.chatId === WORKTREE_CHAT_ID && msg.type === "changes")!.sequence; + await ws.impl.revertChanges(WORKTREE_CHAT_ID, reverted, OWNER); + await submit(ws, WORKTREE_CHAT_ID, { [id]: [["c.txt", { remove: true }]] }); + + expect(await view.listPendingChanges(WORKTREE_CHAT_ID)).toStrictEqual([{ + gadgetId: id, baseCommit, mergedCommit: baseCommit, + added: [], modified: ["a.txt"], removed: ["c.txt"], + }]); + })); + + it("send a pending text by the size of its UTF-8 encoding", () => withWorkspace(async ws => { + let view = await openFull(ws); + // "é" is two bytes: the first text is exactly MAX_BUFFERED_BLOB_SIZE bytes in one character + // fewer, and the second one byte over it in exactly that many characters. + let atLimit = "é" + "a".repeat(MAX_BUFFERED_BLOB_SIZE - 2); + let overLimit = `${atLimit}a`; + expect(overLimit.length).toBe(MAX_BUFFERED_BLOB_SIZE); + await submit(ws, DRAFT_CHAT_ID, { [ws.gadgetId]: [["at-limit.txt", { set: atLimit }]] }); + await submit(ws, DRAFT_CHAT_ID, { [ws.gadgetId]: [["over-limit.txt", { set: overLimit }]] }); + + let whole = await view.readPendingFile(DRAFT_CHAT_ID, ws.gadgetId, "at-limit.txt"); + expect(whole.kind).toBe("text"); + expect(whole.kind === "text" && whole.text === atLimit).toBe(true); + + let streamed = await view.readPendingFile(DRAFT_CHAT_ID, ws.gadgetId, "over-limit.txt"); + expect(streamed).toMatchObject({ kind: "stream", byteLength: MAX_BUFFERED_BLOB_SIZE + 1 }); + let bytes = await streamBytes(streamed); + expect(bytes.byteLength).toBe(MAX_BUFFERED_BLOB_SIZE + 1); + expect(new TextDecoder().decode(bytes) === overLimit).toBe(true); + + let records = (await deploymentRecords(ws)).slice(1); + expect(records.map(r => [r.method, r.args.path, r.exposed])).toStrictEqual([ + ["readPendingFile", "at-limit.txt", "content"], + ["readPendingFile", "over-limit.txt", "content"], + ]); + })); + + it("list a pin rooted at a merge commit against that commit", () => withWorkspace(async ws => { + let impl = ws.impl; + let view = await openFull(ws); + // The gadget's head moves on to a child that adds a file. A pin does not follow the head. + let moved = await commitFiles(ws, + { "notes.txt": "private notes\n", "todo.txt": "todo\n", "other.txt": "other\n" }, + { parents: [ws.commitId], message: "other" }); + impl.storage.gadgets.put({ ...impl.storage.gadgets.get(ws.gadgetId), commitId: moved }); + expect(await view.listPendingChanges(DRAFT_CHAT_ID)).toStrictEqual([seededPendingChange(ws)]); + + // An update from mainline roots the pin at a merge of the head and the chat's files. The chat + // has changed nothing on top of it, while accepting would still change the gadget: by what + // differs between the head it merged and the merge. + expect(await impl.updateChatFromMainline(DRAFT_CHAT_ID, OWNER)) + .toStrictEqual({ conflictPaths: [] }); + let [rooted] = await view.listPendingChanges(DRAFT_CHAT_ID); + let merge = rooted.baseCommit!; + expect([ws.commitId, moved]).not.toContain(merge); + expect(rooted).toStrictEqual({ + gadgetId: ws.gadgetId, baseCommit: merge, mergedCommit: moved, + added: [], modified: [], removed: [], + }); + expect((await view.getCommitLog(merge, 1))[0].parents[0]).toBe(moved); + expect(await view.listChangedPaths(moved, merge)) + .toStrictEqual(["added.txt", "notes.txt", "todo.txt"]); + expect(await view.readPendingFile(DRAFT_CHAT_ID, ws.gadgetId, "other.txt")) + .toStrictEqual({ kind: "text", text: "other\n" }); + + // What the chat changes next is listed against the merge. + await submit(ws, DRAFT_CHAT_ID, { + [ws.gadgetId]: [["other.txt", { set: "other!\n" }], ["more.txt", { set: "more\n" }]], + }); + expect(await view.listPendingChanges(DRAFT_CHAT_ID)).toStrictEqual([{ + gadgetId: ws.gadgetId, baseCommit: merge, mergedCommit: moved, + added: ["more.txt"], modified: ["other.txt"], removed: [], + }]); + })); + + it("leave out a pin that outlived its workpiece", () => withWorkspace(async ws => { + let view = await openFull(ws); + await ws.impl.removeWorkpiece(ws.gadgetId); + expect(await view.listPendingChanges(DRAFT_CHAT_ID)).toStrictEqual([]); + await expect(view.readPendingFile(DRAFT_CHAT_ID, ws.gadgetId, "notes.txt")) + .rejects.toThrow(`Chat ${DRAFT_CHAT_ID} holds no changes to workpiece ${ws.gadgetId}.`); + let record = (await deploymentRecords(ws)).at(-1)!; + expect(record) + .toMatchObject({ method: "readPendingFile", access: "full", exposed: "metadata" }); + expect(record).not.toHaveProperty("refused"); + })); +}); + +// What a workspace refuses a grant that reaches a connection with no creationSpec. +const LEGACY_CONNECTION = "legacy connection that cannot verify collaborators' access"; + +describe("sharing reads", () => { + it("list observers, requirements, share links, rules and blueprints whole, at every level", + () => withWorkspace(async ws => { + let storage = ws.impl.storage; + // Three more observers, each record with every field it stores. Carol's is reserved: her + // first admission has verified nothing. One of Dave's choices is for a connection since + // removed. The stranger has no collaborator record. + let carol: AiChatAuthorInfo = { type: "user", id: "carol@example.test", name: "Carol" }; + let dave: AiChatAuthorInfo = { type: "user", id: "dave@example.test", name: "Dave" }; + for (let profile of [carol, dave]) { + storage.collaborators.put({ + profile, + addedBy: [{ type: "user", sharer: OWNER.id, created: new Date(0), role: "build" }], + }); + } + storage.observers.put( + { profileId: carol.id, observerId: "observer-carol", accountChoices: {} }); + storage.observers.put({ + profileId: dave.id, observerId: "observer-dave", + accountChoices: { 99: 9, [GATEKEEPER_ID]: 8 }, + }); + storage.observers.put({ + profileId: STRANGER_ID, observerId: "observer-stranger", + accountChoices: { [AMBIENT_GATEKEEPER_ID]: 4 }, + }); + // By profile id, each choice by connection id. An entry has its two fields and a choice its + // three: never the record's observer id, and a removed connection's choice has no vendor. + let observers = [ + ...seededObservers(), + { profile: carol, accountChoices: [] }, + { + profile: dave, + accountChoices: [ + { gatekeeperId: GATEKEEPER_ID, accountId: 8, vendorId: "testvendor" }, + { gatekeeperId: 99, accountId: 9, vendorId: undefined }, + ], + }, + { + profile: { type: "user", id: STRANGER_ID, name: STRANGER_ID }, + accountChoices: [ + { gatekeeperId: AMBIENT_GATEKEEPER_ID, accountId: 4, vendorId: "library" }, + ], + }, + ]; + + let metadata = await open(ws); + let full = await openFull(ws); + for (let view of [metadata, full]) { + expect(await view.listObservers()).toStrictEqual(observers); + // An ambient connection has a vendor, so a collaborator needs an account for it too. The + // gadget reaches only the connection it binds. The vendor is spelled as the connection's + // creationSpec spells it, where listObservers() and listGatekeepers() give it in lower case. + expect(await view.listObserverRequirements("workspace")) + .toStrictEqual([REPO_REQUIREMENT, LIBRARY_REQUIREMENT]); + expect(await view.listObserverRequirements({ gadgetId: ws.gadgetId })) + .toStrictEqual([REPO_REQUIREMENT]); + // Each link with its note, role and creator (the owner's profile, the collaborator's, and + // one made of an id), under its id: a link's record holds no key. + expect(await view.listShareLinks()).toStrictEqual(seededShareLinks(ws)); + expect(await view.listAutoApprovals()).toStrictEqual(seededAutoApprovals()); + // A creator is the same at both levels. + expect((await view.listGatekeepers()).map(({ id, createdBy }) => [id, createdBy])) + .toStrictEqual([[GATEKEEPER_ID, OWNER], [AMBIENT_GATEKEEPER_ID, undefined]]); + } + // Only a blueprint's id, and the screenshot URL that holds it, differ by level. + expect(await full.listBlueprints()).toStrictEqual([seededBlueprint()]); + let { screenshotUrl: _screenshotUrl, ...withheld } = seededBlueprint(); + expect(await metadata.listBlueprints()).toStrictEqual([{ ...withheld, id: "" }]); + })); + + it("list a rule that applies nothing, beside what says why", () => withWorkspace(async ws => { + ws.impl.storage.containsRestrictedData.put(true); + let view = await open(ws); + // The rule stays stored, and listed, once the workspace has read restricted data, and + // describe() reports that it has. + expect(await view.listAutoApprovals()).toStrictEqual(seededAutoApprovals()); + expect((await view.describe()).containsRestrictedData).toBe(true); + // An action keeps its author's verdict and its kind at every level, beside its emptied + // description. + let { entries } = await view.listActions(); + expect(entries.find(entry => entry.type === "action")!.description).toMatchObject({ + description: "", autoApprovable: true, actionKind: { tag: "edit", label: "Edits" }, + }); + })); + + it("ask the owner's account for a share link's creator only when the owner created one", + () => withWorkspace(async ws => { + let storage = ws.impl.storage; + let user = ws.impl.users.get(); + let asked = 0; + ws.impl.users = { + idFromString: (id: string) => id, + get: () => ({ ...user, whoami: async () => { asked++; return OWNER; } }), + }; + // A second link of the owner's: one lookup serves both. + storage.shareKeys.put( + { id: "d4".repeat(32), created: new Date(4000), createdBy: OWNER.id, role: "build" }); + let view = await open(ws); + expect(asked).toBe(0); + + let links = await view.listShareLinks(); + expect(links.map(link => link.createdBy.id)) + .toStrictEqual([OWNER.id, BOB.id, STRANGER_ID, OWNER.id]); + expect(asked).toBe(1); + + // With none of the links the owner's, their account is not asked at all. + storage.shareKeys.delete(OWNER_LINK_ID); + storage.shareKeys.delete("d4".repeat(32)); + asked = 0; + expect((await view.listShareLinks()).map(link => link.linkId)) + .toStrictEqual([BOB_LINK_ID, STRANGER_LINK_ID]); + expect(asked).toBe(0); + })); + + it("record a grant whose requirements cannot be listed as a read that failed", + () => withWorkspace(async ws => { + let impl = ws.impl; + let view = await open(ws); + // A connection made before connections could verify collaborators, which no gadget binds. + let legacyId = impl.allocateWorkpieceId(); + impl.storage.gatekeepers.put({ id: legacyId, resourceTitle: "Old", class: {} as any }); + + await expect(view.listObserverRequirements({ gadgetId: 999 })) + .rejects.toThrow("No such gadget: 999"); + // "workspace" reaches every connection, so the legacy one fails it. + await expect(view.listObserverRequirements("workspace")).rejects.toThrow(LEGACY_CONNECTION); + // The gadget does not reach it, so its grant answers. + expect(await view.listObserverRequirements({ gadgetId: ws.gadgetId })) + .toStrictEqual([REPO_REQUIREMENT]); + // Once the gadget binds it, the gadget's grant fails as well. + let gadget = impl.storage.gadgets.get(ws.gadgetId); + impl.storage.gadgets.put( + { ...gadget, bindings: { ...gadget.bindings, OLD: { target: legacyId } } }); + await expect(view.listObserverRequirements({ gadgetId: ws.gadgetId })) + .rejects.toThrow(LEGACY_CONNECTION); + + let records = (await deploymentRecords(ws)).slice(1); + expect(records.map(r => [r.method, r.args, r.access, r.exposed])).toStrictEqual([ + ["listObserverRequirements", { gadgetId: 999 }, "metadata", "metadata"], + ["listObserverRequirements", { grant: "workspace" }, "metadata", "metadata"], + ["listObserverRequirements", { gadgetId: ws.gadgetId }, "metadata", "metadata"], + ["listObserverRequirements", { gadgetId: ws.gadgetId }, "metadata", "metadata"], + ]); + for (let record of records) expect(record).not.toHaveProperty("refused"); + })); + + it("never reach a connection", () => withWorkspace(async ws => { + let reached = 0; + ws.impl.getGatekeeperFacet = () => { + reached++; + throw new Error("an admin read reached a connection"); + }; + for (let view of [await open(ws), await openFull(ws)]) { + await view.listGatekeepers(); + await view.listObservers(); + await view.listObserverRequirements("workspace"); + await view.listObserverRequirements({ gadgetId: ws.gadgetId }); + await view.listShareLinks(); + await view.listAutoApprovals(); + await view.listBlueprints(); + await view.getDiagnostics(); + } + expect(reached).toBe(0); + })); +}); diff --git a/packages/workshop-backend/__tests__/connection-creator.test.ts b/packages/workshop-backend/__tests__/connection-creator.test.ts new file mode 100644 index 0000000000..7530078611 --- /dev/null +++ b/packages/workshop-backend/__tests__/connection-creator.test.ts @@ -0,0 +1,221 @@ +// Who created a connection (GatekeeperRecord.createdBy): the member whose session made it, or the +// initiator of the agent turn whose tool call created its resource. An ambient capsule, which the +// workspace installs from its owner's singleton accounts, has none. +// +// Runs against a real OverseerDurableObject (the TEST_OVERSEER binding). Gatekeeper facets and the +// users' Durable Objects are fakes: a facet whose describe() resolves is all a creation needs. + +import { describe, expect, it } from "vitest"; +import { env, RpcStub as NativeRpcStub } from "cloudflare:workers"; +import { runInDurableObject } from "cloudflare:test"; +import type { + AiChatAuthorInfo, GatekeeperCreationSpec, Overseer, +} from "@gadgets/workshop-shared/api"; +import type { OverseerDurableObject } from "../src/overseer.js"; +import type { GatekeeperRecord } from "../src/storage-schema/overseer-storage.js"; +// Load the whole backend (the pool's `main`) up front, so its slow load does not count against the +// first test's timeout. +import "./test-worker.js"; + +declare module "cloudflare:workers" { + interface ProvidedEnv { + TEST_OVERSEER: DurableObjectNamespace; + } +} + +const OWNER_USER_ID = "owner-user-do"; +const COLLABORATOR_USER_ID = "collaborator-user-do"; +const OWNER: AiChatAuthorInfo = { type: "user", id: "owner@example.test", name: "Owner" }; +const COLLABORATOR: AiChatAuthorInfo = { type: "user", id: "carol@example.test", name: "Carol" }; +// A spawned agent's turn is initiated by the gadget that spawned it. +const GADGET: AiChatAuthorInfo = { type: "gadget", id: OWNER.id, name: "Mailer" }; +const PROFILES: Record = { + [OWNER_USER_ID]: OWNER, + [COLLABORATOR_USER_ID]: COLLABORATOR, +}; + +const CHAT_ID = 1; +// Another chat of the workspace, with a turn of its own under way. +const OTHER_CHAT_ID = 2; +const ACCOUNT_ID = 7; +const VENDOR_ID = "TestVendor"; +const URL_PATTERN = "https://example.com/*"; +const CONNECTION_SPEC = { + type: "gatekeeper" as const, + vendorId: VENDOR_ID, + resourceUrl: "https://example.com/doc", + typeUrlPattern: URL_PATTERN, +}; + +// A user's Durable Object, as far as creating a connection calls it. +function fakeUser(userId: string) { + let profile = PROFILES[userId]; + return { + id: userId, + whoami: async () => profile, + getGatekeeperClassFor: async () => ({ + class: {}, vendorId: VENDOR_ID, typeUrlPattern: URL_PATTERN, + }), + getChatContext: async () => ({ + profile, + aiModel: { + profile: { type: "agent", id: "some-model", name: "Model" }, + config: { provider: "anthropic", model: "some-model", apiToken: "unused" }, + }, + }), + createResourceGatekeeper: async () => ({ + class: {}, typeUrlPattern: URL_PATTERN, + action: { title: "Create", description: "Create", implementsRevert: false }, + }), + recordSharedWorkspaceOpen: async () => {}, + }; +} + +// A workspace owned by OWNER_USER_ID, with the facets and the users' Durable Objects faked. +async function withImpl( + fn: (impl: any, instance: OverseerDurableObject) => Promise): Promise { + let stub = env.TEST_OVERSEER.getByName(`connection-creator-${crypto.randomUUID()}`); + await runInDurableObject(stub, async (instance: OverseerDurableObject) => { + let impl = (instance as unknown as { impl: any }).impl; + impl.ownerId = OWNER_USER_ID; + impl.users = { idFromString: (id: string) => id, get: fakeUser }; + impl.wrapUserDo = (user: unknown) => user; + impl.getGatekeeperFacet = () => ({ + describe: async () => ({ title: "Test", url: "https://example.com/doc" }), + }); + // A real restart would abort the test's Durable Object. + impl.scheduleAccessRestart = async () => {}; + await fn(impl, instance); + }); +} + +// Like withImpl, but also opens the real OverseerClientInterface over the impl, as the owner or as +// a "build" collaborator. The open-time work that is not about the session itself (ambient +// capsules, the deployment directory, the outputs index, observer verification) is stubbed. +async function withClient( + userId: string, fn: (impl: any, client: Overseer) => Promise): Promise { + await withImpl(async (impl, instance) => { + impl.ensureAmbientCapsules = async () => {}; + impl.syncDirectory = () => {}; + impl.markOutputsDirty = () => {}; + impl.syncOutputsTo = async () => {}; + impl.authorizeCollaborator = async () => ({ role: "build" }); + // The pool binds no model gatekeeper class, so its ctx.exports has none to mint a model + // connection from. The facets are fakes, so the class is stored and never instantiated. + Object.defineProperty(impl.ctx, "exports", { value: Object.create(impl.ctx.exports, { + LanguageModelGatekeeper: { value: () => ({}) }, + }) }); + let client = await instance.open( + userId, PROFILES[userId].id, new NativeRpcStub<() => void>(() => {})); + await fn(impl, client); + }); +} + +// The workspace's one connection. +function onlyRecord(impl: any): GatekeeperRecord { + let records: GatekeeperRecord[] = [...impl.storage.gatekeepers.list()]; + expect(records).toHaveLength(1); + return records[0]; +} + +describe("addGatekeeper", () => { + it("records the creator it is given", () => withImpl(async impl => { + await impl.addGatekeeper( + {} as any, CONNECTION_SPEC, OWNER_USER_ID, undefined, undefined, COLLABORATOR); + + expect(onlyRecord(impl).createdBy).toEqual(COLLABORATOR); + })); + + it("records no creator when it is given none", () => withImpl(async impl => { + await impl.addGatekeeper({} as any, CONNECTION_SPEC, OWNER_USER_ID); + + expect(onlyRecord(impl)).not.toHaveProperty("createdBy"); + })); +}); + +// Each way a session creates a connection, with the creationSpec of the record it stores. +const CLIENT_PATHS: [string, GatekeeperCreationSpec, (client: Overseer) => unknown][] = [ + ["newGatekeeper", CONNECTION_SPEC, + client => client.newGatekeeper(ACCOUNT_ID, CONNECTION_SPEC.resourceUrl)], + ["newAiModelGatekeeper", + { type: "aiModel", modelId: "some-model", provider: "anthropic", modelName: "some-model" }, + client => client.newAiModelGatekeeper("some-model")], + ["newAgentSpawnerGatekeeper", + { type: "agentSpawner", config: { displayName: "Helper", modelId: null } }, + client => client.newAgentSpawnerGatekeeper({ displayName: "Helper", env: {}, modelId: null })], +]; + +describe("a connection made in a member's session", () => { + it.each(CLIENT_PATHS)("%s records the owner in the owner's session", + (_method, creationSpec, create) => withClient(OWNER_USER_ID, async (impl, client) => { + await create(client); + + expect(onlyRecord(impl)).toMatchObject({ creationSpec, createdBy: OWNER }); + })); + + it.each(CLIENT_PATHS)("%s records the collaborator in a collaborator's session", + (_method, creationSpec, create) => withClient(COLLABORATOR_USER_ID, async (impl, client) => { + await create(client); + + expect(onlyRecord(impl)).toMatchObject({ creationSpec, createdBy: COLLABORATOR }); + })); +}); + +describe("an ambient capsule", () => { + it("records no creator", () => withImpl(async impl => { + impl.ownerUserDo = () => ({ + listProvidedAccounts: async () => [{ + vendorId: VENDOR_ID, accountId: ACCOUNT_ID, + description: { singleton: { tsType: "Library" } }, + }], + getSingletonGatekeeperClass: async () => ({}), + }); + + await impl.ensureAmbientCapsules(); + + let record = onlyRecord(impl); + expect(record.creationSpec).toEqual( + { type: "ambient", vendorId: VENDOR_ID, accountId: ACCOUNT_ID }); + expect(record).not.toHaveProperty("createdBy"); + })); +}); + +const RESOURCE_INPUT = { + vendorId: VENDOR_ID, resourceUrlPattern: URL_PATTERN, title: "Notes", bindingName: "NOTES", +}; + +// Like withImpl, for createExternalResource(): the creation queues an action for approval, whose +// trace needs a running agent. A collaborator's turn is under way in another chat, so that only +// the turn of the chat that creates the resource can name its creator. +function withResourceImpl(fn: (impl: any) => Promise): Promise { + return withImpl(async impl => { + impl.traceAgentActionApproval = () => {}; + impl.storage.activeAgents.put({ + chatId: OTHER_CHAT_ID, initiatorUserId: COLLABORATOR_USER_ID, modelId: "some-model", + initiator: COLLABORATOR, callbackInitiated: false, + }); + await fn(impl); + }); +} + +describe("a resource an agent creates", () => { + it.each([OWNER, GADGET])("records the initiator of its turn, a $type", + initiator => withResourceImpl(async impl => { + impl.storage.activeAgents.put({ + chatId: CHAT_ID, initiatorUserId: OWNER_USER_ID, modelId: "some-model", initiator, + callbackInitiated: initiator.type === "gadget", + }); + + await impl.createExternalResource(CHAT_ID, RESOURCE_INPUT); + + expect(onlyRecord(impl)).toMatchObject({ resourceTitle: "Notes", createdBy: initiator }); + })); + + it("records no creator when its turn has no record", () => withResourceImpl(async impl => { + await impl.createExternalResource(CHAT_ID, RESOURCE_INPUT); + + let record = onlyRecord(impl); + expect(record.resourceTitle).toBe("Notes"); + expect(record).not.toHaveProperty("createdBy"); + })); +}); diff --git a/packages/workshop-backend/__tests__/deployment-admins.test.ts b/packages/workshop-backend/__tests__/deployment-admins.test.ts index 5d778930d3..33999c2e02 100644 --- a/packages/workshop-backend/__tests__/deployment-admins.test.ts +++ b/packages/workshop-backend/__tests__/deployment-admins.test.ts @@ -24,6 +24,12 @@ describe("isDeploymentAdmin", () => { expect(isDeploymentAdmin(env, "admin@example.com")).toBe(true); }); + it("never takes an empty or absent name for an admin", () => { + let env = withAdmins([""]); + expect(isDeploymentAdmin(env, "")).toBe(false); + expect(isDeploymentAdmin(env, undefined)).toBe(false); + }); + it("names no admin while the binding is unset, and refuses one that is not a list", () => { expect(isDeploymentAdmin(withAdmins(undefined), "admin")).toBe(false); expect(() => isDeploymentAdmin(withAdmins(JSON.stringify({ admin: true })), "admin")) diff --git a/packages/workshop-backend/__tests__/fixtures.ts b/packages/workshop-backend/__tests__/fixtures.ts index 30f5a6f5e4..e2798eb392 100644 --- a/packages/workshop-backend/__tests__/fixtures.ts +++ b/packages/workshop-backend/__tests__/fixtures.ts @@ -89,6 +89,7 @@ export async function openFakeOverseer( ensureAmbientCapsules: async () => {}, markOutputsDirty: () => {}, trackUnreadReader: () => {}, + syncDirectory: () => {}, joinSession: () => () => {}, joinPresence: () => () => {}, joinOutputsFanout: () => () => {}, diff --git a/packages/workshop-backend/__tests__/git-cache-history.test.ts b/packages/workshop-backend/__tests__/git-cache-history.test.ts index 1a4ed629f5..8e9af9acee 100644 --- a/packages/workshop-backend/__tests__/git-cache-history.test.ts +++ b/packages/workshop-backend/__tests__/git-cache-history.test.ts @@ -64,6 +64,27 @@ async function writeFixtureHistory(store: WorkspaceGitCache): Promise { expect(second).toBe(SECOND_COMMIT_OID); } +/** + * A store with a writer of empty-tree commits, each dated a second after the one before, so that + * a log lists them newest first: the reverse of the order they were written in. + */ +function makeHistory() { + let store = makeStore().store; + let seconds = 100; + let commit = (message: string, parents: string[]) => + store.writeFilesAsCommit(new Map(), { + parents, author: ALICE, message, timestamp: new Date(seconds++ * 1000), + }); + return { store, commit }; +} + +/** Commits that no store made by `makeHistory()` holds. */ +const MISSING = COMMIT_1; +const ALSO_MISSING = COMMIT_3; + +/** What a local history read throws for a commit the store does not hold. */ +const notHeld = (oid: string) => `The commit ${oid} is not in the workspace's git store.`; + describe("local history", () => { it("writes commits byte-identical to real git", async () => { // writeFixtureHistory asserts the known-good oids, including a nested tree (lib/util.js), a @@ -175,6 +196,60 @@ describe("local history", () => { expect(await messages(skewed)).toEqual(["merge", "left", "shared", "root", "right"]); }); + it("ends a line of readCommitLog at a parent that is not stored, with stopAtMissing", + async () => { + // The shape of a worktree's checkout: a commit pulled without the history behind it. + let { store, commit } = makeHistory(); + let oldest = await commit("oldest stored", [MISSING]); + let tip = await commit("tip", [oldest]); + + let log = store.readCommitLog(tip, { stopAtMissing: true }); + expect(log.map(entry => entry.oid)).toEqual([tip, oldest]); + // The commit that ends the line still names the parent the walk did not follow. + expect(log[1].parents).toEqual([MISSING]); + expect(() => store.readCommitLog(tip)).toThrow(notHeld(MISSING)); + expect(() => store.readCommitLog(tip, { stopAtMissing: false })).toThrow(notHeld(MISSING)); + }); + + it("lists the stored line of a merge whose other parents are not stored, with stopAtMissing", + async () => { + let { store, commit } = makeHistory(); + let root = await commit("root", []); + let line = await commit("line", [root]); + for (let parents of [[line, MISSING], [MISSING, line], [MISSING, line, ALSO_MISSING]]) { + let merge = await commit("merge", parents); + let log = store.readCommitLog(merge, { stopAtMissing: true }); + expect(log.map(entry => entry.oid)).toEqual([merge, line, root]); + expect(log[0].parents).toEqual(parents); + expect(() => store.readCommitLog(merge)).toThrow(/is not in the workspace's git store/); + } + }); + + it("bounds readCommitLog by depth with stopAtMissing too, and still requires its tip", + async () => { + let { store, commit } = makeHistory(); + let oldest = await commit("oldest stored", [MISSING]); + let middle = await commit("middle", [oldest]); + let tip = await commit("tip", [middle]); + let oids = (depth: number) => + store.readCommitLog(tip, { depth, stopAtMissing: true }).map(entry => entry.oid); + expect(oids(1)).toEqual([tip]); + expect(oids(2)).toEqual([tip, middle]); + expect(oids(10)).toEqual([tip, middle, oldest]); + + // A tip that is not stored throws, with the option or without. + for (let options of [{}, { stopAtMissing: true }]) { + expect(() => store.readCommitLog(MISSING, options)).toThrow(notHeld(MISSING)); + } + // A parent that is stored but is not a commit is not missing: it throws, as without the + // option. (The empty tree is always held.) + let odd = await commit("odd", [EMPTY_TREE]); + for (let options of [{}, { stopAtMissing: true }]) { + expect(() => store.readCommitLog(odd, options)) + .toThrow(`git object ${EMPTY_TREE} is a tree, but a commit was expected`); + } + }); + it("writes and round-trips an empty-tree commit", async () => { // An accepted gadget creation with no files yet commits an empty tree (see mergeChanges), // so the empty map must produce a valid commit -- byte-identical to real git's, over the diff --git a/packages/workshop-backend/__tests__/git-cache.test.ts b/packages/workshop-backend/__tests__/git-cache.test.ts index 35d0722967..01967db301 100644 --- a/packages/workshop-backend/__tests__/git-cache.test.ts +++ b/packages/workshop-backend/__tests__/git-cache.test.ts @@ -11,6 +11,7 @@ import { makeMockStorage } from "./mock-storage"; import { EAGER_BLOB_LIMIT, GitCacheImpl, + UnreadableContentError, WorkspaceGitCache, } from "../src/git-cache"; import { BlobReader, CHUNK_MAX_SIZE } from "../src/git-chunked-blob"; @@ -1743,6 +1744,9 @@ describe("changedFilePathsBetween", () => { ]); expect(await t.cache.changedFilePathsBetween(before, after)).toStrictEqual(expected); expect(await t.cache.changedFilePathsBetween(after, before)).toStrictEqual(expected); + // The local-only form walks the same trees. + expect(await t.cache.changedLocalFilePathsBetween(before, after)).toStrictEqual(expected); + expect(await t.cache.changedLocalFilePathsBetween(after, before)).toStrictEqual(expected); expect(t.pulls).toHaveLength(0); }); @@ -2173,3 +2177,310 @@ describe("chunked blob storage", () => { await (got.content as ReadableStream).cancel(); }); }); + +// The message of what `read` throws, or rejects with. +async function thrownBy(read: () => unknown): Promise { + try { + await read(); + } catch (err) { + return (err as Error).message; + } + throw new Error("expected a throw"); +} + +// What a local-only read throws for an object the store does not hold. +const notStored = (oid: GitOid) => `git object ${oid} is not stored in this workspace`; + +describe("local-only reads", () => { + // The fixture repo stored whole, every object attributed to G1, which could serve them again. + async function fixtureStoredLocally(): Promise { + let t = makeCache(); + t.sources.set(G1, fixtureSource(t, G1)); + for (let object of FIXTURE_OBJECTS) { + if (PACKED_OIDS.includes(object.oid)) { + await t.cache.putFromGatekeeper(G1, object.type, b64Bytes(object.payload)); + } + } + return t; + } + + it("give what the pulling reads give, without pulling", async () => { + let t = await fixtureStoredLocally(); + expect(t.cache.readLocalCommitTree(COMMIT_1)) + .toStrictEqual(await t.cache.readCommitTree(COMMIT_1)); + + let paths = ["README.md", "src/util.js", "nope.txt", "no/such/file.txt", "README.md/child", + "src", "link.md", "vendored"]; + let expected = new Map(await t.cache.readFilesAtCommit(COMMIT_1, paths)); + for (let path of paths) { + expect(t.cache.readLocalFileAtCommit(COMMIT_1, path)).toStrictEqual(expected.get(path)); + } + expect(t.pulls).toStrictEqual([]); + }); + + it("throw naming a missing object, rather than pulling it from a source that has it", + async () => { + // Only the commit is stored, its tree pullable from G1. + let t = makeCache(); + t.sources.set(G1, fixtureSource(t, G1)); + await t.cache.putFromGatekeeper(G1, "commit", fixture(COMMIT_1).payload); + expect(() => t.cache.readLocalCommitTree(COMMIT_1)) + .toThrow(`git object ${TREE_1} is not stored in this workspace`); + expect(() => t.cache.readLocalFileAtCommit(COMMIT_1, "README.md")) + .toThrow(`git object ${TREE_1} is not stored in this workspace`); + + // The trees stored, every blob still pullable from G1. + for (let object of FIXTURE_OBJECTS.filter(o => o.type === "tree")) { + if (PACKED_OIDS.includes(object.oid)) { + await t.cache.putFromGatekeeper(G1, object.type, b64Bytes(object.payload)); + } + } + expect(t.cache.readLocalCommitTree(COMMIT_1).map(node => node.name)) + .toStrictEqual(["README.md", "docs", "link.md", "run.sh", "src", "vendored"]); + let readmeBlob = "ca69e6d08b5b8bb4f11a74f9695e329c203cbfd8"; + let linkBlob = "42061c01a1c70097d1e4579f29a5adf40abdec95"; // link.md -> "README.md" + expect(() => t.cache.readLocalFileAtCommit(COMMIT_1, "README.md")) + .toThrow(`git object ${readmeBlob} is not stored in this workspace`); + expect(() => t.cache.readLocalFileAtCommit(COMMIT_1, "link.md")) + .toThrow(`git object ${linkBlob} is not stored in this workspace`); + expect(t.pulls).toStrictEqual([]); + }); + + it("reject an invalid path", async () => { + let t = await fixtureStoredLocally(); + for (let path of ["../escape", "src//util.js", "", "./README.md"]) { + expect(() => t.cache.readLocalFileAtCommit(COMMIT_1, path)).toThrow(/invalid file path/); + } + }); + + it("stream a file over MAX_BUFFERED_BLOB_SIZE, as readFilesAtCommit() does, and buffer one at it", + async () => { + let t = makeCache(); + let exact = new Uint8Array(MAX_BUFFERED_BLOB_SIZE).fill(0x62); + let over = new Uint8Array(MAX_BUFFERED_BLOB_SIZE + 1).fill(0x63); + over[0] = 0; // not text, but streams carry bytes unchecked + let tree = await storeLocal(t.storage, { + type: "tree", + payload: treePayload([ + { mode: "100644", name: "exact.txt", oid: await storeLocal(t.storage, { type: "blob", payload: exact }) }, + { mode: "100644", name: "over.bin", oid: await storeLocal(t.storage, { type: "blob", payload: over }) }, + ]), + }); + let commit = await storeLocal(t.storage, + { type: "commit", payload: commitPayload(tree, [], "sizes") }); + + expect(t.cache.readLocalFileAtCommit(commit, "exact.txt")) + .toStrictEqual({ kind: "text", text: "b".repeat(MAX_BUFFERED_BLOB_SIZE) }); + let file = t.cache.readLocalFileAtCommit(commit, "over.bin"); + if (file.kind !== "stream") throw new Error("expected a stream"); + expect(file.byteLength).toBe(over.length); + expect(sameBytes(await contentBytes(file.content), over)).toBe(true); + expect(t.pulls).toStrictEqual([]); + }); + + it("report content that is not text as unreadable", async () => { + let t = makeCache(); + let binary = await storeLocal(t.storage, + { type: "blob", payload: new Uint8Array([0x89, 0x50, 0x4e, 0x47, 0x00, 0x01]) }); + let tree = await storeLocal(t.storage, { + type: "tree", + payload: treePayload([{ mode: "100644", name: "image.png", oid: binary }]), + }); + let commit = await storeLocal(t.storage, + { type: "commit", payload: commitPayload(tree, [], "binary") }); + expect(t.cache.readLocalFileAtCommit(commit, "image.png")) + .toStrictEqual({ kind: "unreadable", message: "image.png is not a text file" }); + }); + + it("read a file's text and a path's entry as the pulling reads do, without pulling", + async () => { + let t = await fixtureStoredLocally(); + let readable = ["README.md", "src/util.js", "nope.txt", "no/such/file.txt", "README.md/child", + "src", "run.sh"]; + for (let path of readable) { + expect(t.cache.readLocalFileAtCommitIfExists(COMMIT_1, path)) + .toBe(await t.cache.readFileAtCommitIfExists(COMMIT_1, path)); + } + // The rest name nothing, run through a file, or name a directory. + expect(readable.filter( + path => t.cache.readLocalFileAtCommitIfExists(COMMIT_1, path) !== undefined)) + .toStrictEqual(["README.md", "src/util.js", "run.sh"]); + + let unreadable = ["link.md", "vendored"]; + for (let path of unreadable) { + expect(await thrownBy(() => t.cache.readLocalFileAtCommitIfExists(COMMIT_1, path))) + .toBe(await thrownBy(() => t.cache.readFileAtCommitIfExists(COMMIT_1, path))); + } + expect(() => t.cache.readLocalFileAtCommitIfExists(COMMIT_1, "link.md")) + .toThrow("link.md is a symlink to README.md"); + expect(() => t.cache.readLocalFileAtCommitIfExists(COMMIT_1, "vendored")) + .toThrow(`vendored is a submodule (gitlink) pointing at commit ${GITLINK_TARGET}`); + + let paths = [...readable, ...unreadable]; + for (let path of paths) { + let pulled = await t.cache.pathEntryAtCommit(COMMIT_1, path); + expect(t.cache.localPathEntryAtCommit(COMMIT_1, path)) + .toStrictEqual(pulled && { kind: pulled.kind, oid: pulled.oid }); + } + expect(paths.map(path => t.cache.localPathEntryAtCommit(COMMIT_1, path)?.kind)).toStrictEqual( + ["file", "file", undefined, undefined, undefined, "dir", "executable", "symlink", + "submodule"]); + expect(t.pulls).toStrictEqual([]); + }); + + it("throw naming the object a text, entry or diff read lacks, rather than pulling it", + async () => { + // Only COMMIT_1 is stored, everything beneath it and COMMIT_2 pullable from G1. + let t = makeCache(); + t.sources.set(G1, fixtureSource(t, G1)); + await t.cache.putFromGatekeeper(G1, "commit", fixture(COMMIT_1).payload); + expect(() => t.cache.readLocalFileAtCommitIfExists(COMMIT_1, "README.md")) + .toThrow(notStored(TREE_1)); + expect(() => t.cache.localPathEntryAtCommit(COMMIT_1, "README.md")) + .toThrow(notStored(TREE_1)); + // The diff names the commit it lacks, and with that stored, the first tree. + await expect(t.cache.changedLocalFilePathsBetween(COMMIT_1, COMMIT_2)) + .rejects.toThrow(notStored(COMMIT_2)); + await t.cache.putFromGatekeeper(G1, "commit", fixture(COMMIT_2).payload); + await expect(t.cache.changedLocalFilePathsBetween(COMMIT_1, COMMIT_2)) + .rejects.toThrow(notStored(TREE_1)); + + // The trees stored, every blob still pullable from G1. + for (let object of FIXTURE_OBJECTS.filter(o => o.type === "tree")) { + if (PACKED_OIDS.includes(object.oid)) { + await t.cache.putFromGatekeeper(G1, object.type, b64Bytes(object.payload)); + } + } + let readmeBlob = "ca69e6d08b5b8bb4f11a74f9695e329c203cbfd8"; + let linkBlob = "42061c01a1c70097d1e4579f29a5adf40abdec95"; // link.md -> "README.md" + expect(() => t.cache.readLocalFileAtCommitIfExists(COMMIT_1, "README.md")) + .toThrow(notStored(readmeBlob)); + expect(() => t.cache.readLocalFileAtCommitIfExists(COMMIT_1, "link.md")) + .toThrow(notStored(linkBlob)); + // An entry and a diff are read from trees alone, so neither needs a blob. + expect(t.cache.localPathEntryAtCommit(COMMIT_1, "README.md")) + .toStrictEqual({ kind: "file", oid: readmeBlob }); + expect(t.cache.localPathEntryAtCommit(COMMIT_1, "link.md")) + .toStrictEqual({ kind: "symlink", oid: linkBlob }); + expect(await t.cache.changedLocalFilePathsBetween(COMMIT_1, COMMIT_2)) + .toStrictEqual(new Set(["README.md", "src/big.txt", "src/util.js", "vendored"])); + expect(t.pulls).toStrictEqual([]); + }); + + it("throw for a file whose content is not text, while its entry is still reported", async () => { + let t = makeCache(); + let binary = await storeLocal(t.storage, + { type: "blob", payload: new Uint8Array([0x89, 0x50, 0x4e, 0x47, 0x00, 0x01]) }); + let commit = await commitWithFile(t, "image.png", binary); + let read = () => t.cache.readLocalFileAtCommitIfExists(commit, "image.png"); + expect(read).toThrow(UnreadableContentError); + expect(read).toThrow("image.png is not a text file"); + expect(t.cache.localPathEntryAtCommit(commit, "image.png")) + .toStrictEqual({ kind: "file", oid: binary }); + }); + + it("read a text file over MAX_BUFFERED_BLOB_SIZE whole, where readLocalFileAtCommit() streams", + async () => { + let t = makeCache(); + let text = "c".repeat(MAX_BUFFERED_BLOB_SIZE + 1); + let blob = t.cache.writeBlob(new TextEncoder().encode(text)); + expect(chunksOf(t, blob.oid).length).toBeGreaterThan(1); + let commit = await commitWithFile(t, "big.txt", blob.oid); + + // Compared outside expect(), which would print both megabytes on a mismatch. + expect(t.cache.readLocalFileAtCommitIfExists(commit, "big.txt") === text).toBe(true); + let streamed = t.cache.readLocalFileAtCommit(commit, "big.txt"); + if (streamed.kind !== "stream") throw new Error("expected a stream"); + await streamed.content.cancel(); + expect(t.pulls).toStrictEqual([]); + }); + + it("reject an invalid path from the text and entry reads, the empty path included", + async () => { + let t = await fixtureStoredLocally(); + for (let path of ["../escape", "", "src//util.js"]) { + expect(() => t.cache.readLocalFileAtCommitIfExists(COMMIT_1, path)) + .toThrow(/invalid file path/); + expect(() => t.cache.localPathEntryAtCommit(COMMIT_1, path)).toThrow(/invalid file path/); + } + // The one path on which localPathEntryAtCommit() and pathEntryAtCommit() differ: there the + // empty path names the root directory. + expect(await t.cache.pathEntryAtCommit(COMMIT_1, "")) + .toStrictEqual({ kind: "dir", oid: TREE_1, referencedBy: COMMIT_1 }); + }); + + it("list the paths that differ between two commits as changedFilePathsBetween() does", + async () => { + let t = await fixtureStoredLocally(); + for (let [a, b] of [[COMMIT_1, COMMIT_2], [COMMIT_2, COMMIT_3]]) { + let expected = await t.cache.changedFilePathsBetween(a, b); + expect(expected.size).toBeGreaterThan(0); + expect(await t.cache.changedLocalFilePathsBetween(a, b)).toStrictEqual(expected); + expect(await t.cache.changedLocalFilePathsBetween(b, a)) + .toStrictEqual(await t.cache.changedFilePathsBetween(b, a)); + } + expect(await t.cache.changedLocalFilePathsBetween(COMMIT_2, COMMIT_3)) + .toStrictEqual(new Set(["src/big.txt"])); + expect(await t.cache.changedLocalFilePathsBetween(COMMIT_1, COMMIT_1)).toStrictEqual(new Set()); + expect(t.pulls).toStrictEqual([]); + }); + + it("fail a diff on a tree the store lacks, which changedFilePathsBetween() pulls", async () => { + // Both commits are stored, every tree beneath them pullable from G1. + let t = makeCache(); + t.sources.set(G1, fixtureSource(t, G1)); + await t.cache.putFromGatekeeper(G1, "commit", fixture(COMMIT_1).payload); + await t.cache.putFromGatekeeper(G1, "commit", fixture(COMMIT_2).payload); + await expect(t.cache.changedLocalFilePathsBetween(COMMIT_1, COMMIT_2)) + .rejects.toThrow(notStored(TREE_1)); + expect(t.pulls).toStrictEqual([]); + + // The pulling diff faults in each tree it reads, from a source that serves only the objects + // asked for: the two roots, then the two `src` directories. Each pull names the object that + // references the tree, and has the shape of a fault against a worktree's base, which asks + // for the whole tree and every blob under EAGER_BLOB_LIMIT. + expect(await t.cache.changedFilePathsBetween(COMMIT_1, COMMIT_2)) + .toStrictEqual(new Set(["README.md", "src/big.txt", "src/util.js", "vendored"])); + let tree2 = (await t.cache.pathEntryAtCommit(COMMIT_2, ""))!.oid; + let src = (commit: GitOid) => t.cache.localPathEntryAtCommit(commit, "src")!.oid; + expect(t.pulls).toStrictEqual([ + [TREE_1, COMMIT_1], [tree2, COMMIT_2], [src(COMMIT_1), TREE_1], [src(COMMIT_2), tree2], + ].map(([oid, referencedBy]) => ({ + gatekeeperId: G1, + oids: [oid], + hints: { type: "tree", referencedBy, commitHistory: { kind: "depth", depth: 1 }, + filterBlobSize: EAGER_BLOB_LIMIT }, + }))); + }); + + it("read no subtree that both commits share, and no commit compared with itself", async () => { + let t = makeCache(); + // A commit whose tree holds a.js with `text`, and a `vendor` directory that is `vendor`. + let commitOf = async (text: string, vendor: GitOid) => { + let file = await storeLocal( + t.storage, { type: "blob", payload: new TextEncoder().encode(text) }); + let tree = await storeLocal(t.storage, { + type: "tree", + payload: treePayload([ + { mode: "100644", name: "a.js", oid: file }, + { mode: "40000", name: "vendor", oid: vendor }, + ]), + }); + return await storeLocal( + t.storage, { type: "commit", payload: commitPayload(tree, [], text) }); + }; + // Held nowhere: reading it throws. + let unheld = "ab".repeat(20); + let before = await commitOf("a\n", unheld); + let after = await commitOf("A\n", unheld); + expect(await t.cache.changedLocalFilePathsBetween(before, after)) + .toStrictEqual(new Set(["a.js"])); + expect(await t.cache.changedLocalFilePathsBetween(unheld, unheld)).toStrictEqual(new Set()); + + // A subtree that differs is read, so one that is not stored fails the diff. + let other = await commitOf("a\n", "cd".repeat(20)); + await expect(t.cache.changedLocalFilePathsBetween(before, other)) + .rejects.toThrow(notStored(unheld)); + expect(t.pulls).toStrictEqual([]); + }); +}); diff --git a/packages/workshop-backend/__tests__/user-directory-feed.test.ts b/packages/workshop-backend/__tests__/user-directory-feed.test.ts new file mode 100644 index 0000000000..44a41c2a94 --- /dev/null +++ b/packages/workshop-backend/__tests__/user-directory-feed.test.ts @@ -0,0 +1,85 @@ +import { runInDurableObject } from "cloudflare:test"; +import { env } from "cloudflare:workers"; +import { describe, expect, it } from "vitest"; +import { makeUserStorage } from "../src/storage-schema/user-storage.js"; +import { DIRECTORY_ACTIVITY_INTERVAL_MS, type UserDirectoryDurableObject } from "../src/user-directory.js"; +import type { UserDurableObject } from "../src/user.js"; +// Load the whole backend (the pool's `main`) up front, as user-directory.test.ts does. +import "./test-worker.js"; + +declare module "cloudflare:workers" { + interface ProvidedEnv { + TEST_USER: DurableObjectNamespace; + TEST_USER_DIRECTORY: DurableObjectNamespace; + } +} + +// The deployment-wide directory the user DO feeds (it always addresses getByName("")). +const deploymentDirectory = () => env.TEST_USER_DIRECTORY.getByName(""); + +async function listed(id: string) { + const { users } = await deploymentDirectory().listUsers({ query: id, limit: 10 }); + return users.find(listedUser => listedUser.id === id); +} + +function directoryLoginAt(user: DurableObjectStub): Promise { + return runInDurableObject(user, (_instance, state) => + makeUserStorage(state.storage).directoryLoginAt.get()); +} + +async function newAccount() { + const username = `feed${crypto.randomUUID().replaceAll("-", "")}`; + const user = env.TEST_USER.getByName(username); + const token = await user.createAccount(username, "Feed User", new Uint8Array([1, 2, 3])); + if (token === null) throw new Error(`Failed to create ${username}.`); + return { username, user, token }; +} + +describe("the user DO's directory feed", () => { + it("reports creation and session starts at most hourly, and renames without a login", async () => { + const { username, user, token } = await newAccount(); + await user.authenticate(token); + await expect.poll(() => listed(username)).toMatchObject({ + name: "Feed User", created: expect.any(Date), lastLogin: expect.any(Date), + }); + const first = (await listed(username))!; + const loginAt = await directoryLoginAt(user); + expect(loginAt).toBe(first.lastLogin!.getTime()); + + // Within the interval, a session start is not reported again. + await user.authenticate(token); + expect(await directoryLoginAt(user)).toBe(loginAt); + expect(await listed(username)).toEqual(first); + + // Past it, the next one is. + await runInDurableObject(user, (_instance, state) => { + makeUserStorage(state.storage).directoryLoginAt.put(loginAt - DIRECTORY_ACTIVITY_INTERVAL_MS); + }); + await user.authenticate(token); + await expect.poll(async () => (await listed(username))?.lastLogin?.getTime()) + .toBeGreaterThan(first.lastLogin!.getTime()); + const second = (await listed(username))!; + + // A rename syncs the name and leaves the last login alone. + await user.setOwnDisplayName("Renamed Feed User"); + await expect.poll(() => listed(username)).toEqual({ ...second, name: "Renamed Feed User" }); + }); + + it("reports the creation of an account created by Cloudflare Access", async () => { + const email = `feed-${crypto.randomUUID()}@example.com`; + const user = env.TEST_USER.getByName(email); + await expect(user.authenticateFromCfAccess(email, true)).resolves.toBe(true); + await expect.poll(() => listed(email)).toMatchObject({ + name: email.split("@")[0], created: expect.any(Date), lastLogin: expect.any(Date), + }); + }); + + it("returns a workspace's stored record, its creation time included", async () => { + const { user } = await newAccount(); + const before = Date.now(); + await user.newWorkspace("w1", "Workspace"); + const record = await user.getWorkspace("w1"); + expect(record).toMatchObject({ id: "w1", title: "Workspace" }); + expect(record!.created.getTime()).toBeGreaterThanOrEqual(before); + }); +}); diff --git a/packages/workshop-backend/__tests__/user-directory.test.ts b/packages/workshop-backend/__tests__/user-directory.test.ts index 12612fd5ca..e3bb82f96c 100644 --- a/packages/workshop-backend/__tests__/user-directory.test.ts +++ b/packages/workshop-backend/__tests__/user-directory.test.ts @@ -1,4 +1,6 @@ +import { runInDurableObject } from "cloudflare:test"; import { env } from "cloudflare:workers"; +import { MAX_DIRECTORY_TEXT_LENGTH } from "@gadgets/workshop-shared/api"; import { describe, expect, it } from "vitest"; import type { UserDirectoryDurableObject } from "../src/user-directory.js"; // Load the whole backend (the pool's `main`) up front. Otherwise the pool loads it on the first @@ -145,3 +147,289 @@ describe("UserDirectoryDurableObject", () => { ); }); }); + +// Reads the directory's own tables, to check what a row holds rather than what a read returns. +function stored>( + stub: DurableObjectStub, query: string): Promise { + return runInDurableObject(stub, (_instance, state) => state.storage.sql.exec(query).toArray()); +} + +const HOUR = 60 * 60_000; + +describe("UserDirectoryDurableObject activity", () => { + it("keeps the first creation time and only advances the last login", async () => { + const stub = directory("activity"); + const created = new Date(1_000 * HOUR); + await stub.syncUser(user("ada@example.com", "Ada"), 0, { created, lastLogin: new Date(1_001 * HOUR) }); + await stub.syncUser(user("ada@example.com", "Ada"), 0, { + created: new Date(2_000 * HOUR), lastLogin: new Date(1_000 * HOUR), + }); + await expect(stub.listUsers({ limit: 10 })).resolves.toEqual({ + users: [{ id: "ada@example.com", name: "Ada", created, lastLogin: new Date(1_001 * HOUR) }], + }); + + // A stale profile revision leaves the name alone, but its activity still merges. + await stub.syncUser(user("ada@example.com", "Augusta"), 1); + await stub.syncUser(user("ada@example.com", "Stale"), 0, { lastLogin: new Date(1_005 * HOUR) }); + await expect(stub.listUsers({ limit: 10 })).resolves.toEqual({ + users: [{ id: "ada@example.com", name: "Augusta", created, lastLogin: new Date(1_005 * HOUR) }], + }); + }); + + it("omits activity it has not been told", async () => { + const stub = directory("no-activity"); + await stub.syncUser(user("ada@example.com", "Ada"), 0); + await stub.syncUser(user("grace@example.com", "Grace"), 0, { lastLogin: new Date(HOUR) }); + const { users } = await stub.listUsers({ limit: 10 }); + expect(users).toEqual([ + { id: "ada@example.com", name: "Ada" }, + { id: "grace@example.com", name: "Grace", lastLogin: new Date(HOUR) }, + ]); + expect(Object.keys(users[0])).toEqual(["id", "name"]); + }); +}); + +describe("UserDirectoryDurableObject.listUsers", () => { + it("pages in id order", async () => { + const stub = directory("list-users"); + for (const id of ["e", "c", "a", "d", "b"]) await stub.syncUser(user(id, id.toUpperCase()), 0); + + const first = await stub.listUsers({ limit: 2 }); + expect(first).toEqual({ users: [user("a", "A"), user("b", "B")], nextCursor: "b" }); + const second = await stub.listUsers({ cursor: first.nextCursor, limit: 2 }); + expect(second).toEqual({ users: [user("c", "C"), user("d", "D")], nextCursor: "d" }); + const last = await stub.listUsers({ cursor: second.nextCursor, limit: 2 }); + expect(last).toEqual({ users: [user("e", "E")] }); + // A page that ends exactly at the last user has no cursor either. + await expect(stub.listUsers({ cursor: "c", limit: 2 })).resolves.toEqual({ + users: [user("d", "D"), user("e", "E")], + }); + }); + + it("matches a query against name or id and validates its options", async () => { + const stub = directory("list-users-query"); + await stub.syncUser(user("ada@example.com", "Ada Lovelace"), 0); + await stub.syncUser(user("grace@example.com", "Grace Hopper"), 0); + + await expect(stub.listUsers({ query: " ", limit: 10 })).resolves.toEqual({ + users: [user("ada@example.com", "Ada Lovelace"), user("grace@example.com", "Grace Hopper")], + }); + await expect(stub.listUsers({ query: "HOPPER", limit: 10 })).resolves.toEqual({ + users: [user("grace@example.com", "Grace Hopper")], + }); + await expect(stub.listUsers({ query: "ADA@", limit: 10 })).resolves.toEqual({ + users: [user("ada@example.com", "Ada Lovelace")], + }); + await expectRejection(stub.listUsers({ query: "a".repeat(1001), limit: 10 }), "at most 1000 characters"); + await expectRejection(stub.listUsers({ query: "ada\nlove", limit: 10 }), "no line breaks"); + await expectRejection(stub.listUsers({ limit: 0 }), "positive whole number"); + await expectRejection(stub.listUsers({ limit: 1.5 }), "positive whole number"); + // A cursor is a previous page's last id, which has no length limit. + await expect(stub.listUsers({ cursor: "a".repeat(300_000), limit: 10 })).resolves.toEqual({ + users: [user("ada@example.com", "Ada Lovelace"), user("grace@example.com", "Grace Hopper")], + }); + }); +}); + +function workspace(id: string, title: string, ownerId: string, + dates: { created?: Date; lastActive?: Date } = {}) { + return { id, title, ownerId, ...dates }; +} + +describe("UserDirectoryDurableObject workspaces", () => { + it("lets a higher revision replace the title and owner, and merges dates whatever the revision", async () => { + const stub = directory("sync-workspace"); + await stub.syncWorkspace(workspace("w1", "First", "ada"), 10); + await stub.syncWorkspace(workspace("w1", "Stale", "grace"), 9); + await stub.syncWorkspace(workspace("w1", "Replay", "grace"), 10); + await expect(stub.workspaceListing("w1")).resolves.toEqual({ title: "First", ownerId: "ada" }); + + // A stale sync fills a missing creation time, and nothing else. + await stub.syncWorkspace(workspace("w1", "Stale", "grace", { created: new Date(HOUR) }), 9); + await stub.syncWorkspace(workspace("w1", "Stale", "grace", { created: new Date(2 * HOUR) }), 9); + // The last activity advances and never regresses, whatever the revision. + await stub.syncWorkspace(workspace("w1", "Stale", "grace", { lastActive: new Date(5 * HOUR) }), 9); + await stub.syncWorkspace(workspace("w1", "First", "ada", { lastActive: new Date(4 * HOUR) }), 10); + await expect(stub.listWorkspaces({ limit: 10 })).resolves.toEqual({ + workspaces: [{ + id: "w1", title: "First", owner: user("ada", "ada"), + created: new Date(HOUR), lastActive: new Date(5 * HOUR), + }], + }); + + // A reinitialised workspace outranks its earlier life. + await stub.syncWorkspace(workspace("w1", "Reborn", "grace", { created: new Date(9 * HOUR) }), 11); + await expect(stub.listWorkspaces({ limit: 10 })).resolves.toEqual({ + workspaces: [{ + id: "w1", title: "Reborn", owner: user("grace", "grace"), + created: new Date(9 * HOUR), lastActive: new Date(5 * HOUR), + }], + }); + // A higher revision without a creation time keeps the stored one. + await stub.syncWorkspace(workspace("w1", "Renamed", "grace"), 12); + const { workspaces } = await stub.listWorkspaces({ limit: 10 }); + expect(workspaces[0]).toMatchObject({ title: "Renamed", created: new Date(9 * HOUR) }); + }); + + it("filters by owner and query, joins owner names, pages and forgets", async () => { + const stub = directory("list-workspaces"); + await stub.syncUser(user("ada@example.com", "Ada Lovelace"), 0); + await stub.syncWorkspace(workspace("w1", "Engine notes", "ada@example.com"), 1); + await stub.syncWorkspace(workspace("w2", "Compiler", "grace@example.com"), 1); + await stub.syncWorkspace(workspace("w3", "Analytical", "ada@example.com"), 1); + const w1 = { id: "w1", title: "Engine notes", owner: user("ada@example.com", "Ada Lovelace") }; + // An owner the directory has no user for is named by their id. + const w2 = { id: "w2", title: "Compiler", owner: user("grace@example.com", "grace@example.com") }; + const w3 = { id: "w3", title: "Analytical", owner: user("ada@example.com", "Ada Lovelace") }; + + await expect(stub.listWorkspaces({ owner: "ada@example.com", limit: 10 })).resolves.toEqual({ + workspaces: [w1, w3], + }); + await expect(stub.listWorkspaces({ owner: "ada", limit: 10 })).resolves.toEqual({ workspaces: [] }); + await expect(stub.listWorkspaces({ query: "ENGINE", limit: 10 })).resolves.toEqual({ workspaces: [w1] }); + await expect(stub.listWorkspaces({ query: "w2", limit: 10 })).resolves.toEqual({ workspaces: [w2] }); + await expect(stub.listWorkspaces({ query: "grace@", limit: 10 })).resolves.toEqual({ workspaces: [w2] }); + // A query matches the owner's id, not their display name. + await expect(stub.listWorkspaces({ query: "lovelace", limit: 10 })).resolves.toEqual({ workspaces: [] }); + + const first = await stub.listWorkspaces({ limit: 2 }); + expect(first).toEqual({ workspaces: [w1, w2], nextCursor: "w2" }); + await expect(stub.listWorkspaces({ cursor: first.nextCursor, limit: 2 })).resolves.toEqual({ + workspaces: [w3], + }); + + await stub.removeWorkspace("w2"); + await expect(stub.listWorkspaces({ limit: 10 })).resolves.toEqual({ workspaces: [w1, w3] }); + await expect(stub.workspaceListing("w2")).resolves.toBeNull(); + + await expectRejection(stub.listWorkspaces({ query: "a".repeat(1001), limit: 10 }), "at most 1000 characters"); + await expectRejection(stub.listWorkspaces({ query: "a\rb", limit: 10 }), "no line breaks"); + await expectRejection(stub.listWorkspaces({ limit: -1 }), "positive whole number"); + // Neither a cursor nor an owner filter is capped: both are ids. + await expect(stub.listWorkspaces({ owner: "a".repeat(300_000), limit: 10 })).resolves.toEqual({ + workspaces: [], + }); + await expect(stub.listWorkspaces({ cursor: "a".repeat(300_000), limit: 10 })).resolves.toEqual({ + workspaces: [w1, w3], + }); + }); + + it("looks up a workspace's title and owner", async () => { + const stub = directory("workspace-title"); + await stub.syncWorkspace(workspace("w1", "Engine notes", "ada"), 1); + await expect(stub.workspaceListing("w1")).resolves.toEqual({ title: "Engine notes", ownerId: "ada" }); + await expect(stub.workspaceListing("w9")).resolves.toBeNull(); + await expect(stub.workspaceListing("a".repeat(1000))).resolves.toBeNull(); + await expectRejection(stub.workspaceListing("a".repeat(1001)), "at most 1000 characters"); + }); +}); + +describe("UserDirectoryDurableObject text bounds", () => { + it("keeps and lists a long name as its first MAX_DIRECTORY_TEXT_LENGTH characters", async () => { + const stub = directory("clamp-name"); + const name = "Ada " + "x".repeat(3 * 2 ** 20); + await stub.syncUser(user("ada@example.com", name), 0); + const clamped = name.slice(0, MAX_DIRECTORY_TEXT_LENGTH); + + await expect(stub.listUsers({ limit: 10 })).resolves.toEqual({ + users: [user("ada@example.com", clamped)], + }); + await expect(stub.searchUsers("ada xx", [])).resolves.toEqual([user("ada@example.com", clamped)]); + const [row] = await stored<{ name: number; search_text: number }>(stub, + "SELECT length(name) AS name, length(search_text) AS search_text FROM users"); + expect(row.name).toBeLessThanOrEqual(2 * MAX_DIRECTORY_TEXT_LENGTH + 1); + expect(row.search_text).toBeLessThanOrEqual(2 * MAX_DIRECTORY_TEXT_LENGTH + 1); + }); + + it("keeps a name of exactly the limit whole, and never keeps half a surrogate pair", async () => { + const stub = directory("clamp-edges"); + const exact = "e".repeat(MAX_DIRECTORY_TEXT_LENGTH); + // "😀" is two UTF-16 code units, the first at the last kept position. + const straddling = "s".repeat(MAX_DIRECTORY_TEXT_LENGTH - 1) + "😀tail"; + await stub.syncUser(user("exact", exact), 0); + await stub.syncUser(user("straddling", straddling), 0); + + await expect(stub.listUsers({ limit: 10 })).resolves.toEqual({ + users: [user("exact", exact), user("straddling", "s".repeat(MAX_DIRECTORY_TEXT_LENGTH - 1))], + }); + }); + + it("keeps and lists a long title clamped, with its owner's clamped name", async () => { + const stub = directory("clamp-title"); + const name = "n".repeat(3 * 2 ** 20); + const title = "t".repeat(3 * 2 ** 20); + await stub.syncUser(user("ada", name), 0); + await stub.syncWorkspace({ id: "w1", title, ownerId: "ada" }, 1); + + await expect(stub.listWorkspaces({ limit: 10 })).resolves.toEqual({ + workspaces: [{ + id: "w1", + title: title.slice(0, MAX_DIRECTORY_TEXT_LENGTH), + owner: user("ada", name.slice(0, MAX_DIRECTORY_TEXT_LENGTH)), + }], + }); + await expect(stub.workspaceListing("w1")) + .resolves.toEqual({ title: title.slice(0, MAX_DIRECTORY_TEXT_LENGTH), ownerId: "ada" }); + }); + + it("clamps the names of rows stored before the directory clamped them", async () => { + const stub = directory("clamp-read"); + const name = "Old " + "o".repeat(4_996); + // Stands for a row stored before names were clamped. + await runInDurableObject(stub, (_instance, state) => { + state.storage.sql.exec("INSERT INTO users (id, name, search_text, rev) VALUES (?, ?, ?, 0)", + "old", name, `old\n${name}`.toLowerCase()); + }); + await stub.syncWorkspace({ id: "w1", title: "Old work", ownerId: "old" }, 1); + const clamped = user("old", name.slice(0, MAX_DIRECTORY_TEXT_LENGTH)); + + await expect(stub.listUsers({ limit: 10 })).resolves.toEqual({ users: [clamped] }); + await expect(stub.listWorkspaces({ limit: 10 })).resolves.toEqual({ + workspaces: [{ id: "w1", title: "Old work", owner: clamped }], + }); + await expect(stub.searchUsers("old o", [])).resolves.toEqual([clamped]); + }); + + it("stops a page early once its ids would pass a million characters", async () => { + const stub = directory("page-budget"); + const ids = ["a", "b", "c", "d", "e"].map(letter => letter.repeat(300_000)); + for (const id of ids) await stub.syncUser(user(id, "N"), 0); + + const first = await stub.listUsers({ limit: 50 }); + expect(first.users.map(listed => listed.id)).toEqual(ids.slice(0, 3)); + expect(first.nextCursor).toBe(ids[2]); + const second = await stub.listUsers({ cursor: first.nextCursor, limit: 50 }); + expect(second.users.map(listed => listed.id)).toEqual(ids.slice(3)); + expect(second.nextCursor).toBeUndefined(); + }); + + it("lists a user whose id alone passes the budget on a page of its own", async () => { + const stub = directory("page-budget-huge"); + const huge = "a".repeat(1_200_000); + await stub.syncUser(user(huge, "Huge"), 0); + await stub.syncUser(user("b", "B"), 0); + + const first = await stub.listUsers({ limit: 50 }); + expect(first.users.map(listed => listed.id)).toEqual([huge]); + expect(first.nextCursor).toBe(huge); + await expect(stub.listUsers({ cursor: first.nextCursor, limit: 50 })).resolves.toEqual({ + users: [user("b", "B")], + }); + }); + + it("bounds a page of workspaces by their owners' ids too", async () => { + const stub = directory("page-budget-workspaces"); + const owners = ["a", "b", "c", "d", "e"].map(letter => letter.repeat(300_000)); + for (const [index, owner] of owners.entries()) { + await stub.syncUser(user(owner, "N"), 0); + await stub.syncWorkspace({ id: `w${index}`, title: "T", ownerId: owner }, 1); + } + + const first = await stub.listWorkspaces({ limit: 50 }); + expect(first.workspaces.map(listed => listed.id)).toEqual(["w0", "w1", "w2"]); + expect(first.nextCursor).toBe("w2"); + const second = await stub.listWorkspaces({ cursor: first.nextCursor, limit: 50 }); + expect(second.workspaces.map(listed => listed.id)).toEqual(["w3", "w4"]); + expect(second.nextCursor).toBeUndefined(); + }); +}); diff --git a/packages/workshop-backend/__tests__/workspace-directory-feed.test.ts b/packages/workshop-backend/__tests__/workspace-directory-feed.test.ts new file mode 100644 index 0000000000..8df04f4579 --- /dev/null +++ b/packages/workshop-backend/__tests__/workspace-directory-feed.test.ts @@ -0,0 +1,189 @@ +// The Overseer's feed into the deployment directory (OverseerImpl.syncDirectory(), +// markDirectoryDirty() and removeFromDirectory()): every title write converges the directory on +// the newest title, a workspace's creation time is backfilled once from its owner's record, and a +// deleted workspace leaves the directory once any sync in flight has settled. +// +// Runs against a real OverseerDurableObject (the TEST_OVERSEER binding) feeding the real directory +// it reaches through ctx.exports, read back through the TEST_USER_DIRECTORY binding. + +import { describe, expect, it } from "vitest"; +import { env } from "cloudflare:workers"; +import { runInDurableObject } from "cloudflare:test"; +import { MAX_DIRECTORY_TEXT_LENGTH, type AiChatAuthorInfo } from "@gadgets/workshop-shared/api"; +import type { OverseerDurableObject } from "../src/overseer.js"; +import type { UserDirectoryDurableObject } from "../src/user-directory.js"; +// Load the whole backend (the pool's `main`) up front, so its slow load does not count against the +// first test's timeout. +import "./test-worker.js"; + +declare module "cloudflare:workers" { + interface ProvidedEnv { + TEST_OVERSEER: DurableObjectNamespace; + TEST_USER_DIRECTORY: DurableObjectNamespace; + } +} + +const OWNER_USER_ID = "owner-user-do"; +const CREATED = new Date(1700000000_000); + +type Feed = { + impl: any; + instance: OverseerDurableObject; + state: DurableObjectState; + id: string; + owner: AiChatAuthorInfo; + // How often the owner's record was read for the workspace's creation time. + gadgetReads: { count: number }; +}; + +// A fake owner user DO: their profile, their record of the workspace, and what open() and +// deleteSelf() call on it. +function fakeUsers(feed: Pick) { + let user = { + id: { toString: () => OWNER_USER_ID }, + whoami: async () => feed.owner, + getWorkspace: async () => { + feed.gadgetReads.count++; + return { id: feed.id, title: "Untitled Workspace", created: CREATED, lastActive: CREATED }; + }, + workspaceRemoved: async () => {}, + syncWorkspaceOutputs: async () => {}, + setWorkspaceLastActive: async () => {}, + }; + return { idFromString: (id: string) => id, get: () => user }; +} + +async function withFeed(fn: (feed: Feed) => Promise): Promise { + let stub = env.TEST_OVERSEER.getByName(`directory-feed-${crypto.randomUUID()}`); + await runInDurableObject(stub, async (instance: OverseerDurableObject, state) => { + let impl = (instance as unknown as { impl: any }).impl; + let unique = crypto.randomUUID(); + let feed: Feed = { + impl, instance, state, id: state.id.toString(), + owner: { type: "user", id: `owner-${unique}@example.test`, name: "Owner" }, + gadgetReads: { count: 0 }, + }; + impl.users = fakeUsers(feed); + impl.scheduleAccessRestart = async () => {}; + await fn(feed); + }); +} + +// Makes the workspace an initialised one owned by feed.owner, as far as the feed is concerned. +function initialise(feed: Feed): void { + feed.impl.ownerId = OWNER_USER_ID; + feed.impl.ownerProfileId = feed.owner.id; + feed.impl.storage.ownerId.put(OWNER_USER_ID); +} + +async function listed(feed: Feed, directory = env.TEST_USER_DIRECTORY.getByName("")) { + let { workspaces } = await directory.listWorkspaces({ owner: feed.owner.id, limit: 10 }); + return workspaces; +} + +describe("the workspace's directory feed", () => { + it("converges the directory on the newest title, clamped", () => withFeed(async feed => { + initialise(feed); + feed.impl.storage.title.put("First"); + feed.impl.storage.title.put("Second"); + await expect.poll(() => listed(feed)).toMatchObject([{ + id: feed.id, title: "Second", owner: { id: feed.owner.id }, created: CREATED, + }]); + + let long = "x".repeat(5000); + feed.impl.storage.title.put(long); + await expect.poll(async () => (await listed(feed))[0]?.title) + .toBe(long.slice(0, MAX_DIRECTORY_TEXT_LENGTH)); + })); + + it("records what the directory acknowledged and skips a sync that would change nothing", + () => withFeed(async feed => { + initialise(feed); + feed.impl.storage.title.put("Plans"); + await expect.poll(() => feed.impl.storage.directoryRev.get()) + .toBe(feed.impl.storage.titleRev.get()); + + let real = feed.impl.directories; + let sends = 0; + feed.impl.directories = { + getByName: (name: string) => { + sends++; + return real.getByName(name); + }, + }; + feed.impl.syncDirectory(); + feed.impl.markDirectoryDirty(); + await scheduler.wait(100); + expect(sends).toBe(0); + })); + + it("backfills the creation time once from the owner's record", () => withFeed(async feed => { + initialise(feed); + feed.impl.storage.title.put("Plans"); + await expect.poll(() => feed.impl.storage.createdAt.get()).toStrictEqual(CREATED); + expect(feed.gadgetReads.count).toBe(1); + + feed.impl.storage.title.put("Plans, revised"); + await expect.poll(async () => (await listed(feed))[0]?.title).toBe("Plans, revised"); + expect(feed.gadgetReads.count).toBe(1); + expect((await listed(feed))[0].created).toStrictEqual(CREATED); + })); + + it("reports activity when given it", () => withFeed(async feed => { + initialise(feed); + feed.impl.storage.title.put("Plans"); + await expect.poll(() => listed(feed)).toHaveLength(1); + let lastActive = new Date(Date.now() - 1000); + feed.impl.syncDirectory(lastActive); + await expect.poll(async () => (await listed(feed))[0]?.lastActive).toStrictEqual(lastActive); + })); + + it("takes the workspace out, and adds it back only once re-armed", () => withFeed(async feed => { + initialise(feed); + feed.impl.storage.title.put("Plans"); + await expect.poll(() => listed(feed)).toHaveLength(1); + + await feed.impl.removeFromDirectory(); + expect(await listed(feed)).toStrictEqual([]); + expect(feed.impl.storage.directoryRev.get()).toBe(-1); + + feed.impl.storage.title.put("Plans, after removal"); + feed.impl.markDirectoryDirty(); + feed.impl.syncDirectory(); + await scheduler.wait(200); + expect(await listed(feed)).toStrictEqual([]); + + feed.impl.directoryRemoved = false; + feed.impl.syncDirectory(); + await expect.poll(async () => (await listed(feed))[0]?.title).toBe("Plans, after removal"); + })); + + it("lets a deletion complete while a directory sync is in flight", () => withFeed(async feed => { + // A fresh directory instance is held busy, and the workspace's sync to it parks. + let heldName = crypto.randomUUID(); + let held = () => env.TEST_USER_DIRECTORY.getByName(heldName); + void runInDurableObject(held(), (_directory, state) => + state.blockConcurrencyWhile(() => scheduler.wait(2000))); + await scheduler.wait(100); + + let impl = feed.impl; + impl.directories = { getByName: () => held() }; + impl.ensureAmbientCapsules = async () => {}; + + // The session's close notification, as a plain function. + let notifyClosed: any = Object.assign(async () => {}, { + dup: () => notifyClosed, + [Symbol.dispose]: () => {}, + }); + + // The owner's first open initialises the workspace and starts a sync, parked on the hold. + let client = await feed.instance.open(OWNER_USER_ID, feed.owner.id, notifyClosed); + expect(impl.storage.directoryRev.get()).toBe(-1); + + let started = Date.now(); + await client.deleteSelf(); + expect(Date.now() - started).toBeLessThan(2000 + 5000); + expect([...feed.state.storage.kv.list()]).toStrictEqual([]); + expect(await listed(feed, held())).toStrictEqual([]); + }), 30_000); +}); diff --git a/packages/workshop-backend/__tests__/worktrees.test.ts b/packages/workshop-backend/__tests__/worktrees.test.ts index 168e33346d..bcd0cd3791 100644 --- a/packages/workshop-backend/__tests__/worktrees.test.ts +++ b/packages/workshop-backend/__tests__/worktrees.test.ts @@ -10,7 +10,7 @@ import { diffFiles, type CodeChange, type CodeContent, type FileChange } from "@gadgets/workshop-shared/code-change"; import { keyString } from "@gadgets/typed-storage"; import type { OverseerDurableObject } from "../src/overseer.js"; -import { buildCompactionState } from "../src/agent-compaction"; +import { buildCompactionState, chatChangeStatuses } from "../src/agent-compaction"; import { OVERSEER_STORAGE_VERSION } from "../src/storage-schema/overseer-migrations"; import { CodePreviewManager } from "../src/code-preview"; import { COMMIT_1, FIXTURE_OBJECTS, PACKED_OIDS, b64Bytes } from "./git-cache-fixtures"; @@ -480,6 +480,197 @@ describe("worktrees in the chat change stream", () => { })); }); +// The reader of stored objects alone, which a caller that must not reach a connection gives. +const localReader = (impl: any) => (commit: string, path: string) => + impl.gitCache.readLocalFileAtCommitIfExists(commit, path); + +// Chat 1's stored metadata, which getCurrentChatContent() takes beside the chat's id. +const metaOf = (impl: any) => impl.storage.chatMeta.get(1)!; + +// Chat 1 with a worktree at a commit of three files, a.txt edited by a step's message and +// src/b.txt by a live row: the two places a fold reads a base. +async function editedWorktree(impl: any): Promise<{ id: number, c1: string }> { + addChat(impl, 1); + let c1 = await commitFiles(impl, + { "a.txt": "one\n", "src/b.txt": "bee\n", "src/c.txt": "sea\n" }); + let { id } = await createThroughBarrier(impl, 1, c1); + await barrier(impl, 1, { changes: [{ change: editChange(id, "a.txt", "one\n", "one!\n") }]}); + await submit(impl, 1, "cli", editChange(id, "src/b.txt", "bee\n", "bee!\n")); + expect(chatMessages(impl, 1).filter(msg => msg.type === "changes" && msg.change !== undefined)) + .toHaveLength(1); + expect(impl.listLiveChatChanges(1, codeBaseOf(impl, 1).generation)).toHaveLength(1); + return { id, c1 }; +} + +// Records the paths the fold asks its default reader, readFileAtCommit(), for. +function recordDefaultReads(impl: any): string[] { + let asked: string[] = []; + let readFileAtCommit = impl.readFileAtCommit.bind(impl); + impl.readFileAtCommit = (commit: string, path: string) => { + asked.push(path); + return readFileAtCommit(commit, path); + }; + return asked; +} + +describe("the reader a fold takes for worktree edit bases", () => { + it("folds the same content from stored objects alone, never asking the reader that pulls", + () => withImpl(async impl => { + let { id } = await editedWorktree(impl); + let asked = recordDefaultReads(impl); + + impl.invalidateChatContent(1); + let byDefault = await impl.getCurrentChatContent(1, metaOf(impl)); + expect(Object.fromEntries(byDefault.get(id))) + .toEqual({ "a.txt": "one!\n", "src/b.txt": "bee!\n" }); + // The base of the message's edit, then of the live row's. + expect(asked).toEqual(["a.txt", "src/b.txt"]); + + impl.invalidateChatContent(1); + let local = await impl.getCurrentChatContent(1, metaOf(impl), localReader(impl)); + expect(local).not.toBe(byDefault); + expect(local).toEqual(byDefault); + expect(asked).toEqual(["a.txt", "src/b.txt"]); + + // A fold as the log stood, which reads no live row, takes the reader too. + let through = chatMessages(impl, 1).at(-1)!.sequence; + let built = await impl.buildChatContent(1, through, localReader(impl)); + expect(Object.fromEntries(built.get(id))).toEqual({ "a.txt": "one!\n" }); + expect(asked).toEqual(["a.txt", "src/b.txt"]); + })); + + it("fails rather than pulls when a base is not stored, and caches nothing", + () => withImpl(async impl => { + let { id, c1 } = await editedWorktree(impl); + // A connection that every missing object below is recorded as pullable from. + impl.storage.gatekeepers.put({ id: 99, resourceTitle: "Conn", class: {} as any }); + let reached = 0; + impl.getGatekeeperFacet = () => { + reached++; + throw new Error("the fold reached a connection"); + }; + + // The base of the message's edit, then of the live row's. + for (let path of ["a.txt", "src/b.txt"]) { + let blob = impl.gitCache.localPathEntryAtCommit(c1, path).oid; + let stored = impl.storage.gitObjects.get(blob)!; + impl.storage.gitObjectMetadata.put( + { oid: blob, type: "blob", onRemote: [99], pullableFrom: [], pendingPush: [] }); + impl.storage.gitObjects.delete(blob); + impl.invalidateChatContent(1); + + await expect(impl.getCurrentChatContent(1, metaOf(impl), localReader(impl))) + .rejects.toThrow(`git object ${blob} is not stored in this workspace`); + expect(reached).toBe(0); + + // The failed fold cached nothing: the default one reads the base itself, through the + // connection while the blob is missing and from the store once it is back. + await expect(impl.getCurrentChatContent(1, metaOf(impl))) + .rejects.toThrow("the fold reached a connection"); + expect(reached).toBe(1); + reached = 0; + impl.storage.gitObjects.put(stored); + expect(await workpieceContent(impl, 1, id)) + .toEqual({ "a.txt": "one!\n", "src/b.txt": "bee!\n" }); + expect(reached).toBe(0); + } + })); + + it("takes a reader's undefined as final, and reads through readFileAtCommit() without one", + () => withImpl(async impl => { + let { id, c1 } = await editedWorktree(impl); + let asked = recordDefaultReads(impl); + let content: CodeContent = new Map([[id, new Map()]]); + let change = editChange(id, "src/c.txt", "sea\n", "sea!\n"); + let worktreeBases = new Map([[id, c1]]); + + expect(await impl.seedWorktreeEditBases(content, change, worktreeBases, () => undefined)) + .toBe(content); + expect(asked).toEqual([]); + let seeded = await impl.seedWorktreeEditBases(content, change, worktreeBases); + expect(asked).toEqual(["src/c.txt"]); + expect(Object.fromEntries(seeded.get(id))).toEqual({ "src/c.txt": "sea\n" }); + + // The whole fold likewise, at the message's edit and at the live row's: with no base text + // the edit is of an absent file, and the default reader is not asked instead. + asked.length = 0; + impl.invalidateChatContent(1); + await expect(impl.getCurrentChatContent(1, metaOf(impl), () => undefined)) + .rejects.toThrow(`edit of absent file: gadget ${id} a.txt`); + let onlyMessageBase = (commit: string, path: string) => + path === "a.txt" ? localReader(impl)(commit, path) : undefined; + await expect(impl.getCurrentChatContent(1, metaOf(impl), onlyMessageBase)) + .rejects.toThrow(`edit of absent file: gadget ${id} src/b.txt`); + expect(asked).toEqual([]); + })); + + it("counts a live row's paths among the touched ones, a removed path included", + () => withImpl(async impl => { + let { id } = await editedWorktree(impl); + await submit(impl, 1, "cli-2", + { [id]: [["src/c.txt", { remove: true }], ["new.txt", { set: "fresh\n" }]] }); + let messages = chatMessages(impl, 1); + let statuses = chatChangeStatuses(messages); + let liveRows = impl.listLiveChatChanges(1, codeBaseOf(impl, 1).generation); + expect(liveRows).toHaveLength(2); + + // With two arguments, the messages alone. + expect(impl.worktreeTouchedPaths(messages, statuses)) + .toEqual(new Map([[id, new Set(["a.txt"])]])); + let touched = impl.worktreeTouchedPaths(messages, statuses, liveRows); + expect(touched) + .toEqual(new Map([[id, new Set(["a.txt", "src/b.txt", "src/c.txt", "new.txt"])]])); + // A worktree no message touched is listed for its live rows alone. + expect(impl.worktreeTouchedPaths([], new Map(), liveRows)) + .toEqual(new Map([[id, new Set(["src/b.txt", "src/c.txt", "new.txt"])]])); + + // An accept materializes every row before it asks, and so reads the same paths from the + // messages alone. + impl.materializeChatChanges(1); + expect(impl.listLiveChatChanges(1, codeBaseOf(impl, 1).generation)).toEqual([]); + let materialized = chatMessages(impl, 1); + expect(impl.worktreeTouchedPaths(materialized, chatChangeStatuses(materialized))) + .toEqual(touched); + })); + + it("counts no path of a reverted message, from before an accept, or of a gadget", + () => withImpl(async impl => { + let { id } = await editedWorktree(impl); + // What chat 1's worktrees have touched, as its log and its live rows stand. + let touched = () => { + let messages = chatMessages(impl, 1); + return impl.worktreeTouchedPaths(messages, chatChangeStatuses(messages), + impl.listLiveChatChanges(1, codeBaseOf(impl, 1).generation)); + }; + // The pin a client declares with a first modification: the worktree's accepted commit. + let pin = () => [{ gadgetId: id, baseCommit: impl.storage.gadgets.get(id)!.pinBase }]; + expect(touched()).toEqual(new Map([[id, new Set(["a.txt", "src/b.txt"])]])); + + // A revert of everything discards the live row, and the message it reverted counts for + // nothing. + await impl.revertChanges(1, 0, USER); + expect(touched()).toEqual(new Map()); + + // An accept closes the epoch: its boundary clears the paths of the messages before it, and + // a row of the epoch it opens is counted after it. + await submit(impl, 1, "cli-2", { [id]: [["new.txt", { set: "fresh\n" }]] }, pin()); + expect(touched()).toEqual(new Map([[id, new Set(["new.txt"])]])); + expect(await impl.mergeChanges(1, USER_META, "client-user")).toEqual({ outcome: "merged" }); + expect(touched()).toEqual(new Map()); + await submit(impl, 1, "cli-3", { [id]: [["later.txt", { set: "later\n" }]] }, pin()); + expect(touched()).toEqual(new Map([[id, new Set(["later.txt"])]])); + + // Only worktrees are listed: a step that changes a gadget beside one adds the worktree's + // path alone. + let gadget = impl.createGadget("Gadget", "GADGET_X", 1); + await barrier(impl, 1, { changes: [{ change: { + [gadget.id]: [["main.js", { set: "code\n" }]], + [id]: [["src/c.txt", { remove: true }]], + } }]}); + expect(touched()).toEqual(new Map([[id, new Set(["later.txt", "src/c.txt"])]])); + })); +}); + describe("worktree lifecycle", () => { it("a revert covering the creation keeps the worktree, rolling back its content and head", () => withImpl(async impl => { diff --git a/packages/workshop-backend/src/admin-access.ts b/packages/workshop-backend/src/admin-access.ts index 31fd94fb8c..4451b6ac13 100644 --- a/packages/workshop-backend/src/admin-access.ts +++ b/packages/workshop-backend/src/admin-access.ts @@ -1,32 +1,65 @@ -// The bound on how long the admin audit log keeps a record, and the text a record keeps. +// Deployment admin status, the bounds on admin access to workspaces, and the text an audit record +// keeps. // -// This is the one place the bound is decided, and the one place audit text is bounded. The bound -// comes from the environment, beside ADMINS, so that a compromised admin session cannot change it: -// admins choose a retention within it (in AdminConfig), and adminAccessPolicy() holds that choice -// inside it wherever it is enforced. This is the reason authentication config stays in -// auth/config.ts rather than in AdminConfig. +// This is the one place the admin access bounds are decided, and the one place audit text is +// bounded (admin status is isDeploymentAdmin() in auth/config.ts). The bounds come from the environment, beside ADMINS, so that a compromised +// admin session can change neither: admins choose an access level and a retention within them (in +// AdminConfig), and adminAccessPolicy() holds that choice inside them wherever it is enforced. This +// is the reason authentication config stays in auth/config.ts rather than in AdminConfig. +import { createHash } from "node:crypto"; import { - ADMIN_LIST_DEFAULT_LIMIT, ADMIN_LIST_MAX_LIMIT, DEFAULT_ADMIN_AUDIT_RETENTION_DAYS, - MAX_ADMIN_AUDIT_ARG_LENGTH, MAX_ADMIN_AUDIT_RETENTION_DAYS, MAX_ADMIN_AUDIT_STATS_DAYS, - MAX_ADMIN_AUDIT_VALUE_LENGTH, - type AdminSettingChange, type AdminSettingRecord, + ADMIN_ACCESS_ERROR_CODES, ADMIN_LIST_DEFAULT_LIMIT, ADMIN_LIST_MAX_LIMIT, + ADMIN_WORKSPACE_ACCESS_LEVELS, DEFAULT_ADMIN_AUDIT_RETENTION_DAYS, DEFAULT_ADMIN_COMMIT_LOG_DEPTH, + DEFAULT_ADMIN_VIEW_LIFETIME_SECONDS, MAX_ADMIN_ACCESS_REASON_LENGTH, MAX_ADMIN_AUDIT_ARG_LENGTH, + MAX_ADMIN_AUDIT_RETENTION_DAYS, MAX_ADMIN_AUDIT_STATS_DAYS, MAX_ADMIN_AUDIT_VALUE_LENGTH, + MAX_ADMIN_COMMIT_LOG_DEPTH, MAX_ADMIN_VIEW_LIFETIME_SECONDS, createAdminAccessError, + isAdminWorkspaceAccess, + type ActionLogEntry, type AdminAccessRecord, type AdminChatState, type AdminSettingChange, + type AdminSettingRecord, type AdminViewAccess, type AdminVia, type AdminWorkspaceAccess, + type AdminWorkspaceMethod, type AiChatHistoryPage, type AiChatMetadata, + type BlueprintGadgetSummary, type BoundHookInfo, type CommitInfo, type FileAtCommit, + type WorkpieceSummary, } from "@gadgets/workshop-shared/api"; import type { AdminConfig } from "./storage-schema/admin-settings-storage.js"; +import { isDeploymentAdmin } from "./auth/config.js"; import { createWorkshopLogger } from "./observability"; const logger = createWorkshopLogger("workshop.admin.access"); -// --- The environment's bound --- +// --- The environment's bounds --- // The raw values already reported as invalid, so that each is logged once per isolate rather than // on every request that reads it. +const reportedInvalidAccessMax = new Set(); const reportedInvalidRetentionMin = new Set(); function isBlank(raw: unknown): boolean { return raw === undefined || raw === null || String(raw).trim() === ""; } +/** The most access ADMIN_WORKSPACE_ACCESS_MAX lets admins choose: "off", "metadata" or "full", in + * any case and trimmed; "metadata" while it is unset or blank. Any other value misconfigures a + * security bound, so it fails closed to "off" and is logged as an error (once per isolate per + * value, never the value itself). */ +export function maxAdminWorkspaceAccess(env: Cloudflare.Env): AdminWorkspaceAccess { + let raw: unknown = env.ADMIN_WORKSPACE_ACCESS_MAX; + if (isBlank(raw)) return "metadata"; + let level = String(raw).trim().toLowerCase(); + if (isAdminWorkspaceAccess(level)) return level; + + // This bounds a security setting, so a value meant to turn inspection off ("none", "false", a + // typo of "off") must fail closed, never grant the default. + if (!reportedInvalidAccessMax.has(raw)) { + reportedInvalidAccessMax.add(raw); + logger.error( + "ADMIN_WORKSPACE_ACCESS_MAX is not off, metadata or full, so admin access to workspaces " + + "is off", + { event: "admin.access.config.invalid" }); + } + return "off"; +} + /** The fewest days ADMIN_AUDIT_RETENTION_MIN_DAYS lets admins keep audit records for: a positive * whole number (string or number), MAX_ADMIN_AUDIT_RETENTION_DAYS when larger; * DEFAULT_ADMIN_AUDIT_RETENTION_DAYS while it is unset or blank, and when it is anything else @@ -49,8 +82,16 @@ export function minAdminAuditRetentionDays(env: Cloudflare.Env): number { return DEFAULT_ADMIN_AUDIT_RETENTION_DAYS; } -/** What the deployment enforces now: the admins' choices held within the environment's bound. */ +/** The lower of two access levels (ADMIN_WORKSPACE_ACCESS_LEVELS lists them in order). */ +export function lowerAdminWorkspaceAccess(a: L, b: L): L { + let order = ADMIN_WORKSPACE_ACCESS_LEVELS; + return order.indexOf(a) <= order.indexOf(b) ? a : b; +} + +/** What the deployment enforces now: the admins' choices held within the environment's bounds. */ export type AdminAccessPolicy = { + /** The level views get: the chosen level, at most maxAdminWorkspaceAccess(). */ + access: AdminWorkspaceAccess; /** How many days the audit log keeps a record: the chosen retention, at least * minAdminAuditRetentionDays(). */ retentionDays: number; @@ -59,25 +100,69 @@ export type AdminAccessPolicy = { /** The policy `config` and the environment give. Every enforcement point computes it from a config * it has just read; none holds one from earlier. */ export function adminAccessPolicy( - config: Pick, env: Cloudflare.Env): AdminAccessPolicy { + config: Pick, + env: Cloudflare.Env): AdminAccessPolicy { return { + access: lowerAdminWorkspaceAccess(config.adminWorkspaceAccess, maxAdminWorkspaceAccess(env)), retentionDays: Math.max(config.adminAuditRetentionDays, minAdminAuditRetentionDays(env)), }; } -// --- Who acts --- +// --- Who acts, and on what terms --- /** Who changes a setting, and through which AdminApi method: what an AdminSettingRecord records of * the call besides its time and changes. AdminApiImpl passes one with every AdminSettings * mutation. */ export type AdminSettingActor = Pick; +/** What an AdminWorkspaceView is minted with (see OverseerDurableObject.openForAdmin()), fixed for + * the view's life. */ +export type AdminViewTerms = { + /** Recorded on every record the view makes (see AdminVia). */ + via: AdminVia; + /** The level the open asked for (AdminInspectOptions.access), recorded on every record the view + * makes (AdminAccessRecord.requested). */ + requested: AdminViewAccess; + /** The view's level when minted: the lower of `requested` and the deployment's policy then. Each + * call runs at this level or lower (see adminViewCallAccess()). */ + access: AdminViewAccess; + /** When the view stops working. */ + expiresAt: Date; +}; + +/** The view methods that read only content, refused below "full". */ +export const ADMIN_CONTENT_METHODS: ReadonlySet = + new Set( + ["getChat", "readChatAttachment", "readFile", "readFileAtCommit", "readPendingFile"]); + // --- Validation --- function isWholeNumberIn(value: number, min: number, max: number): boolean { return Number.isInteger(value) && value >= min && value <= max; } +/** `reason`, trimmed; throws unless it is non-blank and at most MAX_ADMIN_ACCESS_REASON_LENGTH. */ +export function validateAdminAccessReason(reason: string): string { + let trimmed = reason.trim(); + if (!trimmed || trimmed.length > MAX_ADMIN_ACCESS_REASON_LENGTH) { + throw new TypeError( + `A reason is required, and must be at most ${MAX_ADMIN_ACCESS_REASON_LENGTH} characters.`); + } + return trimmed; +} + +/** The lifetime `lifetimeSeconds` asks for, in ms; throws unless absent + * (DEFAULT_ADMIN_VIEW_LIFETIME_SECONDS) or a whole number from 1 to MAX_ADMIN_VIEW_LIFETIME_SECONDS. */ +export function adminViewLifetimeMs(lifetimeSeconds: number | undefined): number { + if (lifetimeSeconds === undefined) return DEFAULT_ADMIN_VIEW_LIFETIME_SECONDS * 1000; + if (!isWholeNumberIn(lifetimeSeconds, 1, MAX_ADMIN_VIEW_LIFETIME_SECONDS)) { + throw new TypeError( + `The view's lifetime must be a whole number of seconds from 1 to ` + + `${MAX_ADMIN_VIEW_LIFETIME_SECONDS}.`); + } + return lifetimeSeconds * 1000; +} + /** The page size `limit` asks for; throws unless absent (ADMIN_LIST_DEFAULT_LIMIT) or a whole * number from 1 to ADMIN_LIST_MAX_LIMIT. */ export function adminListLimit(limit: number | undefined): number { @@ -88,6 +173,17 @@ export function adminListLimit(limit: number | undefined): number { return limit; } +/** The depth `depth` asks AdminWorkspaceView.getCommitLog() for; throws unless absent + * (DEFAULT_ADMIN_COMMIT_LOG_DEPTH) or a whole number from 1 to MAX_ADMIN_COMMIT_LOG_DEPTH. */ +export function adminCommitLogDepth(depth: number | undefined): number { + if (depth === undefined) return DEFAULT_ADMIN_COMMIT_LOG_DEPTH; + if (!isWholeNumberIn(depth, 1, MAX_ADMIN_COMMIT_LOG_DEPTH)) { + throw new TypeError( + `The commit log's depth must be a whole number from 1 to ${MAX_ADMIN_COMMIT_LOG_DEPTH}.`); + } + return depth; +} + /** The statistics window `days` asks for; throws unless absent (7) or a whole number from 1 to * MAX_ADMIN_AUDIT_STATS_DAYS. */ export function adminAuditStatsDays(days: number | undefined): number { @@ -100,6 +196,34 @@ export function adminAuditStatsDays(days: number | undefined): number { return days; } +// --- View calls --- + +/** Throws the refusal that stops a view call before anything is recorded: `viewExpired` at or after + * `terms.expiresAt`, then `notAdmin` once `adminId` is no longer a deployment admin. */ +export function assertAdminViewLive(terms: AdminViewTerms, adminId: string, env: Cloudflare.Env, + now: number = Date.now()): void { + if (now >= terms.expiresAt.getTime()) { + throw createAdminAccessError(ADMIN_ACCESS_ERROR_CODES.viewExpired); + } + if (!isDeploymentAdmin(env, adminId)) { + throw createAdminAccessError(ADMIN_ACCESS_ERROR_CODES.notAdmin); + } +} + +/** The level one view call runs at (the view's own lowered to `policy.access`) and the refusal that + * level makes, if any: accessOff at "off", contentWithheld for one of ADMIN_CONTENT_METHODS at + * "metadata". Never throws: the caller records a refused call before refusing it. */ +export function adminViewCallAccess(terms: AdminViewTerms, method: AdminWorkspaceMethod, + policy: AdminAccessPolicy) + : { access: AdminWorkspaceAccess; refused?: AdminAccessRecord["refused"] } { + let access = lowerAdminWorkspaceAccess(terms.access, policy.access); + if (access === "off") return { access, refused: ADMIN_ACCESS_ERROR_CODES.accessOff }; + if (access === "metadata" && ADMIN_CONTENT_METHODS.has(method)) { + return { access, refused: ADMIN_ACCESS_ERROR_CODES.contentWithheld }; + } + return { access }; +} + // --- Audit text --- function isHighSurrogate(code: number): boolean { @@ -112,8 +236,8 @@ function isLowSurrogate(code: number): boolean { /** `text` as an audit record keeps it: whole while at most `maxLength` characters, else its first * `maxLength` - 1 characters and "…" (one fewer when the cut would split a surrogate pair). The one - * cut the audit log applies, marked so that a record never passes a fragment off as the whole: to - * setting keys. */ + * cut the audit stores apply, marked so that a record never passes a fragment off as the whole: to + * arguments, workspace titles and setting keys. */ export function cutAdminAuditText(text: string, maxLength: number): string { if (text.length <= maxLength) return text; let end = maxLength - 1; @@ -121,6 +245,35 @@ export function cutAdminAuditText(text: string, maxLength: number): string { return `${text.slice(0, end)}…`; } +// Hex digits of a long owner id's SHA-256 its audit form keeps: 80 bits, so that no two ids, however +// alike, share one. +const OWNER_DIGEST_LENGTH = 20; + +/** A workspace owner's id as an audit record keeps it (see AdminAccessRecord.workspaceOwner): whole + * while at most MAX_ADMIN_AUDIT_ARG_LENGTH characters, else its start, "…#" and a digest of the + * whole id, MAX_ADMIN_AUDIT_ARG_LENGTH characters in all. An id is an identity, so unlike + * cutAdminAuditText() this keeps two long ids sharing a start apart. Applied to its own result it + * returns it unchanged, so the owner filter takes either form. */ +export function auditOwnerId(id: string): string { + if (id.length <= MAX_ADMIN_AUDIT_ARG_LENGTH) return id; + let digest = createHash("sha256").update(id).digest("hex").slice(0, OWNER_DIGEST_LENGTH); + let start = cutAdminAuditText(id, MAX_ADMIN_AUDIT_ARG_LENGTH - OWNER_DIGEST_LENGTH - 1); + return `${start}#${digest}`; +} + +/** The arguments an AdminAccessRecord keeps (see AdminAccessRecord.args): undefined ones left out, + * each string cut by cutAdminAuditText() to MAX_ADMIN_AUDIT_ARG_LENGTH. */ +export function redactAdminAuditArgs(args: Record) + : Record { + let kept: Record = {}; + for (let [name, value] of Object.entries(args)) { + if (value === undefined) continue; + kept[name] = typeof value === "string" + ? cutAdminAuditText(value, MAX_ADMIN_AUDIT_ARG_LENGTH) : value; + } + return kept; +} + // Canonical JSON: object keys sorted at every level, written out as text. Building a sorted copy // by assignment instead would turn a "__proto__" key into a prototype change and drop it. function canonicalJson(value: unknown): string { @@ -210,3 +363,107 @@ export function adminSettingChange(setting: string, key: string | undefined, bef } return change; } + +// --- What a view below "full" withholds, and whether an answer holds any of it --- +// +// Each *HoldsContent() predicate takes what a view returns at any level and says whether it holds +// something the "metadata" level withholds (AdminAccessRecord.exposed). Present but empty is not +// content, so each is false for anything its withholder returns. + +/** A copy of `meta` as a view below "full" lists it: without `queuedSteers`. */ +export function withholdChatContent(meta: AiChatMetadata): AiChatMetadata { + let { queuedSteers: _queuedSteers, ...rest } = meta; + return rest; +} + +/** Whether `meta` holds what withholdChatContent() drops: a queued steer. */ +export function chatHoldsContent(meta: AiChatMetadata): boolean { + return (meta.queuedSteers?.length ?? 0) > 0; +} + +/** A copy of `state` as a view below "full" reports it: each queued call's `argsSummary` + * emptied. */ +export function withholdChatStateContent(state: AdminChatState): AdminChatState { + return { ...state, queuedCalls: state.queuedCalls.map(call => ({ ...call, argsSummary: "" })) }; +} + +/** Whether `state` holds what withholdChatStateContent() empties: a queued call's argument + * summary. */ +export function chatStateHoldsContent(state: AdminChatState): boolean { + return state.queuedCalls.some(call => call.argsSummary !== ""); +} + +/** A copy of `commit` as a view below "full" lists it: `message` emptied. */ +export function withholdCommitContent(commit: CommitInfo): CommitInfo { + return { ...commit, message: "" }; +} + +/** Whether `commit` holds what withholdCommitContent() empties: a message. */ +export function commitHoldsContent(commit: CommitInfo): boolean { + return commit.message !== ""; +} + +/** A copy of `entry` as a view below "full" lists it: `description.description` emptied for an + * action, observation or hook entry, and `description.fields` dropped; never mutates `entry`. */ +export function withholdActionContent(entry: ActionLogEntry): ActionLogEntry { + switch (entry.type) { + case "action": { + let { fields: _fields, ...description } = entry.description; + return { ...entry, description: { ...description, description: "" } }; + } + case "observation": { + let { fields: _fields, ...description } = entry.description; + return { ...entry, description: { ...description, description: "" } }; + } + case "bindHook": + return { ...entry, description: { ...entry.description, description: "" } }; + } +} + +/** Whether `entry` holds what withholdActionContent() withholds: a description, or a field. */ +export function actionHoldsContent(entry: ActionLogEntry): boolean { + return entry.description.description !== "" || + (entry.type !== "bindHook" && (entry.description.fields?.length ?? 0) > 0); +} + +/** A copy of `hook` as a view below "full" lists it: `description.description` emptied. */ +export function withholdHookContent(hook: BoundHookInfo): BoundHookInfo { + return { ...hook, description: { ...hook.description, description: "" } }; +} + +/** Whether `hook` holds what withholdHookContent() empties: a description. */ +export function hookHoldsContent(hook: BoundHookInfo): boolean { + return hook.description.description !== ""; +} + +/** Whether `summary` holds what workpieceSummary() drops below "full": the blueprint a gadget + * follows (`upstream: {}` marks a gadget built from scratch, which follows none). */ +export function workpieceHoldsContent(summary: WorkpieceSummary): boolean { + return summary.type === "gadget" && summary.upstream?.blueprintId !== undefined; +} + +/** A copy of `blueprint` as a view below "full" lists it: `id` emptied, and `screenshotUrl`, + * which holds the id, left out. A blueprint's id is a share link to its code, which is why + * describe() leaves a gadget's `upstream` out below "full" too. */ +export function withholdBlueprintContent(blueprint: BlueprintGadgetSummary) + : BlueprintGadgetSummary { + let { screenshotUrl: _screenshotUrl, ...rest } = blueprint; + return { ...rest, id: "" }; +} + +/** Whether `blueprint` holds what withholdBlueprintContent() empties: an id. */ +export function blueprintHoldsContent(blueprint: BlueprintGadgetSummary): boolean { + return blueprint.id !== ""; +} + +/** Whether `page` holds what getChat() refuses below "full": a message or a compaction summary. */ +export function chatPageHoldsContent(page: AiChatHistoryPage): boolean { + return page.messages.length > 0 || page.compacted !== undefined; +} + +/** Whether `file` holds what a file read (readFile(), readFileAtCommit(), readPendingFile()) + * refuses below "full": any file found at the path, an unreadable one too, since its message can + * quote the file (a symlink's quotes its target). */ +export function fileHoldsContent(file: FileAtCommit): boolean { + return file.kind !== "absent"; +} diff --git a/packages/workshop-backend/src/admin-audit.ts b/packages/workshop-backend/src/admin-audit.ts index 02ca8b3320..78b00fcec9 100644 --- a/packages/workshop-backend/src/admin-audit.ts +++ b/packages/workshop-backend/src/admin-audit.ts @@ -1,41 +1,53 @@ import { + MAX_ADMIN_AUDIT_ARG_LENGTH, type AdminAuditPage, type AdminAuditQuery, type AdminAuditRecord, type AdminAuditStats, type AdminAuditStatsQuery, } from "@gadgets/workshop-shared/api"; import { DurableObject } from "cloudflare:workers"; -import { adminAuditStatsDays, adminListLimit, minAdminAuditRetentionDays } from "./admin-access.js"; +import { + adminAuditStatsDays, adminListLimit, auditOwnerId, cutAdminAuditText, minAdminAuditRetentionDays, +} from "./admin-access.js"; import { utcDayKey } from "./ai-gateway-billing/limits/config.js"; const DAY_MS = 24 * 60 * 60_000; // Rows older than the retention that one call deletes at most, so that a log left unused for long // is trimmed over several calls rather than in one long write. const PRUNE_BATCH = 1000; -// The longest admin filter list() takes. +// The longest admin or workspace filter list() takes; workspace ids are 64 characters. const MAX_FILTER_LENGTH = 1000; const TOP_LIMIT = 10; -// A record's method, which stays in the JSON (see the schema below). +// A record's method, and an access record's workspace owner, which stay in the JSON (see the +// schema below). const METHOD = "json_extract(record, '$.method')"; +const OWNER = "json_extract(record, '$.workspaceOwner')"; +// Rows counted as content reads, by FILTER rather than SUM(exposed = 'content'), which setting rows +// (exposed NULL) would make NULL. +const CONTENT_READS = "COUNT(*) FILTER (WHERE exposed = 'content')"; type AuditRow = { seq: number; at: number; kind: string; admin: string; + workspace_id: string | null; + workspace_title: string | null; + exposed: string | null; record: string; }; /** * The deployment-wide admin audit log behind AdminApi.listAdminAudit() and getAdminAuditStats(): - * one row per AdminAuditRecord. An AdminSettingRecord is appended by AdminSettings before its - * change takes effect. Always addressed as getByName(""). + * one row per AdminAuditRecord. An AdminAccessRecord is appended by the inspected workspace's + * Overseer, once its call has read what it answers with and before that answer leaves, or by + * AdminApiImpl before it refuses an open that never reached the workspace. An AdminSettingRecord is + * appended by AdminSettings before its change takes effect. Always addressed as getByName(""). * Callers pass the retention they read for the call; whatever they pass, a record is kept at least - * minAdminAuditRetentionDays() days. Each method first deletes up to 1000 rows older than the - * retention, and never lists or counts one. + * minAdminAuditRetentionDays() days. Each method given one first deletes up to 1000 rows older + * than it, and never lists or counts one. * * Schema (the integration suite installs a trigger on the table by this name). Each field of a - * record is stored once: kind, at (epoch ms) and admin are columns; the rest of the record is JSON - * in `record`. workspace_id, workspace_title and exposed belong to the "workspace" kind, a record - * of admin access to a workspace, and are null for a "setting" record. + * record is stored once: kind, at (epoch ms), admin and, for an access record, workspace_id, + * workspace_title and exposed are columns; the rest of the record is JSON in `record`. * admin_audit(seq INTEGER PRIMARY KEY AUTOINCREMENT, at INTEGER NOT NULL, * kind TEXT NOT NULL CHECK (kind IN ('workspace', 'setting')), admin TEXT NOT NULL, * workspace_id TEXT, workspace_title TEXT, @@ -44,7 +56,7 @@ type AuditRow = { * CHECK ((kind = 'workspace') = (workspace_id IS NOT NULL AND workspace_title IS NOT NULL)), * CHECK ((kind = 'workspace') = (exposed IS NOT NULL))) STRICT; * indexes (at), (admin, seq), (workspace_id, seq), (kind, seq), (exposed, seq), - * (json_extract(record, '$.method'), seq) + * (json_extract(record, '$.method'), seq), (json_extract(record, '$.workspaceOwner'), seq) */ export class AdminAuditDurableObject extends DurableObject { constructor(ctx: DurableObjectState, env: Cloudflare.Env) { @@ -71,16 +83,47 @@ export class AdminAuditDurableObject extends DurableObject { // Every record names its method in the JSON. SQLite uses an index on an expression only for a // query that repeats the expression exactly, so list() filters on this same METHOD. sql.exec(`CREATE INDEX IF NOT EXISTS admin_audit_method ON admin_audit (${METHOD}, seq)`); + sql.exec(`CREATE INDEX IF NOT EXISTS admin_audit_owner ON admin_audit (${OWNER}, seq)`); } - /** Store one record. Throws when it cannot be stored, which fails the change it records. */ - append(record: AdminAuditRecord, retentionDays: number): void { - this.#prune(retentionDays); - // Each field is stored once, in its column or in the JSON, never both. - let { kind: _kind, at: _at, admin: _admin, ...rest } = record; + /** Store one record. Throws when it cannot be stored, which fails the call or change it records. + * Prunes first with `retentionDays`, which only a caller holding the authoritative config passes: + * an AdminWorkspaceView reads the KV mirror, whose retention may be one an admin has since raised, + * so its records are stored without pruning, and the next listing prunes. An access record's + * workspaceTitle is stored cut by cutAdminAuditText() to MAX_ADMIN_AUDIT_ARG_LENGTH, and its + * workspaceOwner as auditOwnerId() gives it, whatever the caller sent: the fields a workspace's + * members choose, neither bounded otherwise. An access record without `exposed` fails the table's + * CHECK, so its append throws. */ + append(record: AdminAuditRecord, retentionDays?: number): void { + if (retentionDays !== undefined) this.#prune(retentionDays); + // Each field is stored once, in its column or in the JSON, never both: a title stored twice let + // an over-long one overflow the row (the platform caps one at 8 MiB) and fail every append. + let workspaceId: string | null = null; + let workspaceTitle: string | null = null; + let exposed: string | null = null; + let rest: Record; + if (record.kind === "workspace") { + let { + kind: _kind, at: _at, admin: _admin, workspaceId: id, workspaceTitle: title, + workspaceOwner: owner, exposed: shown, ...fields + } = record; + workspaceId = id ?? null; + exposed = shown ?? null; + // Its callers cut the title already, but neither a title nor a username has a length limit of + // its own, and a page of records holding them whole could pass the RPC size limit, so the store + // holds the bound itself. + workspaceTitle = typeof title === "string" + ? cutAdminAuditText(title, MAX_ADMIN_AUDIT_ARG_LENGTH) : null; + rest = typeof owner === "string" ? { ...fields, workspaceOwner: auditOwnerId(owner) } : fields; + } else { + let { kind: _kind, at: _at, admin: _admin, ...fields } = record; + rest = fields; + } this.ctx.storage.sql.exec( - "INSERT INTO admin_audit (at, kind, admin, record) VALUES (?, ?, ?, ?)", - record.at.getTime(), record.kind, record.admin, JSON.stringify(rest)); + `INSERT INTO admin_audit (at, kind, admin, workspace_id, workspace_title, exposed, record) + VALUES (?, ?, ?, ?, ?, ?, ?)`, + record.at.getTime(), record.kind, record.admin, workspaceId, workspaceTitle, exposed, + JSON.stringify(rest)); } /** One page, newest first (see AdminApi.listAdminAudit()), each record rebuilt from its columns @@ -90,8 +133,10 @@ export class AdminAuditDurableObject extends DurableObject { : AdminAuditPage { let limit = adminListLimit(query.limit); let cutoff = this.#prune(retentionDays); - if (query.admin !== undefined && query.admin.length > MAX_FILTER_LENGTH) { - throw new TypeError(`Audit filters must be at most ${MAX_FILTER_LENGTH} characters.`); + for (let filter of [query.admin, query.workspace, query.owner]) { + if (filter !== undefined && filter.length > MAX_FILTER_LENGTH) { + throw new TypeError(`Audit filters must be at most ${MAX_FILTER_LENGTH} characters.`); + } } let where = ["at >= ?"]; @@ -108,6 +153,19 @@ export class AdminAuditDurableObject extends DurableObject { where.push(`${METHOD} = ?`); params.push(query.method); } + if (query.workspace !== undefined) { + where.push("workspace_id = ?"); + params.push(query.workspace); + } + if (query.exposed !== undefined) { + where.push("exposed = ?"); + params.push(query.exposed); + } + if (query.owner !== undefined) { + // As append() keeps it, so a long owner's whole id finds its records, and only its records. + where.push(`${OWNER} = ?`); + params.push(auditOwnerId(query.owner)); + } if (query.since !== undefined) { let since = query.since.getTime(); if (Number.isNaN(since)) throw new TypeError("Invalid since."); @@ -122,7 +180,7 @@ export class AdminAuditDurableObject extends DurableObject { } let rows = this.ctx.storage.sql.exec( - `SELECT seq, at, kind, admin, record FROM admin_audit + `SELECT seq, at, kind, admin, workspace_id, workspace_title, exposed, record FROM admin_audit WHERE ${where.join(" AND ")} ORDER BY seq DESC LIMIT ?`, ...params, limit + 1).toArray(); let page: AdminAuditPage = { records: rows.slice(0, limit).map(recordFromRow) }; @@ -130,7 +188,9 @@ export class AdminAuditDurableObject extends DurableObject { return page; } - /** The counts getAdminAuditStats() reports for the last `days` UTC days of this object's clock. */ + /** The counts getAdminAuditStats() reports for the last `days` UTC days of this object's clock. + * The content reads are the access records whose `exposed` is "content", counted in all and for + * each top admin. */ stats(query: Pick & { days: number }, retentionDays: number) : AdminAuditStats { let days = adminAuditStatsDays(query.days); @@ -150,11 +210,14 @@ export class AdminAuditDurableObject extends DurableObject { let counts = new Map(); let totalEvents = 0; - for (let { day, count } of sql.exec<{ day: number; count: number }>( - `SELECT at / ${DAY_MS} AS day, COUNT(*) AS count + let contentReads = 0; + type DayRow = { day: number; count: number; contentCount: number }; + for (let { day, count, contentCount } of sql.exec( + `SELECT at / ${DAY_MS} AS day, COUNT(*) AS count, ${CONTENT_READS} AS contentCount FROM admin_audit WHERE ${where} GROUP BY day`, ...params)) { counts.set(day, count); totalEvents += count; + contentReads += contentCount; } let firstDay = start / DAY_MS; let byDay = Array.from({ length: days }, (_, i) => ({ @@ -162,13 +225,15 @@ export class AdminAuditDurableObject extends DurableObject { count: counts.get(firstDay + i) ?? 0, })); - return { days, totalEvents, byDay, topAdmins: this.#topAdmins(where, params) }; + return { days, totalEvents, byDay, topAdmins: this.#topAdmins(where, params), contentReads }; } - // The admins with the most rows matching `where`, most first. - #topAdmins(where: string, params: (string | number)[]): { admin: string; count: number }[] { - return this.ctx.storage.sql.exec<{ admin: string; count: number }>( - `SELECT admin, COUNT(*) AS count FROM admin_audit WHERE ${where} + // The admins with the most rows matching `where`, most first, with their content reads. + #topAdmins(where: string, params: (string | number)[]) + : { admin: string; count: number; contentReads: number }[] { + return this.ctx.storage.sql.exec<{ admin: string; count: number; contentReads: number }>( + `SELECT admin, COUNT(*) AS count, ${CONTENT_READS} AS contentReads FROM admin_audit + WHERE ${where} GROUP BY admin ORDER BY count DESC, admin ASC LIMIT ${TOP_LIMIT}`, ...params).toArray(); } @@ -193,5 +258,9 @@ export class AdminAuditDurableObject extends DurableObject { function recordFromRow(row: AuditRow): AdminAuditRecord { let rest = JSON.parse(row.record) as Record; - return { ...rest, kind: row.kind, at: new Date(row.at), admin: row.admin } as AdminAuditRecord; + let columns = row.kind === "workspace" + ? { workspaceId: row.workspace_id, workspaceTitle: row.workspace_title, exposed: row.exposed } + : {}; + return { ...rest, kind: row.kind, at: new Date(row.at), admin: row.admin, ...columns } as + AdminAuditRecord; } diff --git a/packages/workshop-backend/src/admin-config.ts b/packages/workshop-backend/src/admin-config.ts index facaf83320..2a063511ee 100644 --- a/packages/workshop-backend/src/admin-config.ts +++ b/packages/workshop-backend/src/admin-config.ts @@ -4,12 +4,13 @@ // // This covers the "soft" deployment customizations only (branding, agent instructions, which // gatekeeper connectors/resources are offered, the models an AI Gateway deployment provides, and -// the audit retention admins choose within the bound admin-access.ts reads from the environment). +// the access level and audit retention admins choose within the bounds admin-access.ts reads from +// the environment). // Authentication/authorization config (sign-in providers, password login) is deliberately NOT // here — it stays env-var driven so it can't be changed by a compromised admin session. Everything // here is enabled by default; the admin UI opts things *out*. -import { AdminSettingChange, AiModelProvider, AmbientGatekeeperMode, BlueprintBinding, BlueprintMetadata, BlueprintOutput, DEFAULT_BANNER_COLOR, GatewayModel, GatewayModelCapabilities, GatewayModelMode, GatewayModelSettings, MAX_ADMIN_AUDIT_RETENTION_DAYS, MAX_UPDATE_CHECK_INTERVAL_MINUTES, MAX_UPDATE_HOURS, OutputFormatOffer, REASONING_LEVELS, SUGGESTED_MODELS, isAmbientGatekeeperMode, isBannerColor, isGatewayModelMode, isOutputIcon, isReasoningLevel } from "@gadgets/workshop-shared/api"; +import { AdminSettingChange, AiModelProvider, AmbientGatekeeperMode, BlueprintBinding, BlueprintMetadata, BlueprintOutput, DEFAULT_BANNER_COLOR, GatewayModel, GatewayModelCapabilities, GatewayModelMode, GatewayModelSettings, MAX_ADMIN_AUDIT_RETENTION_DAYS, MAX_UPDATE_CHECK_INTERVAL_MINUTES, MAX_UPDATE_HOURS, OutputFormatOffer, REASONING_LEVELS, SUGGESTED_MODELS, isAdminWorkspaceAccess, isAmbientGatekeeperMode, isBannerColor, isGatewayModelMode, isOutputIcon, isReasoningLevel } from "@gadgets/workshop-shared/api"; import { SupportedResource } from "@gadgets/workshop-shared/gatekeeper"; import { adminSettingChange } from "./admin-access.js"; import { sanitizeBlueprintOutput } from "./blueprint-archive.js"; @@ -452,8 +453,11 @@ export function normalizeAdminConfig(p: Partial): AdminConfig { updateNoticeSnoozeHours: isUpdateHours(p.updateNoticeSnoozeHours) ? p.updateNoticeSnoozeHours : DEFAULT_ADMIN_CONFIG.updateNoticeSnoozeHours, - // The admins' choice, kept as chosen: adminAccessPolicy() (admin-access.ts) holds it within the - // environment's bound wherever it is enforced, so changing the bound needs no rewrite. + // The admins' choices, kept as chosen: adminAccessPolicy() (admin-access.ts) holds them within + // the environment's bounds wherever they are enforced, so changing a bound needs no rewrite. + adminWorkspaceAccess: isAdminWorkspaceAccess(p.adminWorkspaceAccess) + ? p.adminWorkspaceAccess + : DEFAULT_ADMIN_CONFIG.adminWorkspaceAccess, adminAuditRetentionDays: isAdminAuditRetention(p.adminAuditRetentionDays) ? p.adminAuditRetentionDays : DEFAULT_ADMIN_CONFIG.adminAuditRetentionDays, diff --git a/packages/workshop-backend/src/admin-settings.ts b/packages/workshop-backend/src/admin-settings.ts index 46fd096be1..ef57b9ffc4 100644 --- a/packages/workshop-backend/src/admin-settings.ts +++ b/packages/workshop-backend/src/admin-settings.ts @@ -1,7 +1,7 @@ -import { AdminApi, AdminAuditPage, AdminAuditQuery, AdminAuditStats, AdminAuditStatsQuery, AdminFormat, AdminFormatPatch, AdminModel, AdminResourceVendor, AdminSettingChange, AdminSettingMethod, AdminSettingsView, AdminVia, AiModelConfig, AiModelProvider, AmbientGatekeeperMode, BannerColor, BlueprintPublicInfo, DeploymentUpdateStatus, GatewayModel, GatewayModelLevelTest, GatewayModelMode, GatewayModelSettings, GatewayModelTest, MAX_ADMIN_AUDIT_RETENTION_DAYS, MAX_ANNOUNCEMENT_LENGTH, MAX_INSTANCE_INSTRUCTIONS_LENGTH, MAX_SITE_NAME_LENGTH, MAX_UPDATE_CHECK_INTERVAL_MINUTES, MAX_UPDATE_HOURS, ReasoningLevel, SUGGESTED_MODELS, isAmbientGatekeeperMode, isBannerColor, isHexColor } from '@gadgets/workshop-shared/api'; +import { ADMIN_ACCESS_ERROR_CODES, AdminApi, AdminAuditPage, AdminAuditQuery, AdminAuditStats, AdminAuditStatsQuery, AdminFormat, AdminFormatPatch, AdminInspectOptions, AdminModel, AdminResourceVendor, AdminSettingChange, AdminSettingMethod, AdminSettingsView, AdminUserPage, AdminUserQuery, AdminVia, AdminWorkspaceAccess, AdminWorkspacePage, AdminWorkspaceQuery, AdminWorkspaceView, AiModelConfig, AiModelProvider, AmbientGatekeeperMode, BannerColor, BlueprintPublicInfo, DeploymentUpdateStatus, GatewayModel, GatewayModelLevelTest, GatewayModelMode, GatewayModelSettings, GatewayModelTest, MAX_ADMIN_AUDIT_ARG_LENGTH, MAX_ADMIN_AUDIT_RETENTION_DAYS, MAX_ANNOUNCEMENT_LENGTH, MAX_INSTANCE_INSTRUCTIONS_LENGTH, MAX_SITE_NAME_LENGTH, MAX_UPDATE_CHECK_INTERVAL_MINUTES, MAX_UPDATE_HOURS, ReasoningLevel, SUGGESTED_MODELS, createAdminAccessError, getAdminAccessErrorCode, isAdminWorkspaceAccess, isAmbientGatekeeperMode, isBannerColor, isHexColor } from '@gadgets/workshop-shared/api'; import { GatekeeperVendor } from '@gadgets/workshop-shared/gatekeeper'; import { DurableObject } from 'cloudflare:workers'; -import { RpcTarget } from 'capnweb'; +import { RpcTarget, type RpcStub } from 'capnweb'; import { validateRpc } from 'capnweb-validate'; import { createWorkshopLogger } from "./observability"; import { sanitizeBlueprintOutput } from './blueprint-archive.js'; @@ -16,8 +16,11 @@ import { SITE_LOGO_R2_KEY, siteLogoImage, validateSiteLogo } from './site-logo.j import { ambientGatekeeperMode, DEFAULT_AMBIENT_GATEKEEPER_MODE } from './provisioning-policy.js'; import { buildGatekeeperVendorMap } from './auth/auth-vendors.js'; import { UserDurableObject } from './user.js'; +import type { UserDirectoryDurableObject } from './user-directory.js'; import type { AdminAuditDurableObject } from './admin-audit.js'; -import { type AdminSettingActor, adminAccessPolicy, adminAuditStatsDays, adminListLimit, adminSettingChange, minAdminAuditRetentionDays } from './admin-access.js'; +import type { OverseerDurableObject } from './overseer.js'; +import { type AdminSettingActor, type AdminViewTerms, adminAccessPolicy, adminAuditStatsDays, adminListLimit, adminSettingChange, adminViewLifetimeMs, cutAdminAuditText, lowerAdminWorkspaceAccess, maxAdminWorkspaceAccess, minAdminAuditRetentionDays, validateAdminAccessReason } from './admin-access.js'; +import { isDeploymentAdmin } from './auth/config.js'; import { bundledBlueprintsManifestVersion, installBundledBlueprints } from './bundled-blueprints.js'; import { BUNDLED_BLUEPRINTS } from './generated/bundled-blueprints.js'; import { type DeployServiceInstall, NO_ANSWER, type VersionAnswer, deployServiceInstall, deploymentUpdateStatus, fetchLatestRelease, updateCheckDue } from './deployment-updates.js'; @@ -567,6 +570,11 @@ export class AdminSettings extends DurableObject { resourceVendors: await this.#listResourceConfig(config, adminUserId), formats: await this.#listFormatConfig(config), gatewayModels: this.#listGatewayModels(config), + adminWorkspaceAccess: { + effective: policy.access, + chosen: config.adminWorkspaceAccess, + max: maxAdminWorkspaceAccess(this.env), + }, adminAuditRetentionDays: { effective: policy.retentionDays, chosen: config.adminAuditRetentionDays, @@ -575,6 +583,21 @@ export class AdminSettings extends DurableObject { }; } + /** + * Set the access level admins choose for inspectWorkspace(), recording the change `actor` makes. + * Throws for a level above the one ADMIN_WORKSPACE_ACCESS_MAX allows, which a compromised admin + * session can never raise. + */ + async setAdminWorkspaceAccess(access: AdminWorkspaceAccess, actor: AdminSettingActor) + : Promise { + let max = maxAdminWorkspaceAccess(this.env); + if (lowerAdminWorkspaceAccess(access, max) !== access) { + throw new Error(`Admin access to workspaces can't be set above "${max}", the most this ` + + "deployment's ADMIN_WORKSPACE_ACCESS_MAX allows."); + } + await this.#mutateAdminConfig(actor, config => ({ ...config, adminWorkspaceAccess: access })); + } + /** * Set how many days admins choose to keep audit records for, recording the change `actor` makes. * Throws unless a whole number from the minimum ADMIN_AUDIT_RETENTION_MIN_DAYS sets to @@ -1231,32 +1254,40 @@ export class AdminSettings extends DurableObject { } } -// Capability for managing deployment-wide admin settings, obtained via -// AuthenticatedApi.getAdminApi() (which is null for non-admins). The admin access check happens once -// when the capability is minted in server.ts, so these methods don't re-check. This is a thin -// validation+forwarding facade over the AdminSettings DO — fully user-independent — so a disabled -// gatekeeper/resource can't be re-enabled via a crafted request, and the client never receives a -// stub to the DO's internal methods. Every setter's change is recorded in the deployment's audit -// log before it takes effect (see #mutateAdminConfig()). Covers branding, agent instructions, -// signups, gatekeeper connector/resource availability, AI Gateway models, whether an update is -// available, and the audit log itself; authentication config stays env-var driven. +// Capability for managing and investigating the deployment, obtained via +// AuthenticatedApi.getAdminApi() (which is null for non-admins). The admin access check happens +// once, when the capability is minted in server.ts, so its methods don't re-check, with one +// exception: inspectWorkspace() re-checks, as openForAdmin() and every call on the view it mints +// do, since a view is the one thing an admin holds that reads users' data. Setting by setting, +// this is a thin validation+forwarding facade over the AdminSettings DO — fully user-independent — +// so a disabled gatekeeper/resource can't be re-enabled via a crafted request, and the client +// never receives a stub to the DO's internal methods. Every setter's change is recorded in the +// deployment's audit log before it takes effect (see #mutateAdminConfig()). Covers branding, agent +// instructions, signups, gatekeeper connector/resource availability, AI Gateway models, whether an +// update is available, the user and workspace directory, and admin access to workspaces; +// authentication config stays env-var driven. @validateRpc() export class AdminApiImpl extends RpcTarget implements AdminApi { private admin: DurableObjectStub; + private directories: DurableObjectNamespace; private audits: DurableObjectNamespace; + private overseers: DurableObjectNamespace; #sessionId = crypto.randomUUID(); /** * `adminUserId` is the requesting admin's canonical id and `via` the label derived from their * client's User-Agent (AdminVia). The capability's audit `sessionId` is minted here, so there is - * one per AuthenticatedApi.getAdminApi() call. Reaches the AdminSettings and AdminAudit namespaces - * through `ctx.exports`, and reads the audit retention's bound from `env`. + * one per AuthenticatedApi.getAdminApi() call. Reaches the AdminSettings, UserDirectory, + * AdminAudit and Overseer namespaces through `ctx.exports`, and reads ADMINS and the admin access + * bounds from `env`. */ constructor(ctx: ExecutionContext, private env: Cloudflare.Env, private adminUserId: string, private via: AdminVia) { super(); this.admin = ctx.exports.AdminSettings.getByName(""); + this.directories = ctx.exports.UserDirectoryDurableObject; this.audits = ctx.exports.AdminAuditDurableObject; + this.overseers = ctx.exports.OverseerDurableObject; } // Who makes a change through `method` of this capability, as its audit record says. @@ -1452,13 +1483,108 @@ export class AdminApiImpl extends RpcTarget implements AdminApi { hours, this.#actor("setUpdateNoticeSnoozeHours")); } - // --- The audit log --- + // --- Users, workspaces and admin access --- + + async listUsers(options: AdminUserQuery = {}): Promise { + let limit = adminListLimit(options.limit); + // Built field by field, so that nothing else the client sent reaches the directory. + let { users, nextCursor } = await this.directories.getByName("").listUsers( + { query: options.query, cursor: options.cursor, limit }); + let page = users.map(user => ({ ...user, isAdmin: isDeploymentAdmin(this.env, user.id) })); + return nextCursor === undefined ? { users: page } : { users: page, nextCursor }; + } + + async listWorkspaces(options: AdminWorkspaceQuery = {}): Promise { + let limit = adminListLimit(options.limit); + return await this.directories.getByName("").listWorkspaces( + { owner: options.owner, query: options.query, cursor: options.cursor, limit }); + } + + async inspectWorkspace(workspaceId: string, reason: string, options: AdminInspectOptions = {}) + : Promise> { + reason = validateAdminAccessReason(reason); + let lifetimeMs = adminViewLifetimeMs(options.lifetimeSeconds); + let requested = options.access ?? "metadata"; + if (!isDeploymentAdmin(this.env, this.adminUserId)) { + throw createAdminAccessError(ADMIN_ACCESS_ERROR_CODES.notAdmin); + } + let id: DurableObjectId; + try { + id = this.overseers.idFromString(workspaceId); + } catch { + throw createAdminAccessError(ADMIN_ACCESS_ERROR_CODES.workspaceNotFound); + } + + // The committed authoritative config rather than readAdminConfig()'s KV mirror: a mirror still + // saying "full" after an admin lowered the level must not mint a view at the old one. The view's + // calls then read the mirror, which reaches them within about a minute. + let { access } = adminAccessPolicy(await this.admin.committedAdminConfig(), this.env); + let expiresAt = new Date(Date.now() + lifetimeMs); + if (access === "off") { + // Refused before any workspace is reached, so its title and owner come from the directory. + // Appended without a retention, as a view's records are: the directory read awaits, and a + // retention an admin raised meanwhile must not be pruned to the one read above. + let listing = await this.#directoryListing(id.toString()); + let workspaceTitle = cutAdminAuditText(listing.title, MAX_ADMIN_AUDIT_ARG_LENGTH); + await this.audits.getByName("").append({ + kind: "workspace", at: new Date(), admin: this.adminUserId, sessionId: this.#sessionId, + via: this.via, workspaceId: id.toString(), workspaceOwner: listing.ownerId, workspaceTitle, + method: "inspectWorkspace", args: { expiresAt: expiresAt.toISOString() }, reason, + requested, access: "off", exposed: "metadata", refused: ADMIN_ACCESS_ERROR_CODES.accessOff, + }); + throw createAdminAccessError(ADMIN_ACCESS_ERROR_CODES.accessOff); + } + + // The view runs at the lower of the level asked for and the deployment's. Asking for more than + // the deployment allows is not refused, and every record of the view keeps what was asked. + // @ts-expect-error Cap'n Web RPC stubs and native RPC stubs are compatible but the type + // system doesn't know this. + return this.#openForAdmin(id, reason, { + via: this.via, requested, access: lowerAdminWorkspaceAccess(requested, access), expiresAt, + }); + } + + // Mint the view. A workspace that turns out not to exist leaves the directory, which lists it + // only until something finds it gone. + async #openForAdmin(id: DurableObjectId, reason: string, terms: AdminViewTerms) { + try { + return await this.overseers.get(id).openForAdmin( + this.adminUserId, reason, this.#sessionId, terms); + } catch (error) { + if (getAdminAccessErrorCode(error) === ADMIN_ACCESS_ERROR_CODES.workspaceNotFound) { + try { + await this.directories.getByName("").removeWorkspace(id.toString()); + } catch (removal) { + logger.warn("failed to drop a missing workspace from the directory", { + event: "admin.directory.heal.failed", error: removal, + }); + } + } + throw error; + } + } + + // The title (clamped there to MAX_DIRECTORY_TEXT_LENGTH) and owner the directory lists for a + // workspace, for the record of an open refused before it reaches the workspace: "" for each when + // the directory lists none or cannot be asked, since the refusal is recorded either way. + async #directoryListing(workspaceId: string): Promise<{ title: string; ownerId: string }> { + try { + return await this.directories.getByName("").workspaceListing(workspaceId) ?? + { title: "", ownerId: "" }; + } catch (error) { + logger.warn("failed to read a workspace's listing from the directory", { + event: "admin.directory.listing.failed", error, + }); + return { title: "", ownerId: "" }; + } + } async listAdminAudit(options: AdminAuditQuery = {}): Promise { let limit = adminListLimit(options.limit); - let { kind, admin, method, since, cursor } = options; + let { kind, admin, method, workspace, owner, exposed, since, cursor } = options; return await this.audits.getByName("").list( - { kind, admin, method, since, cursor, limit }, await this.#retentionDays()); + { kind, admin, method, workspace, owner, exposed, since, cursor, limit }, + await this.#retentionDays()); } async getAdminAuditStats(options: AdminAuditStatsQuery = {}): Promise { @@ -1473,6 +1599,13 @@ export class AdminApiImpl extends RpcTarget implements AdminApi { return adminAccessPolicy(await this.admin.committedAdminConfig(), this.env).retentionDays; } + setAdminWorkspaceAccess(access: AdminWorkspaceAccess): Promise { + if (!isAdminWorkspaceAccess(access)) { + throw new Error(`Invalid admin workspace access: ${access}`); + } + return this.admin.setAdminWorkspaceAccess(access, this.#actor("setAdminWorkspaceAccess")); + } + setAdminAuditRetentionDays(days: number): Promise { return this.admin.setAdminAuditRetentionDays(days, this.#actor("setAdminAuditRetentionDays")); } diff --git a/packages/workshop-backend/src/auth/config.ts b/packages/workshop-backend/src/auth/config.ts index fe8bea361c..ebbbdb3dbd 100644 --- a/packages/workshop-backend/src/auth/config.ts +++ b/packages/workshop-backend/src/auth/config.ts @@ -26,11 +26,12 @@ export function hasAuthGatekeepers(env: Cloudflare.Env): boolean { * Whether the user `name` is one of the deployment admins the ADMINS binding lists. Names match * regardless of the case of ASCII letters, as Cloudflare Access matches its email rules: the * list is typed by hand, while a sign-in names its user by the email as the identity provider - * spells it, or by a password username, which is stored lowercased. + * spells it, or by a password username, which is stored lowercased. False for an empty or absent + * name. */ -export function isDeploymentAdmin(env: Cloudflare.Env, name: string): boolean { +export function isDeploymentAdmin(env: Cloudflare.Env, name: string | undefined): boolean { let admins: unknown = env.ADMINS; - if (!admins) return false; + if (!name || !admins) return false; if (typeof admins === "string") { // Admins should be a JSON binding of array type, but `.env` doesn't actually let you diff --git a/packages/workshop-backend/src/env.d.ts b/packages/workshop-backend/src/env.d.ts index 4e13b01ce0..7c63951311 100644 --- a/packages/workshop-backend/src/env.d.ts +++ b/packages/workshop-backend/src/env.d.ts @@ -20,10 +20,16 @@ declare global { // any other value is a deployment. A local instance lifts the limit on what `cf-os api` may do // (see the /api route in server.ts). WORKSHOP_ENVIRONMENT?: "development" | "test"; - // The fewest days the admin audit log keeps a record, whatever admins choose in the admin - // panel: a positive whole number, 90 when unset, blank or invalid (see admin-access.ts, and - // docs/admin-access.md, which also says where each kind of deployment sets it). An - // environment variable so that a compromised admin session cannot shorten it. + // The bounds on admin access to workspaces (see admin-access.ts, and docs/admin-access.md, + // which also says where each kind of deployment sets them). Admins choose an access level + // and an audit retention within them in the admin panel. They are environment variables so + // that a compromised admin session cannot widen them. + // + // The most an admin may let AdminApi.inspectWorkspace() show: "off", "metadata" or "full". + // "metadata" when unset or blank, and "off" for any value that is not a level. + ADMIN_WORKSPACE_ACCESS_MAX?: string; + // The fewest days the admin audit log keeps a record, whatever admins choose: a positive + // whole number. 90 when unset, blank or invalid. ADMIN_AUDIT_RETENTION_MIN_DAYS?: string; // Workers AI binding (injected by generate-wrangler-prod / run-dev-server; not in base wrangler.jsonc). diff --git a/packages/workshop-backend/src/git-cache.ts b/packages/workshop-backend/src/git-cache.ts index cfa520335c..d4f2bc1e4a 100644 --- a/packages/workshop-backend/src/git-cache.ts +++ b/packages/workshop-backend/src/git-cache.ts @@ -619,9 +619,13 @@ export class WorkspaceGitCache { /** * Walks the commit graph from `oid` (the commit itself first, then its ancestry), returning up * to `depth` commits' metadata, each commit once. Traversal order for merge commits follows - * git log's default (reverse chronological). + * git log's default (reverse chronological). With `stopAtMissing`, a parent this store does not + * hold ends that line of the walk instead of throwing: a commit pulled without its history (a + * worktree's checkout) still lists as far as the store goes. `oid` itself must be held either + * way. */ - readCommitLog(oid: string, options: { depth?: number } = {}): CommitInfo[] { + readCommitLog(oid: string, options: { depth?: number, stopAtMissing?: boolean } = {}) + : CommitInfo[] { // A commit that two parents of a merge both lead to is listed once, however its date // compares with the commits between: commits written in the same second tie. A gadget that // has merged releases of blueprints has such commits: the releases that those were built on. @@ -645,6 +649,7 @@ export class WorkspaceGitCache { for (let parent of next.commit.parents) { if (reached.has(parent)) continue; reached.add(parent); + if (options.stopAtMissing && !this.hasLocalObject(parent)) continue; tips.push({ oid: parent, commit: this.readCommitObject(parent) }); } } @@ -1259,6 +1264,129 @@ export class WorkspaceGitCache { return out; } + /** + * A commit's whole tree, as readCommitTree() gives it, read only from objects this store holds: + * never pulls, so it never reaches a connection. Throws when the commit or a tree is missing or + * of another type. + */ + readLocalCommitTree(commitOid: GitOid): TreeNode[] { + let walk = (treeOid: GitOid): TreeNode[] => + parseGitTree(this.#localObject(treeOid, "tree").payload, treeOid).map(entry => { + let kind = MODE_KINDS[entry.mode]; + return kind === "dir" + ? { name: entry.name, kind, children: walk(entry.oid) } + : { name: entry.name, kind }; + }); + return walk(parseGitCommitRefs(this.#localObject(commitOid, "commit").payload, commitOid).tree); + } + + /** + * One file of a commit, as readFilesAtCommit() reports it ("absent" for a path naming nothing or + * a directory, "unreadable" for a submodule, a symlink or content that is not text, "stream" for + * a regular file over MAX_BUFFERED_BLOB_SIZE, never read whole here, else "text"), read only from + * objects this store holds: never pulls. Throws for an invalid path and when an object along the + * path, or the file's blob, is missing. + */ + readLocalFileAtCommit(commitOid: GitOid, path: string): FileAtCommit { + let entry = this.#localEntryAt(commitOid, path); + if (entry === undefined || entry.mode === "40000") return { kind: "absent" }; + switch (entry.mode) { + case "160000": + return { kind: "unreadable", message: submoduleMessage(path, entry.oid) }; + case "120000": + return { kind: "unreadable", + message: symlinkMessage(path, this.#localObject(entry.oid, "blob").payload) }; + default: { + let reader = this.openLocalBlob(entry.oid); + if (reader === undefined) { + throw new Error(`git object ${entry.oid} is not stored in this workspace`); + } + if (reader.size > MAX_BUFFERED_BLOB_SIZE) { + return { kind: "stream", byteLength: reader.size, content: blobByteStream(reader) }; + } + let payload = reader.read(0, reader.size); + try { + return { kind: "text", text: decodeBlobText(payload, path) }; + } catch (err) { + if (!(err instanceof UnreadableContentError)) throw err; + return { kind: "unreadable", message: err.message }; + } + } + } + } + + /** + * One file of a commit as text, as readFileAtCommitIfExists() reads it (undefined for a path + * naming nothing or a directory; a symlink, a submodule or content that is not text throws its + * descriptive error; a file of any size is read whole), read only from objects this store holds: + * never pulls. The base resolver of a chat's worktree content when it is folded for a reader + * that must not reach a connection. + */ + readLocalFileAtCommitIfExists(commitOid: GitOid, path: string): string | undefined { + let entry = this.#localEntryAt(commitOid, path); + if (entry === undefined || entry.mode === "40000") return undefined; + if (entry.mode === "160000") throw new Error(submoduleMessage(path, entry.oid)); + let payload = this.#localObject(entry.oid, "blob").payload; + if (entry.mode === "120000") throw new Error(symlinkMessage(path, payload)); + return decodeBlobText(payload, path); + } + + /** + * The kind and oid of the entry at `path` in a commit's tree, undefined when the path names + * nothing, read only from objects this store holds: never pulls, and never reads a blob. What + * pathEntryAtCommit() gives for a path below the root; an empty path, which names the root + * there, is invalid here. Throws for an invalid path, and when the commit or a tree along the + * path is missing. + */ + localPathEntryAtCommit(commitOid: GitOid, path: string) + : { kind: GitTreeEntryKind, oid: GitOid } | undefined { + let entry = this.#localEntryAt(commitOid, path); + return entry && { kind: MODE_KINDS[entry.mode], oid: entry.oid }; + } + + /** + * The paths that differ between two commits' trees, as changedFilePathsBetween() gives them, + * read only from objects this store holds: never pulls. Throws when a commit or a differing tree + * is missing. + */ + async changedLocalFilePathsBetween(aCommit: GitOid, bCommit: GitOid): Promise> { + let out = new Set(); + if (aCommit === bCommit) return out; + let treeOf = (oid: GitOid) => + parseGitCommitRefs(this.#localObject(oid, "commit").payload, oid).tree; + await this.#diffTrees(oid => this.#localObject(oid, "tree"), + treeOf(aCommit), aCommit, treeOf(bCommit), bCommit, "", out); + return out; + } + + // The entry at `path` in a commit's tree, read only from objects this store holds; undefined + // when the path names nothing, or runs through something that is not a directory. Throws for an + // invalid path and when the commit or a tree along the path is missing. + #localEntryAt(commitOid: GitOid, path: string): GitTreeEntry | undefined { + let segments = splitTreePath(path); + let name = segments.pop()!; + let treeOid = + parseGitCommitRefs(this.#localObject(commitOid, "commit").payload, commitOid).tree; + let entryIn = (oid: GitOid, segment: string) => + parseGitTree(this.#localObject(oid, "tree").payload, oid).find(e => e.name === segment); + for (let segment of segments) { + let dir = entryIn(treeOid, segment); + if (dir === undefined || dir.mode !== "40000") return undefined; + treeOid = dir.oid; + } + return entryIn(treeOid, name); + } + + // A locally-stored object of the expected type, for the reads that must never pull. + #localObject(oid: GitOid, type: GitObjectType): PackableObject { + let local = this.readLocalObject(oid); + if (local === undefined) throw new Error(`git object ${oid} is not stored in this workspace`); + if (local.type !== type) { + throw new Error(`git object ${oid} is a ${local.type}, but a ${type} was expected`); + } + return local; + } + /** * The set of paths whose non-directory entry differs between two commits' trees (added, * removed, or changed in oid or mode), walking only differing subtrees and fault-pulling @@ -1273,19 +1401,24 @@ export class WorkspaceGitCache { let commit = await this.ensureObject(oid, { type: "commit", eagerTree: true }); return parseGitCommitRefs(commit.payload, oid).tree; }; - await this.#diffTreesLazy( + await this.#diffTrees( + (oid, referencedBy) => + this.ensureObject(oid, { type: "tree", referencedBy, eagerTree: true }), await treeOf(aCommit), aCommit, await treeOf(bCommit), bCommit, "", out); return out; } // Accumulates the differing non-directory paths of two trees (either may be absent) into - // `out`. `aRef`/`bRef` are the referencing objects for pull hints. - async #diffTreesLazy(aOid: GitOid | undefined, aRef: GitOid, bOid: GitOid | undefined, - bRef: GitOid, prefix: string, out: Set): Promise { + // `out`, reading each tree through `readTree` alone. `aRef`/`bRef` are the referencing objects, + // which a reader that pulls takes as its hints. + async #diffTrees( + readTree: (oid: GitOid, referencedBy: GitOid) => PackableObject | Promise, + aOid: GitOid | undefined, aRef: GitOid, bOid: GitOid | undefined, bRef: GitOid, + prefix: string, out: Set): Promise { if (aOid === bOid) return; let entriesOf = async (oid: GitOid | undefined, referencedBy: GitOid) => { if (oid === undefined) return new Map(); - let tree = await this.ensureObject(oid, { type: "tree", referencedBy, eagerTree: true }); + let tree = await readTree(oid, referencedBy); return new Map(parseGitTree(tree.payload, oid).map(entry => [entry.name, entry])); }; let aEntries = await entriesOf(aOid, aRef); @@ -1299,8 +1432,8 @@ export class WorkspaceGitCache { let bDir = b?.mode === "40000"; if (aDir || bDir) { // Descend the tree side(s); a non-tree entry opposite a tree is one more difference. - await this.#diffTreesLazy(aDir ? a!.oid : undefined, aOid ?? aRef, - bDir ? b!.oid : undefined, bOid ?? bRef, `${path}/`, out); + await this.#diffTrees(readTree, aDir ? a!.oid : undefined, aOid ?? aRef, + bDir ? b!.oid : undefined, bOid ?? bRef, `${path}/`, out); if ((a !== undefined && !aDir) || (b !== undefined && !bDir)) out.add(path); } else { out.add(path); diff --git a/packages/workshop-backend/src/overseer.ts b/packages/workshop-backend/src/overseer.ts index 8d883a8a39..565053de12 100644 --- a/packages/workshop-backend/src/overseer.ts +++ b/packages/workshop-backend/src/overseer.ts @@ -2,10 +2,19 @@ import { RpcCompatible, RpcStub, RpcTarget } from "capnweb"; import type { ChatTimelineHead } from "@gadgets/workshop-shared/api"; import { validateRpc } from "capnweb-validate"; import { Overseer, WorkspaceMetadata, UiBundle, WorkpieceId, WorkpieceSummary, WorkpiecesSubscriber, GadgetClient, GadgetBindingInfo, GatekeeperClient, ActionLogEntry, ActionsSubscriber, ActionHistoryFilter, ActionHistoryPage, ChatGadgetPinRecord, MainlineMergeGadget, ChatCodeBase, ChatGadgetPinState, CodeChangeSubmission, CommitInfo, FileAtCommit, MAX_READ_FILES_PER_CALL, TreeNode, MergeChangesResult, AiChatMetadata, AiChatMessage, AiChatHistoryPage, AiChatSubscriber, AiChatAuthorInfo, AiModelConfig, AiToolCall, AiChatMessageBody, AgentSpawnerConfig, AGENT_SPAWNER_CONNECTION_ID, ConsoleLogSubscriber, ConsoleLogEvent, CapsuleSpecifier, CollaboratorInfo, CollaboratorRole, CollaboratorAccess, CollaboratorGrant, AffectedCollaborator, ShareLinkInfo, GatekeeperCreationSpec, ObserverConfigCallback, ObserverBindingNeed, ObserverBindingFailure, BlueprintBindingAnnotation, BlueprintBinding, BlueprintMetadata, BlueprintOutput, BlueprintMerge, ApplyBlueprintResult, GadgetUpstream, MessageFormatRef, isOutputIcon, SpawnerEnvTarget, BlueprintGadgetSummary, AiChatStreamEvent, BlueprintScreenshotUpload, BLUEPRINT_SCREENSHOT_R2_PREFIX, blueprintScreenshotUrl, ChatAttachmentHandle, ChatAttachmentRef, BoundHookInfo, PreApprovableAction, PresenceParticipant, PresenceSubscriber, SlashCommandChoice, SlashCommandRequest, validateBindingName, createOpenWorkspaceError, OPEN_WORKSPACE_ERROR_CODES, createGatekeeperError, GATEKEEPER_ERROR_CODES, resolveSiteName, modelKind } from '@gadgets/workshop-shared/api' +import { + ADMIN_ACCESS_ERROR_CODES, MAX_ADMIN_AUDIT_ARG_LENGTH, + MAX_ADMIN_VIEW_LIFETIME_SECONDS, createAdminAccessError, + type AdminAccessExposure, type AdminAccessRecord, type AdminAutoApproval, + type AdminChatAttachment, type AdminChatState, type AdminGatekeeperInfo, type AdminObserver, + type AdminPendingChange, type AdminViewAccess, type AdminWorkspaceAccess, + type AdminWorkspaceDescription, type AdminWorkspaceDiagnostics, type AdminWorkspaceMethod, + type AdminWorkspaceView, +} from "@gadgets/workshop-shared/api"; import { applyCodeChange, changedGadgets, codeChangeSerializedSize, composeCodeChange, MAX_CODE_CHANGE_SIZE, transformCodeChange, validateCodeChangeContent, validateCodeChangeSchema, requireTextFiles, type FileContent, type CodeContent, type CodeChange } from "@gadgets/workshop-shared/code-change"; -import { type AgentCatalog, Gatekeeper, GatekeeperUserVerifier, HookInitiator, ResourceDescription, ApprovalQueue, ActionDescription, ObservationAuthorizer, ObservationDescription, VendorDescription, SupportedResource, resolveRequestedResource, HookController, HookDescription, ActionKind, GitCache, GitPullHints } from "@gadgets/workshop-shared/gatekeeper"; +import { type AgentCatalog, Gatekeeper, GatekeeperUserVerifier, HookInitiator, ResourceDescription, ApprovalQueue, ActionDescription, ObservationAuthorizer, ObservationDescription, VendorDescription, SupportedResource, resolveRequestedResource, HookController, HookDescription, ActionKind, GitCache, GitPullHints, MAX_BUFFERED_BLOB_SIZE } from "@gadgets/workshop-shared/gatekeeper"; import { DurableObject, WorkerEntrypoint, RpcStub as NativeRpcStub, RpcTarget as NativeRpcTarget, restore, @@ -27,7 +36,7 @@ import { import type { UserAiModelRecord, WorkspaceOutputEntry } from "./storage-schema/user-storage"; import { filesEqual, threeWayMerge, type MergeResult } from "./three-way-merge"; import { GitCacheImpl, WorkspaceGitCache, assertNoNul, commitIdentityForAuthor } from "./git-cache"; -import type { BlobReader } from "./git-chunked-blob"; +import { blobByteStream, type BlobReader } from "./git-chunked-blob"; import { OVERSEER_STORAGE_VERSION, migrateToActionIndexes, migrateToBlueprintUpstreams, migrateToGitStorage, migrateToMultiGadget, migrateToSpawnerBindings, migrateToWorkpieceTypes, @@ -53,6 +62,17 @@ import { scanWorkpieceForGrep, type GrepScan } from "./grep"; import WORKTREE_BINDING_TYPES from "./worktree-binding.txt"; import EXPORTS_BINDING_TYPES from "./exports-binding.txt"; import { deploymentOutputForBlueprint, FormatOffer, listFormatOffers, readAdminConfig } from "./admin-config"; +import { + actionHoldsContent, adminAccessPolicy, adminCommitLogDepth, adminViewCallAccess, + assertAdminViewLive, blueprintHoldsContent, chatHoldsContent, chatPageHoldsContent, + chatStateHoldsContent, commitHoldsContent, cutAdminAuditText, fileHoldsContent, + hookHoldsContent, lowerAdminWorkspaceAccess, redactAdminAuditArgs, validateAdminAccessReason, + withholdActionContent, withholdBlueprintContent, withholdChatContent, withholdChatStateContent, + withholdCommitContent, withholdHookContent, workpieceHoldsContent, + type AdminAccessPolicy, type AdminViewTerms, +} from "./admin-access"; +import type { AdminAuditDurableObject } from "./admin-audit"; +import { DIRECTORY_ACTIVITY_INTERVAL_MS, type UserDirectoryDurableObject } from "./user-directory"; import { chatChangeStatuses, foldCompactedCode, isTurnPrompt } from "./agent-compaction"; import type { CachedRequest } from "./ai-models"; import { ChatCacheWarmer } from "./chat-cache-warming"; @@ -64,7 +84,7 @@ import { buildReleasePack, buildSnapshotRelease, encodeReleaseCommit, releaseMergeHeader, releasesMergedBy, type Release, } from "./blueprint-release"; -import { EMPTY_TREE_OID, gitObjectOid, parseGitCommitRefs } from "./git-codec"; +import { EMPTY_TREE_OID, blobOid, gitObjectOid, parseGitCommitRefs } from "./git-codec"; import { listFeaturedBlueprintsFromKv, readBlueprintKvRecord, type BlueprintKvRecord, } from "./storage-schema/blueprints-kv"; @@ -570,6 +590,41 @@ async function computeSessionAffinity(gadgetId: string, chatId: number): Promise return new Uint8Array(hash).toHex(); } +// A workpiece as clients see it (see subscribeToWorkpieces()). `includeUpstream` says whether to +// name the blueprint a gadget follows: its id is a share link to code the reader may have no other +// way to read. +function workpieceSummary(record: WorkpieceRecord, includeUpstream: boolean): WorkpieceSummary { + if (record.type === "worktree") { + return { + id: record.id, + type: "worktree", + title: record.title, + chatId: record.chatId, + pinBase: record.pinBase, + headCommit: record.headCommit, + baseCommit: record.baseCommit, + }; + } + let summary: WorkpieceSummary = { + id: record.id, + type: "gadget", + title: record.title, + }; + if (record.commitId !== undefined) { + summary.commitId = record.commitId; + } + if (record.output) { + summary.output = record.output; + } + if (record.upstream && includeUpstream) { + summary.upstream = record.upstream; + } + if (record.pending) { + summary.chatId = record.pending.chatId; + } + return summary; +} + function actionRecordToLog(record: ActionRecord): ActionLogEntry { // TODO: ActionRecord and ActionLogEntry are almost identical. The main difference is that // ActionRecord includes `action`, which should NOT be provided to the client. We could make @@ -694,6 +749,34 @@ export const ACTION_REPLAY_PAGE_SIZE = 256; /** listActions() entries returned per page. Exported for tests. */ export const ACTION_HISTORY_PAGE_DEFAULT_LIMIT = 50; +// Throws for a listActions() `beforeId` that cannot name an action. +function validateBeforeId(beforeId: number | undefined): void { + if (beforeId !== undefined && (!Number.isSafeInteger(beforeId) || beforeId < 0)) { + throw new TypeError(`Invalid beforeId: ${beforeId}`); + } +} + +// One page of the action log, newest first (see Overseer.listActions()). +function actionHistoryPage(storage: OverseerStorage, + options?: {beforeId?: number, filter?: ActionHistoryFilter}): ActionHistoryPage { + let {beforeId, filter = "all"} = options ?? {}; + validateBeforeId(beforeId); + + // One ranged read -- off the collection itself for "all" (already id-ordered), off + // byHistoryFilter otherwise -- so the work is O(page) however sparse the matches. Pages are + // full until the last; the +1 record probes whether an older page exists. + let actions = storage.actions; + let range = {end: beforeId, reverse: true, limit: ACTION_HISTORY_PAGE_DEFAULT_LIMIT + 1}; + let page = [...(filter === "all" + ? actions.list(range) : actions.byHistoryFilter.get(filter, range))]; + let more = page.length > ACTION_HISTORY_PAGE_DEFAULT_LIMIT; + if (more) page.pop(); + return { + entries: page.map(actionRecordToLog), + nextBeforeId: more ? page.at(-1)!.id : undefined, + }; +} + /** * Keeps `commandPosition` only if it's a real index into `args`. Anything else becomes undefined, * and the command renders at the front. Display-only, so a bad value isn't worth an error. @@ -748,6 +831,12 @@ export function sanitizeMessageFormatRefs( // collaborator roles. type SessionKind = CollaboratorRole | "owner"; +// How the fold of a chat's content reads the base text of a worktree edit (see +// OverseerImpl.seedWorktreeEditBases): one file of a commit as text, undefined when the commit has +// no file there. +type BaseTextReader = + (commit: string, path: string) => string | undefined | Promise; + class OverseerImpl implements AgentHooks { public storage: OverseerStorage; readonly logger: ReturnType; @@ -771,6 +860,17 @@ class OverseerImpl implements AgentHooks { users: DurableObjectNamespace; + /** The deployment-wide admin audit log; a field, like `users`, so tests can replace it. */ + adminAudits: DurableObjectNamespace; + + /** The deployment directory this workspace mirrors itself into; a field, like `users`, so tests + * can replace it. */ + directories: DurableObjectNamespace; + + /** Set by removeFromDirectory() so that this instance syncs nothing more; cleared when the + * workspace is initialised. */ + directoryRemoved = false; + // The workspace's git store (see git-cache.ts): all gadgets' committed code and every other // history the workspace holds, with the gatekeeper-facing provenance metadata, the pull driver, // and push authorization over it. Per-gatekeeper GitCache stubs are minted from this via @@ -941,6 +1041,11 @@ class OverseerImpl implements AgentHooks { }; } + /** A snapshot of the live session counts (see joinSession()). */ + liveSessionCounts(): Record { + return {...this.#liveSessions}; + } + // Whether some non-owner session is live right now that holds "build", or one of `affected`'s // gadgets, or anything at all when `affected` is omitted. The owner is never an observer, so // their session never counts here. @@ -950,12 +1055,17 @@ class OverseerImpl implements AgentHooks { return Object.entries(this.#liveSessions).some(([kind, n]) => kind !== "owner" && n > 0); } + // Who has the workspace open now: the roster a presence subscriber is sent first. + presenceSnapshot(): PresenceParticipant[] { + return [...this.#presence.keys()].map(id => this.#toParticipant(id)); + } + // Subscribe to roster changes. The current roster is delivered immediately via init(). addPresenceSubscriber(subscriber: RpcStub): RpcStub<{}> { subscriber = subscriber.dup(); let token = {}; this.#presenceSubscribers.set(token, subscriber); - let snapshot = [...this.#presence.keys()].map(id => this.#toParticipant(id)); + let snapshot = this.presenceSnapshot(); subscriber.init(snapshot).catch(() => this.#removePresenceSubscriber(token)); subscriber.onRpcBroken(() => this.#removePresenceSubscriber(token)); // @ts-expect-error Bugs in native RPC types make this not work currently. @@ -1612,6 +1722,8 @@ class OverseerImpl implements AgentHooks { pull: (gatekeeperId, oids, hints) => this.#pullGitObjects(gatekeeperId, oids, hints), }); this.users = this.ctx.exports.UserDurableObject; + this.adminAudits = this.ctx.exports.AdminAuditDurableObject; + this.directories = this.ctx.exports.UserDirectoryDurableObject; this.ownerId = this.storage.deleting.get() ? undefined : this.storage.ownerId.get(); this.observerLedger = new ObserverLedger(this); @@ -1642,6 +1754,19 @@ class OverseerImpl implements AgentHooks { remove: () => this.markOutputsDirty(), }); + // Mirror the title into the deployment directory. The title singleton is the one chokepoint + // every writer passes (renames, agent-chosen titles, blueprint and external-message + // initialisation). typed-storage runs this inside the write's transaction, before the value + // lands, so the revision bump is atomic with the write and the sync itself is deferred. The + // revision is time-based so that a workspace reinitialised under the same id (an external + // message names its workspace by key) outranks whatever its earlier life reported. + this.storage.title.subscribe({ + update: () => { + this.storage.titleRev.put(Math.max(this.storage.titleRev.get() + 1, Date.now())); + this.markDirectoryDirty(); + }, + }); + this.ctx.blockConcurrencyWhile(async () => { if (this.storage.version.get() === 1) await migrateToGitStorage(this); migrateToActionIndexes(this); @@ -2305,37 +2430,7 @@ class OverseerImpl implements AgentHooks { let gadgets = this.storage.gadgets; subscriber = subscriber.dup(); // keep stub after return - let toSummary = (record: WorkpieceRecord): WorkpieceSummary => { - if (record.type === "worktree") { - return { - id: record.id, - type: "worktree", - title: record.title, - chatId: record.chatId, - pinBase: record.pinBase, - headCommit: record.headCommit, - baseCommit: record.baseCommit, - }; - } - let summary: WorkpieceSummary = { - id: record.id, - type: "gadget", - title: record.title, - }; - if (record.commitId !== undefined) { - summary.commitId = record.commitId; - } - if (record.output) { - summary.output = record.output; - } - if (record.upstream && includePending) { - summary.upstream = record.upstream; - } - if (record.pending) { - summary.chatId = record.pending.chatId; - } - return summary; - }; + let toSummary = (record: WorkpieceRecord) => workpieceSummary(record, includePending); let disposed = false; let unsubscribe = () => { @@ -2496,7 +2591,10 @@ class OverseerImpl implements AgentHooks { // Live (unmaterialized) change rows are deliberately NOT included: callers that need them either // materialize first (accept, update-from-mainline, UI bundle loads) or apply them on top // themselves (getCurrentChatContent). - async buildChatContent(chatId: number, through?: number): Promise { + // + // `readBase` is the reader seedWorktreeEditBases takes for the base texts of worktree edits. + async buildChatContent(chatId: number, through?: number, readBase?: BaseTextReader) + : Promise { let messages = [...this.storage.chats.list({prefix: chatKeyPrefix(chatId)})]; if (through !== undefined) { messages = messages.filter(msg => msg.sequence <= through); @@ -2538,7 +2636,7 @@ class OverseerImpl implements AgentHooks { } } if (msg.change !== undefined) { - content = await this.seedWorktreeEditBases(content, msg.change, worktreeBases); + content = await this.seedWorktreeEditBases(content, msg.change, worktreeBases, readBase); content = applyCodeChange(content, msg.change); } } @@ -2556,8 +2654,14 @@ class OverseerImpl implements AgentHooks { // out for hand-rolled clients only: a row that edits a path an earlier row removed re-seeds // the base text rather than failing -- every fold applies the same rule in the same order, so // all replicas agree; the shipped producers never emit that sequence.) - async seedWorktreeEditBases(content: CodeContent, change: CodeChange, - worktreeBases: Map): Promise { + // + // `readBase` does the read. The default, readFileAtCommit(), pulls what the store lacks; a + // caller that must not reach a connection gives a reader of stored objects alone. A reader's + // `undefined` is final: no other reader is asked. + async seedWorktreeEditBases( + content: CodeContent, change: CodeChange, worktreeBases: Map, + readBase: BaseTextReader = (commit, path) => this.readFileAtCommit(commit, path)) + : Promise { let result = content; for (let [key, entries] of Object.entries(change)) { let worktreeId = Number(key); @@ -2565,7 +2669,7 @@ class OverseerImpl implements AgentHooks { if (base === undefined) continue; for (let [path, fileChange] of entries) { if (!("edit" in fileChange) || result.get(worktreeId)?.has(path)) continue; - let text = await this.gitCache.readFileAtCommitIfExists(base, path); + let text = await readBase(base, path); if (text === undefined) continue; if (result === content) result = new Map(content); let files = new Map(result.get(worktreeId)); @@ -2650,7 +2754,12 @@ class OverseerImpl implements AgentHooks { // The chat's current content: what the next change row will apply to. Cached; treat the result as // immutable (it is shared with the cache and with code-change's structure sharing). - async getCurrentChatContent(chatId: number, meta: AiChatMetadata): Promise { + // + // `readBase` reads the base texts of worktree edits (see seedWorktreeEditBases). The cache holds + // the same content whichever reader filled it: a commit is immutable, so every reader that + // reads a base returns the same text, and one that cannot throws before anything is cached. + async getCurrentChatContent(chatId: number, meta: AiChatMetadata, readBase?: BaseTextReader) + : Promise { for (let attempt = 0; ; attempt++) { let codeBase = this.chatCodeBase(meta); let cached = this.#chatContentCache.get(chatId); @@ -2663,7 +2772,7 @@ class OverseerImpl implements AgentHooks { // materialization retiring rows out from under the row scan), and the re-reads pick up // rows appended meanwhile. let token = this.nextChatSequencePeek(chatId); - let content = await this.buildChatContent(chatId); + let content = await this.buildChatContent(chatId, undefined, readBase); // Pins established by live rows are not yet declared in any message; their bases enter // the content before the rows apply. (Rows before the establishing row never touch the // gadget, so establishing all of them up front is equivalent to establishing in order.) @@ -2684,7 +2793,8 @@ class OverseerImpl implements AgentHooks { if (worktreeBases.size > 0) { let probe = content; for (let row of this.listLiveChatChanges(chatId, codeBase.generation)) { - probe = await this.#prefetchWorktreeSeeds(probe, row.change, worktreeBases, seeds); + probe = await this.#prefetchWorktreeSeeds( + probe, row.change, worktreeBases, seeds, readBase); probe = applyCodeChange(probe, row.change); } } @@ -2727,9 +2837,11 @@ class OverseerImpl implements AgentHooks { // seedWorktreeEditBases, but additionally records every looked-up base into `seeds` // (`${worktreeId}:${path}` -> text, or null for a path absent from the base), so the // synchronous tail can re-seed without awaiting. - async #prefetchWorktreeSeeds(content: CodeContent, change: CodeChange, - worktreeBases: Map, - seeds: Map): Promise { + async #prefetchWorktreeSeeds( + content: CodeContent, change: CodeChange, worktreeBases: Map, + seeds: Map, + readBase: BaseTextReader = (commit, path) => this.readFileAtCommit(commit, path)) + : Promise { let result = content; for (let [key, entries] of Object.entries(change)) { let worktreeId = Number(key); @@ -2740,7 +2852,7 @@ class OverseerImpl implements AgentHooks { let key = `${worktreeId}:${path}`; let text = seeds.get(key); if (text === undefined) { - text = await this.gitCache.readFileAtCommitIfExists(base, path) ?? null; + text = await readBase(base, path) ?? null; seeds.set(key, text); } if (text === null) continue; @@ -4414,7 +4526,7 @@ class OverseerImpl implements AgentHooks { // content-addressed object writes -- harmless if the accept turns out stale below. let worktreeAccepts = new Map(); if (entryCodeBase.pins.some(pin => this.isWorktree(pin.gadgetId))) { - let touchedByWorktree = this.#worktreeTouchedPaths(messages, statuses); + let touchedByWorktree = this.worktreeTouchedPaths(messages, statuses); for (let pin of entryCodeBase.pins) { let record = this.storage.gadgets.get(pin.gadgetId); if (record?.type !== "worktree") continue; @@ -4656,10 +4768,21 @@ class OverseerImpl implements AgentHooks { // epoch boundaries, reverted messages excluded). The accept's re-pin plan needs this alongside // the flattened content: content answers what a path holds now, but only the change stream // knows which paths were touched at all -- a removed path is simply absent from content, and - // must enter the auto-commit's change map as a deletion. - #worktreeTouchedPaths(messages: AiChatMessage[], statuses: Map) + // must enter the auto-commit's change map as a deletion. `liveRows` are the rows not yet + // materialized into a message, whose paths are touched too. + worktreeTouchedPaths(messages: AiChatMessage[], statuses: Map, + liveRows: readonly ChatChangeRecord[] = []) : Map> { let touched = new Map>(); + let touch = (change: CodeChange) => { + for (let [key, entries] of Object.entries(change)) { + let id = Number(key); + if (!this.isWorktree(id)) continue; + let paths = touched.get(id); + if (paths === undefined) touched.set(id, paths = new Set()); + for (let [path] of entries) paths.add(path); + } + }; for (let msg of messages) { if (msg.type === "merge" && msg.epochBoundary) { touched.clear(); @@ -4668,15 +4791,9 @@ class OverseerImpl implements AgentHooks { if (msg.type !== "changes") continue; if (statuses.get(msg.sequence) === "reverted") continue; if (msg.conversionBoundary) touched.clear(); - if (msg.change === undefined) continue; - for (let [key, entries] of Object.entries(msg.change)) { - let id = Number(key); - if (!this.isWorktree(id)) continue; - let paths = touched.get(id); - if (paths === undefined) touched.set(id, paths = new Set()); - for (let [path] of entries) paths.add(path); - } + if (msg.change !== undefined) touch(msg.change); } + for (let row of liveRows) touch(row.change); return touched; } @@ -5495,11 +5612,13 @@ class OverseerImpl implements AgentHooks { // `joinAs` counts the returned client toward #hasCollaboratorSession for its lifetime; passed by // the collaborator-facing mints, omitted for the owner's and for internal callers (see // GadgetClientImpl). `actorUserId` is who the returned client acts for, for analytics only. - // `bindings` are an agent spawner's (see GatekeeperRecord.bindings). + // `bindings` are an agent spawner's (see GatekeeperRecord.bindings). `createdBy` is the person + // creating the connection, recorded on it (GatekeeperRecord.createdBy); an ambient capsule has + // none. async addGatekeeper( cls: GatekeeperClass, creationSpec: GatekeeperCreationSpec, actorUserId: string, - joinAs?: SessionKind, bindings?: Record) - : Promise> { + joinAs?: SessionKind, bindings?: Record, + createdBy?: AiChatAuthorInfo): Promise> { let id = this.allocateWorkpieceId(); let gatekeeperRecord: GatekeeperRecord = { id, @@ -5507,6 +5626,7 @@ class OverseerImpl implements AgentHooks { creationSpec, }; if (bindings) gatekeeperRecord.bindings = bindings; + if (createdBy) gatekeeperRecord.createdBy = createdBy; // The record is published only once, below, after describe() resolves -- the facet takes the // class directly so it needs no record to exist yet. Publishing it before the await instead @@ -5819,7 +5939,12 @@ class OverseerImpl implements AgentHooks { }); } - async getChatAttachmentData(chatId: number, id: string): Promise { + // The bytes of an attachment committed to `chatId` (see AgentHooks.getChatAttachmentData()): + // the record's own bytes, else its stored blob, else, for a record that holds only a commit and + // a path, the file there. That last read can pull a missing object through a connection, + // unless `local` is set: then it reads only the objects the workspace stores, and a missing + // one throws (see AdminWorkspaceView.readChatAttachment()). + async getChatAttachmentData(chatId: number, id: string, local = false): Promise { let content = this.storage.chatAttachmentContent.get(validateChatAttachmentId(id)); if (!content || content.state.type !== "committed" || content.state.chatId !== chatId) { throw new Error("Chat attachment not found."); @@ -5830,6 +5955,14 @@ class OverseerImpl implements AgentHooks { content.state.workspacePath === undefined) { throw new Error("Chat attachment has no stored content."); } + if (local) { + let entry = this.gitCache.localPathEntryAtCommit( + content.state.workspaceCommitId, content.state.workspacePath); + if (!entry || entry.kind === "dir" || entry.kind === "submodule") { + throw new Error("Chat attachment content is missing from Git."); + } + return this.gitCache.readBlob(entry.oid); + } const entry = await this.gitCache.pathEntryAtCommit( content.state.workspaceCommitId, content.state.workspacePath); if (!entry || entry.kind === "dir" || entry.kind === "submodule") { @@ -6550,9 +6683,17 @@ class OverseerImpl implements AgentHooks { let owner = this.users.get(this.users.idFromString(this.ownerId)); - this.#lastActiveTimeKnownToUserDo = this.#lastActiveTimeKnownToUs!; - await owner.setWorkspaceLastActive(this.ctx.id.toString(), this.#lastActiveTimeKnownToUs!, + let lastActive = this.#lastActiveTimeKnownToUs!; + this.#lastActiveTimeKnownToUserDo = lastActive; + await owner.setWorkspaceLastActive(this.ctx.id.toString(), lastActive, this.storage.totalCost.get()); + + // The directory's activity is precise only to DIRECTORY_ACTIVITY_INTERVAL_MS, so it hears + // of activity far less often than the owner does. + if (Date.now() - this.#directoryActivityAt >= DIRECTORY_ACTIVITY_INTERVAL_MS) { + this.#directoryActivityAt = Date.now(); + this.syncDirectory(lastActive); + } } catch (err) { this.logger.warn("failed to bump gadget last-active on user DO", { event: "gadget.last.active.bump.failed", @@ -6704,6 +6845,103 @@ class OverseerImpl implements AgentHooks { // The snapshot every watcher last acknowledged, to suppress pushes that would change nothing. #lastOutputsPushed?: string; + // --- Deployment directory ---------------------------------------------------------- + // + // The workspace mirrors its title, owner and activity into the deployment directory (see + // UserDirectoryDurableObject), which AdminApi.listWorkspaces() reads. Durable Objects cannot be + // enumerated, so a workspace older than the directory is listed once it is next opened: open() + // syncs on every open, a no-op once the directory has acknowledged the title's revision. + + // The last sync in flight or queued. Syncs chain onto it so they run one at a time, and awaiting + // it awaits every one. Never rejects. + #directorySync: Promise = Promise.resolve(); + #directorySyncScheduled = false; + + // When this instance last reported activity to the directory (epoch ms), so that it reports at + // most once per DIRECTORY_ACTIVITY_INTERVAL_MS. + #directoryActivityAt = 0; + + /** syncDirectory() after the current turn, coalesced: the title subscriber runs before the new + * title is stored. */ + markDirectoryDirty(): void { + if (this.#directorySyncScheduled || !this.ownerId) return; + this.#directorySyncScheduled = true; + scheduler.wait(0).then(() => { + this.#directorySyncScheduled = false; + this.syncDirectory(); + }); + } + + #directorySyncNeeded(lastActive: Date | undefined): boolean { + return this.ownerId !== undefined && !this.directoryRemoved && + (lastActive !== undefined || + this.storage.titleRev.get() !== this.storage.directoryRev.get()); + } + + /** Mirror this workspace into the deployment directory (WorkspaceDirectoryEntry, the title + * whole: the directory clamps it): best-effort, never awaited by its callers, never restarting + * the workspace. Syncs run one at a time; a no-op without an owner, while directoryRemoved is + * set, and once the directory has acknowledged the title's revision when no `lastActive` is + * given. */ + syncDirectory(lastActive?: Date): void { + if (!this.#directorySyncNeeded(lastActive)) return; + this.#directorySync = this.#directorySync.then(() => this.#syncDirectoryStep(lastActive)); + } + + async #syncDirectoryStep(lastActive: Date | undefined): Promise { + try { + // Checked again now that its turn has come: a step queued behind removeFromDirectory() + // sends nothing. + if (!this.#directorySyncNeeded(lastActive)) return; + let ownerId = this.ownerId!; + let id = this.ctx.id.toString(); + let title = this.storage.title.get(); + let created = this.storage.createdAt.get(); + let rev = this.storage.titleRev.get(); + let ownerProfileId = await this.getOwnerProfileId(); + // A deletion that began meanwhile waits for this step before it deletes storage, so a write + // is safe while the owner is unchanged and the workspace has not left the directory. + let current = () => this.ownerId === ownerId && !this.directoryRemoved; + + if (created === undefined) { + // A workspace initialised before it recorded this: its owner's record says, once. A raw + // stub rather than wrapUserDo(), since the directory feed must never restart the + // workspace. + let record = await this.users.get(this.users.idFromString(ownerId)).getWorkspace(id); + if (record && !record.owner) { + created = record.created; + if (current()) this.storage.createdAt.put(created); + } + } + + await this.directories.getByName("").syncWorkspace( + { id, title, ownerId: ownerProfileId, created, lastActive }, rev); + if (current() && this.storage.directoryRev.get() < rev) this.storage.directoryRev.put(rev); + } catch (err) { + this.logger.warn("failed to sync workspace to the deployment directory", { + event: "workspace.directory.sync.failed", error: err, + }); + } + } + + /** + * Take this workspace out of the directory, for deleteSelf() once beginDeletion() has committed. + * Sets directoryRemoved and resets directoryRev, waits for every sync in flight or queued (none + * of which then sends or acknowledges), and removes the entry. Logs a failure and never throws. + */ + async removeFromDirectory(): Promise { + this.directoryRemoved = true; + this.storage.directoryRev.put(-1); + await this.#directorySync; + try { + await this.directories.getByName("").removeWorkspace(this.ctx.id.toString()); + } catch (err) { + this.logger.warn("failed to remove workspace from the deployment directory", { + event: "workspace.directory.remove.failed", error: err, + }); + } + } + // Increment the code version and restart the affected gadgets so they reload. If // `affectedGadgetIds` is omitted, conservatively restarts every gadget (e.g. for code commits, // which are whole-doc updates that may span gadget roots); binding changes pass the one gadget @@ -9241,6 +9479,56 @@ class OverseerImpl implements AgentHooks { this.storage.blueprints.delete(record.id); } + // Every blueprint published from the workspace (see Overseer.listBlueprints()). + blueprintSummaries(): BlueprintGadgetSummary[] { + let result: BlueprintGadgetSummary[] = []; + for (let record of Array.from(this.storage.blueprints.list())) { + result.push({ + id: record.id, + title: record.metadata.title, + description: record.metadata.description, + version: record.metadata.version, + codeVersionDate: this.#blueprintCodeDate(record), + screenshotUrl: blueprintScreenshotUrl(record.id, record.metadata), + dirty: record.dirty, + ...(this.#hasUnpublishedChanges(record) ? {unpublishedChanges: true} : {}), + }); + } + return result; + } + + // Whether the gadget a blueprint exports has committed files other than the ones the + // blueprint last published, which are those of the record's `commitId`. A gadget that has + // since been removed has nothing left to publish, and a record from before git-backed code + // storage has no commit to compare with until it is migrated. + // + // The trees are what is compared, not the commits: publishing an unchanged tree mints nothing + // and leaves `commitId` where it was (see mintBlueprintRelease), so a head that has moved on + // to the same files would otherwise count as a change that no publish could clear. + #hasUnpublishedChanges(record: BlueprintGadgetRecord): boolean { + let gadgetId = record.gadgetId ?? this.defaultGadgetId; + let gadget = gadgetId === undefined ? undefined : this.storage.gadgets.get(gadgetId); + let head = gadget?.type === "gadget" ? gadget.commitId : undefined; + let published = record.commitId; + if (head === undefined || published === undefined || head === published) return false; + let store = this.gitCache; + return store.commitTree(head) !== store.commitTree(published); + } + + // The timestamp of the code exported into a blueprint: the exported commit's author date, or + // for legacy (pre-git-storage) records the legacy log entry's, falling back to the metadata's + // own last-updated time. + #blueprintCodeDate(record: BlueprintGadgetRecord): Date { + if (record.commitId !== undefined) { + return this.gitCache.readCommitLog(record.commitId, {depth: 1})[0].timestamp; + } + if (record.codeVersion !== undefined) { + let codeUpdate = this.storage.code.get(record.codeVersion); + if (codeUpdate) return codeUpdate.timestamp; + } + return record.metadata.lastUpdated; + } + postAgentChatMessage(chatId: number, author: AiChatAuthorInfo, message: string) { let meta = this.storage.chatMeta.get(chatId); if (!meta) { @@ -9992,12 +10280,15 @@ class OverseerImpl implements AgentHooks { let minted = await this.#ownerUserStub().createResourceGatekeeper( input.vendorId, input.resourceUrlPattern, input.title); let id = this.allocateWorkpieceId(); + // The resource is created by whoever started the turn this tool call belongs to. + let createdBy = this.storage.activeAgents.get(chatId)?.initiator; this.storage.gatekeepers.put({ id, class: minted.class, resourceTitle: input.title, creationSpec: {type: "gatekeeper", vendorId: input.vendorId, typeUrlPattern: minted.typeUrlPattern}, + ...(createdBy !== undefined ? {createdBy} : {}), }); await this.submitAction(id, "create", minted.action, {from: "agent", chatId}) .catch(error => { this.removeGatekeeper(id); throw error; }); @@ -10849,6 +11140,50 @@ class OverseerImpl implements AgentHooks { return this.#sharingManager; } + // The workspace's active share links (see Overseer.listShareLinks()). `ownerProfile` reads the + // owner's profile from their account, and is called only when the owner created a link. + async shareLinkInfos(ownerProfile: () => Promise): Promise { + let sharing = await this.getSharingManager(); + + // Collect all records synchronously to release the kv.list() iterator before any await + // points below. Only one kv.list() iterator can be active at a time, and concurrent RPC + // calls (e.g. listCollaborators) may start their own. + let records = sharing.listShareLinkRecords(); + + let result: ShareLinkInfo[] = []; + // Cache profile lookups. + let profileCache = new Map(); + + for (let record of records) { + let createdBy = profileCache.get(record.createdBy); + if (!createdBy) { + // Check if the creator is the owner (requires an RPC to the owner's DO). + let ownerProfileId = await this.getOwnerProfileId(); + if (ownerProfileId === record.createdBy) { + createdBy = await ownerProfile(); + } + // Check if the creator is a collaborator (resolved locally). + if (!createdBy) { + createdBy = sharing.getCreatorProfile(record.createdBy); + } + // Fallback. + if (!createdBy) { + createdBy = { type: "user", id: record.createdBy, name: record.createdBy }; + } + profileCache.set(record.createdBy, createdBy); + } + result.push({ + linkId: record.id, + note: record.note, + created: record.created, + createdBy, + role: record.role ?? "build", + ...(record.gadgetId !== undefined ? {gadgetId: record.gadgetId} : {}), + }); + } + return result; + } + #codeIdMap = new Map; // Gadgets that had persistent restore stubs forged during each chat's currently-running @@ -11016,6 +11351,12 @@ export class OverseerDurableObject extends DurableObject { // A workspace initialized by this version of the code is born at the current schema version; // there is nothing to migrate. this.impl.storage.version.put(OVERSEER_STORAGE_VERSION); + + this.impl.storage.createdAt.put(new Date()); + // Outranks whatever the directory still lists under this id (from an earlier life whose + // removal from the directory failed), as the title's time-based revision does. + this.impl.storage.titleRev.put(Math.max(this.impl.storage.titleRev.get() + 1, Date.now())); + this.impl.directoryRemoved = false; } /** @@ -11075,6 +11416,10 @@ export class OverseerDurableObject extends DurableObject { this.impl.ownerProfileId = profileId; } + // The directory's lazy backfill: a workspace older than the directory is listed once it is + // next opened. A no-op once the directory has acknowledged the title's revision. + this.impl.syncDirectory(); + // Make singleton gatekeepers (e.g. the Context Library) available to the agent as unnamed // capsules. Idempotent and best-effort, so a library hiccup never blocks opening the gadget. // On the very first open we block so the agent's first turn sees the capsules; later opens let the @@ -11184,6 +11529,70 @@ export class OverseerDurableObject extends DurableObject { ensureCapsules); } + /** + * Mint a deployment admin's read-only view of this workspace (AdminWorkspaceView), for + * AdminApiImpl.inspectWorkspace(): reachable only over the Workshop's Durable Object binding, + * never through a capability a user holds. Every environment read on this path goes through the + * OverseerImpl (`this.impl.env`), never this object's own `env`. + * + * Refuses (`workspaceNotFound`) a workspace never initialised or since deleted, before anything + * else and writing nothing: unlike open()'s first-open path it never asks a user DO whether the + * workspace is theirs and never claims ownership. Rejects (TypeError) malformed terms: a blank or + * over-long reason, a `via` that is not "api" or "cli", an `access` or a `requested` that is not + * "metadata" or "full", an `access` above `requested`, and an `expiresAt` that is not a valid + * Date or lies more than MAX_ADMIN_VIEW_LIFETIME_SECONDS (plus a minute of clock skew) ahead. An + * `expiresAt` already past is not malformed: the open is then refused `viewExpired`, unrecorded, + * as any call on an expired view is. + * + * By explicit policy it skips authorizeCollaborator() and observer verification (see + * docs/observers.md): the admin is not a collaborator, is never registered with the workspace's + * gatekeepers, and the audit log stands in for that check. It joins no presence, session count + * or outputs fan-out, and starts no agent, ambient capsule or gatekeeper facet; waking the + * Durable Object still resumes turns a previous instance left running, as any wake does. + * + * The open is the view's first call ("inspectWorkspace"): it passes the view's checks (a caller + * no longer an admin is refused unrecorded), runs at most at the deployment's current level (so + * an open while that is "off" is recorded as refused and throws `accessOff`), and is recorded in + * the deployment-wide log before the view is returned; a record that cannot be written refuses + * the open. + */ + async openForAdmin(adminId: string, reason: string, sessionId: string, terms: AdminViewTerms) + : Promise { + if (!this.impl.ownerId) { + throw createAdminAccessError(ADMIN_ACCESS_ERROR_CODES.workspaceNotFound); + } + + let trimmedReason = validateAdminAccessReason(reason); + if (terms.via !== "api" && terms.via !== "cli") { + throw new TypeError("An admin view's via must be \"api\" or \"cli\"."); + } + if (terms.access !== "metadata" && terms.access !== "full") { + throw new TypeError("An admin view's access must be \"metadata\" or \"full\"."); + } + if (terms.requested !== "metadata" && terms.requested !== "full") { + throw new TypeError("An admin view's requested level must be \"metadata\" or \"full\"."); + } + if (lowerAdminWorkspaceAccess(terms.access, terms.requested) !== terms.access) { + throw new TypeError("An admin view's access must not be above the level it asked for."); + } + let expiresAt = terms.expiresAt instanceof Date ? terms.expiresAt.getTime() : NaN; + if (Number.isNaN(expiresAt) || + expiresAt > Date.now() + (MAX_ADMIN_VIEW_LIFETIME_SECONDS + 60) * 1000) { + throw new TypeError( + "An admin view's expiry must be a valid time within its longest lifetime."); + } + + // The terms' level is not clamped here: the open, like every call, runs at most at the + // deployment's current level, the environment's cap included. The owner is resolved now, since + // a call's record is built before its first await (see AdminWorkspaceViewImpl), and never + // changes. + let ownerId = await this.impl.getOwnerProfileId(); + let view = await AdminWorkspaceViewImpl.open( + this.impl, adminId, trimmedReason, sessionId, terms, ownerId); + this.impl.logger.info("admin workspace view opened", {event: "admin.view.opened"}); + return view; + } + #getExternalChat(externalChatKey: string): ExternalChatRecord | undefined { let externalChat = this.impl.storage.externalChats.get(externalChatKey); if (externalChat && !this.impl.storage.chatMeta.get(externalChat.chatId)) { @@ -11791,6 +12200,62 @@ function joinSessionPresence( }; } +// A stored chat message as clients see it, an action message carrying its action's current log +// entry. +function chatMessageForClient(impl: OverseerImpl, msg: StoredChatMessage): AiChatMessage { + if (msg.type === "action") { + let record = impl.storage.actions.get(msg.actionId); + if (record) { + msg.actionLog = actionRecordToLog(record); + } + } + return impl.hydrateChatMessageForClient(msg); +} + +// One page of a chat's history (see Overseer.getChatHistory()). Compaction boundaries delimit the +// pages: the newest page is the tail replay still scans, and each earlier page is the span one +// checkpoint summarized. A thread that was never compacted has a single page. +function chatHistoryPage(impl: OverseerImpl, chatId: number, beforeSequence?: number) + : AiChatHistoryPage { + let checkpoint = beforeSequence === undefined + ? impl.getActiveChatCompaction(chatId) + : impl.getChatCompactionBelow(chatId, beforeSequence); + let result = [...impl.storage.chats.list({ + prefix: chatKeyPrefix(chatId), + start: checkpoint && chatKey(chatId, checkpoint.compactedTo), + end: beforeSequence === undefined ? undefined : chatKey(chatId, beforeSequence), + })]; + return { + messages: result.map((msg) => chatMessageForClient(impl, msg)), + compacted: checkpoint && { + to: checkpoint.compactedTo, + summary: checkpoint.summary, + proposedChange: checkpoint.proposedChange, + }, + }; +} + +// Every hook the workspace's connections deliver (see Overseer.listHooks()). +function boundHookInfos(impl: OverseerImpl): BoundHookInfo[] { + let defaultGadgetId = impl.defaultGadgetId; + let result: BoundHookInfo[] = []; + for (let record of impl.storage.boundHooks.list()) { + let gatekeeper = impl.storage.gatekeepers.get(record.gatekeeperId); + result.push({ + id: record.id, + gatekeeperId: record.gatekeeperId, + // Hooks recorded before multi-gadget support carry no gadgetId; they belong to the + // default gadget, which necessarily exists in any workspace old enough to have them. + gadgetId: (record.gadgetId ?? defaultGadgetId)!, + resourceTitle: gatekeeper?.resourceTitle, + resourceUrl: gatekeeper?.resourceUrl, + description: record.description, + enabled: record.enabled, + }); + } + return result; +} + @validateRpc() class OverseerClientInterface extends RpcTarget implements Overseer { #clientProfilePromise: Promise | undefined; @@ -12119,6 +12584,7 @@ class OverseerClientInterface extends RpcTarget implements Overseer { let readers = new Set([this.impl.ownerId, ...sharing.listCollaborators() .map(reader => this.impl.users.idFromName(reader.profile.id).toString())]); this.impl.beginDeletion(readers); + await this.impl.removeFromDirectory(); this.impl.recordGadgetAnalytics({ event_name: "gadget_deleted", user_id: this.#clientUser.id.toString(), @@ -12209,7 +12675,8 @@ class OverseerClientInterface extends RpcTarget implements Overseer { typeUrlPattern, }; let result = await this.impl.addGatekeeper( - cls, creationSpec, this.clientUserId, this.#mintedCapabilityKind()); + cls, creationSpec, this.clientUserId, this.#mintedCapabilityKind(), undefined, + await this.#getClientProfile()); await this.recordConnectionCreated(result, "gatekeeper", vendorId); return result; } @@ -12235,7 +12702,8 @@ class OverseerClientInterface extends RpcTarget implements Overseer { modelKind(props.config.provider, props.config.model) === "decision" ? this.impl.ctx.exports.DecisionModelGatekeeper({props}) : this.impl.ctx.exports.LanguageModelGatekeeper({props}), - creationSpec, this.clientUserId, this.#mintedCapabilityKind()); + creationSpec, this.clientUserId, this.#mintedCapabilityKind(), undefined, + await this.#getClientProfile()); await this.recordConnectionCreated(result, "ai_model"); return result; } @@ -12269,31 +12737,15 @@ class OverseerClientInterface extends RpcTarget implements Overseer { let result = await this.impl.addGatekeeper( this.impl.ctx.exports.AgentSpawnerGatekeeper({props}), creationSpec, - this.clientUserId, this.#mintedCapabilityKind(), bindings); + this.clientUserId, this.#mintedCapabilityKind(), bindings, + await this.#getClientProfile()); await this.recordConnectionCreated(result, "agent_spawner"); return result; } async listActions(options?: {beforeId?: number, filter?: ActionHistoryFilter}) : Promise { - let {beforeId, filter = "all"} = options ?? {}; - if (beforeId !== undefined && (!Number.isSafeInteger(beforeId) || beforeId < 0)) { - throw new TypeError(`Invalid beforeId: ${beforeId}`); - } - - // One ranged read -- off the collection itself for "all" (already id-ordered), off - // byHistoryFilter otherwise -- so the work is O(page) however sparse the matches. Pages are - // full until the last; the +1 record probes whether an older page exists. - let actions = this.impl.storage.actions; - let range = {end: beforeId, reverse: true, limit: ACTION_HISTORY_PAGE_DEFAULT_LIMIT + 1}; - let page = [...(filter === "all" - ? actions.list(range) : actions.byHistoryFilter.get(filter, range))]; - let more = page.length > ACTION_HISTORY_PAGE_DEFAULT_LIMIT; - if (more) page.pop(); - return { - entries: page.map(actionRecordToLog), - nextBeforeId: more ? page.at(-1)!.id : undefined, - }; + return actionHistoryPage(this.impl.storage, options); } async approveAction(id: number, creation?: CreationApproval): Promise { @@ -12359,24 +12811,7 @@ class OverseerClientInterface extends RpcTarget implements Overseer { } async listHooks(): Promise { - let defaultGadgetId = this.impl.defaultGadgetId; - let result: BoundHookInfo[] = []; - for (let record of this.impl.storage.boundHooks.list()) { - let gatekeeper = this.impl.storage.gatekeepers.get(record.gatekeeperId); - result.push({ - id: record.id, - gatekeeperId: record.gatekeeperId, - // Hooks recorded before multi-gadget support carry no gadgetId; they belong to the - // default gadget, which necessarily exists in any workspace old enough to have them. - gadgetId: (record.gadgetId ?? defaultGadgetId)!, - resourceTitle: gatekeeper?.resourceTitle, - resourceUrl: gatekeeper?.resourceUrl, - description: record.description, - enabled: record.enabled, - }); - } - - return result; + return boundHookInfos(this.impl); } async enableHook(id: number): Promise { @@ -12767,41 +13202,13 @@ class OverseerClientInterface extends RpcTarget implements Overseer { return await this.impl.getChatAttachmentData(chatId, id); } - // Compaction boundaries delimit the pages: the newest page is the tail replay still scans, and each - // earlier page is the span one checkpoint summarized. A thread that was never compacted has a - // single page. async getChatHistory(chatId: number, beforeSequence?: number): Promise { - let checkpoint = beforeSequence === undefined - ? this.impl.getActiveChatCompaction(chatId) - : this.impl.getChatCompactionBelow(chatId, beforeSequence); - let result = [...this.impl.storage.chats.list({ - prefix: chatKeyPrefix(chatId), - start: checkpoint && chatKey(chatId, checkpoint.compactedTo), - end: beforeSequence === undefined ? undefined : chatKey(chatId, beforeSequence), - })]; - return { - messages: result.map((msg) => this.#getChatMessageForClient(msg)), - compacted: checkpoint && { - to: checkpoint.compactedTo, - summary: checkpoint.summary, - proposedChange: checkpoint.proposedChange, - }, - }; + return chatHistoryPage(this.impl, chatId, beforeSequence); } async getChatMessage(chatId: number, sequence: number): Promise { let msg = this.impl.storage.chats.get(chatKey(chatId, sequence)); - return msg && this.#getChatMessageForClient(msg); - } - - #getChatMessageForClient(msg: StoredChatMessage): AiChatMessage { - if (msg.type === "action") { - let record = this.impl.storage.actions.get(msg.actionId); - if (record) { - msg.actionLog = actionRecordToLog(record); - } - } - return this.impl.hydrateChatMessageForClient(msg); + return msg && chatMessageForClient(this.impl, msg); } async subscribeToChat(subscriber: RpcStub, startAfter?: Date) @@ -12834,7 +13241,7 @@ class OverseerClientInterface extends RpcTarget implements Overseer { let self = this; function deliverMessage(record: AiChatMessage) { - subscriber.message(self.#getChatMessageForClient(record)).catch(unsubscribe); + subscriber.message(chatMessageForClient(impl, record)).catch(unsubscribe); } let msgSubscriber = { @@ -13095,7 +13502,7 @@ class OverseerClientInterface extends RpcTarget implements Overseer { // or web fetch it stopped to wait on. Not the other records logged under it: the `changes` a // step that replied and edited records after its reply, nor an agentNudge (as Stop logs one). if (msg.author.type === "agent" && WAIT_FOR_AGENT_RESULT_TYPES.has(msg.type)) { - return this.#getChatMessageForClient(msg); + return chatMessageForClient(this.impl, msg); } } return null; @@ -13120,52 +13527,7 @@ class OverseerClientInterface extends RpcTarget implements Overseer { // --- Blueprint management --- async listBlueprints(): Promise { - let result: BlueprintGadgetSummary[] = []; - for (let record of Array.from(this.impl.storage.blueprints.list())) { - result.push({ - id: record.id, - title: record.metadata.title, - description: record.metadata.description, - version: record.metadata.version, - codeVersionDate: this.#blueprintCodeDate(record), - screenshotUrl: blueprintScreenshotUrl(record.id, record.metadata), - dirty: record.dirty, - ...(this.#hasUnpublishedChanges(record) ? {unpublishedChanges: true} : {}), - }); - } - return result; - } - - // Whether the gadget a blueprint exports has committed files other than the ones the - // blueprint last published, which are those of the record's `commitId`. A gadget that has - // since been removed has nothing left to publish, and a record from before git-backed code - // storage has no commit to compare with until it is migrated. - // - // The trees are what is compared, not the commits: publishing an unchanged tree mints nothing - // and leaves `commitId` where it was (see mintBlueprintRelease), so a head that has moved on - // to the same files would otherwise count as a change that no publish could clear. - #hasUnpublishedChanges(record: BlueprintGadgetRecord): boolean { - let gadgetId = record.gadgetId ?? this.impl.defaultGadgetId; - let gadget = gadgetId === undefined ? undefined : this.impl.storage.gadgets.get(gadgetId); - let head = gadget?.type === "gadget" ? gadget.commitId : undefined; - let published = record.commitId; - if (head === undefined || published === undefined || head === published) return false; - let store = this.impl.gitCache; - return store.commitTree(head) !== store.commitTree(published); - } - - // The timestamp of the code exported into a blueprint: the exported commit's author date, or - // for legacy (pre-git-storage) records the legacy log entry's, falling back to the metadata's - // own last-updated time. - #blueprintCodeDate(record: BlueprintGadgetRecord): Date { - if (record.commitId !== undefined) { - return this.impl.gitCache.readCommitLog(record.commitId, {depth: 1})[0].timestamp; - } - if (record.codeVersion !== undefined) { - let codeUpdate = this.impl.storage.code.get(record.codeVersion); - if (codeUpdate) return codeUpdate.timestamp; - } - return record.metadata.lastUpdated; + return this.impl.blueprintSummaries(); } async updateBlueprint(blueprintId: string, options: { @@ -13341,45 +13703,8 @@ class OverseerClientInterface extends RpcTarget implements Overseer { } async listShareLinks(): Promise { - let sharing = await this.impl.getSharingManager(); - - // Collect all records synchronously to release the kv.list() iterator before any await - // points below. Only one kv.list() iterator can be active at a time, and concurrent RPC - // calls (e.g. listCollaborators) may start their own. - let records = sharing.listShareLinkRecords(); - - let result: ShareLinkInfo[] = []; - // Cache profile lookups. - let profileCache = new Map(); - - for (let record of records) { - let createdBy = profileCache.get(record.createdBy); - if (!createdBy) { - // Check if the creator is the owner (requires an RPC to the owner's DO). - let ownerProfileId = await this.impl.getOwnerProfileId(); - if (ownerProfileId === record.createdBy) { - createdBy = await retryOnDoReset(() => this.#owner.whoami(), this.impl.logger); - } - // Check if the creator is a collaborator (resolved locally). - if (!createdBy) { - createdBy = sharing.getCreatorProfile(record.createdBy); - } - // Fallback. - if (!createdBy) { - createdBy = { type: "user", id: record.createdBy, name: record.createdBy }; - } - profileCache.set(record.createdBy, createdBy); - } - result.push({ - linkId: record.id, - note: record.note, - created: record.created, - createdBy, - role: record.role ?? "build", - ...(record.gadgetId !== undefined ? {gadgetId: record.gadgetId} : {}), - }); - } - return result; + return this.impl.shareLinkInfos( + () => retryOnDoReset(() => this.#owner.whoami(), this.impl.logger)); } async updateShareLink(linkId: string, note?: string): Promise { @@ -13702,6 +14027,502 @@ class UseOverseerInterface extends RpcTarget implements Overseer { async previewRevokeShareLink(_linkId: string): Promise { this.#deny(); } } +// A deployment admin's read-only view of the workspace (see AdminWorkspaceView in api.ts and +// OverseerDurableObject.openForAdmin()). Deliberately not an Overseer: it holds no session, joins +// no presence or fan-out, and never reads through a connection: its reads of files, history, +// attachments and a chat's pending content never pull, and what a connection could tell it, it +// reads from the workspace's own records instead. Beyond the workspace's storage it reads only +// the owner's profile, from their account (describe() and listShareLinks()). Every method runs +// through #call, which reads the call's answer and records the call, with what that answer +// exposed, in the deployment-wide log before returning anything. +@validateRpc() +class AdminWorkspaceViewImpl extends RpcTarget implements AdminWorkspaceView { + #impl: OverseerImpl; + #adminId: string; + #reason: string; + #sessionId: string; + #terms: AdminViewTerms; + #ownerId: string; + + constructor(impl: OverseerImpl, adminId: string, reason: string, sessionId: string, + terms: AdminViewTerms, ownerId: string) { + super(); + this.#impl = impl; + this.#adminId = adminId; + this.#reason = reason; + this.#sessionId = sessionId; + this.#terms = terms; + this.#ownerId = ownerId; + } + + // Mints a view, recording the open as its first call (see openForAdmin()). Static, so that + // auditing stays off the view's RPC surface. + static async open(impl: OverseerImpl, adminId: string, reason: string, sessionId: string, + terms: AdminViewTerms, ownerId: string): Promise { + let view = new AdminWorkspaceViewImpl(impl, adminId, reason, sessionId, terms, ownerId); + await view.#call( + "inspectWorkspace", {expiresAt: terms.expiresAt.toISOString()}, () => undefined); + return view; + } + + // The checks that stop a call unrecorded: the view has not expired, its admin is still one, and + // the workspace still exists. + #assertLive(): void { + assertAdminViewLive(this.#terms, this.#adminId, this.#impl.env); + if (!this.#impl.ownerId) { + throw createAdminAccessError(ADMIN_ACCESS_ERROR_CODES.workspaceNotFound); + } + } + + // Runs one call: checks the view, reads the deployment's setting, checks again, and takes the + // level that gives. A call that level refuses is recorded and refused, having read nothing. Any + // other call reads its answer with `read` (never through a connection), is checked once more, + // and is recorded with what that answer exposed (`holdsContent`). Only then is it + // answered, so nothing it read leaves the server unrecorded, and a call whose record cannot be + // written fails and returns nothing. A read that fails is recorded as an attempt that exposed + // nothing, and the call fails with the read's error. + async #call(method: AdminWorkspaceMethod, args: Record, + read: (access: AdminViewAccess, policy: AdminAccessPolicy) => T | Promise, + holdsContent: (answer: T) => boolean = () => false): Promise { + this.#assertLive(); + let config = await readAdminConfig(this.#impl.env); + // Checked again, with nothing awaited from here to a refusal's record: the read may have + // spanned the expiry, a demotion or the workspace's deletion, and a deleted workspace's storage + // must stay empty, with nothing appended for it deployment-wide either. + this.#assertLive(); + + let policy = adminAccessPolicy(config, this.#impl.env); + let {access, refused} = adminViewCallAccess(this.#terms, method, policy); + if (refused !== undefined || access === "off") { + let code = refused ?? ADMIN_ACCESS_ERROR_CODES.accessOff; + await this.#record(method, args, access, "metadata", code); + throw createAdminAccessError(code); + } + + let answer: T; + try { + answer = await read(access, policy); + } catch (err) { + this.#assertLive(); + await this.#record(method, args, access, "metadata"); + throw err; + } + // Checked again, with nothing awaited before the record, for the same reasons: the read may + // have awaited. + this.#assertLive(); + await this.#record(method, args, access, holdsContent(answer) ? "content" : "metadata"); + return answer; + } + + // Records one call in the deployment-wide log. The record is built and sent before the first + // await, so that the caller's last check still holds. + async #record(method: AdminWorkspaceMethod, args: Record, + access: AdminWorkspaceAccess, exposed: AdminAccessExposure, + refused?: AdminAccessRecord["refused"]): Promise { + let impl = this.#impl; + let record: AdminAccessRecord = { + kind: "workspace", + workspaceId: impl.ctx.id.toString(), + workspaceOwner: this.#ownerId, + at: new Date(), + admin: this.#adminId, + sessionId: this.#sessionId, + via: this.#terms.via, + // A title has no length limit of its own, and an uncut one could overflow the + // deployment-wide row and so refuse every investigation of this workspace. + workspaceTitle: cutAdminAuditText(impl.storage.title.get(), MAX_ADMIN_AUDIT_ARG_LENGTH), + method, + args: redactAdminAuditArgs(args), + reason: this.#reason, + requested: this.#terms.requested, + access, + exposed, + ...(refused ? {refused} : {}), + }; + try { + // No retention: the policy above comes from the KV mirror, and a stale one must not prune. + await impl.adminAudits.getByName("").append(record); + } catch (err) { + impl.logger.warn("failed to append to the admin audit log", { + event: "admin.audit.append.failed", operation: method, error: err, + }); + throw err; + } + } + + // The head commit of a permanent gadget, the only workpiece with committed code to read. + #committedHead(gadgetId: WorkpieceId): string { + let record = this.#impl.storage.gadgets.get(gadgetId); + if (record?.type !== "gadget" || record.pending || record.commitId === undefined) { + throw new Error(`No gadget with committed code has id ${gadgetId}.`); + } + return record.commitId; + } + + // A commit the workspace stores, by id (a gadget's history, a worktree's commits, a chat pin's + // base); throws for a malformed id and for one naming no stored commit. + #storedCommit(commitId: string): string { + if (this.#impl.gitCache.localObjectType(validateOid(commitId)) !== "commit") { + throw new Error(`No commit ${commitId} is stored in this workspace.`); + } + return commitId; + } + + // The chat's current content, its worktree bases read only from stored objects. + #pendingContent(chatId: number): Promise { + let impl = this.#impl; + return impl.getCurrentChatContent(chatId, impl.getChatMetaOrThrow(chatId), + (commit, path) => impl.gitCache.readLocalFileAtCommitIfExists(commit, path)); + } + + // The owner's profile, from their account: the one thing a view reads beyond the workspace's + // storage (describe() and listShareLinks()). + #ownerProfile(): Promise { + return retryOnDoReset(() => this.#impl.ownerUserDo().whoami(), this.#impl.logger); + } + + async describe(): Promise { + let impl = this.#impl; + return this.#call("describe", {}, async access => { + let owner = await this.#ownerProfile(); + let collaborators = await (await impl.getSharingManager()).listCollaborators(); + return { + id: impl.ctx.id.toString(), + // Live data, not a record, so whole. + title: impl.storage.title.get(), + totalCost: impl.storage.totalCost.get(), + containsRestrictedData: impl.storage.containsRestrictedData.get(), + ownerInvitesOnly: impl.storage.ownerInvitesOnly.get(), + defaultGadgetId: impl.defaultGadgetId, + owner, + workpieces: [...impl.storage.gadgets.list()] + .map(record => workpieceSummary(record, access === "full")), + collaborators, + access, + expiresAt: this.#terms.expiresAt, + }; + }, description => description.workpieces.some(workpieceHoldsContent)); + } + + async listChats(): Promise { + return this.#call("listChats", {}, + access => [...this.#impl.storage.chatMeta.list({reverse: true})] + .map(meta => this.#impl.chatMetaForClient(meta)) + .map(meta => access === "full" ? meta : withholdChatContent(meta)), + chats => chats.some(chatHoldsContent)); + } + + async getChat(chatId: number, beforeSequence?: number): Promise { + return this.#call("getChat", {chatId, beforeSequence}, + () => chatHistoryPage(this.#impl, chatId, beforeSequence), chatPageHoldsContent); + } + + // Each part of the answer names the fields its type declares, since the stored records hold + // more: a turn's and a queued call's hold their initiator's user id, and a queued call's its + // raw arguments. Each storage listing is read to its end before the next starts. + async getChatState(chatId: number): Promise { + return this.#call("getChatState", {chatId}, access => { + let storage = this.#impl.storage; + let meta = this.#impl.getChatMetaOrThrow(chatId); + let turn = storage.activeAgents.get(chatId); + let prefix = chatKeyPrefix(chatId); + let [newest] = [...storage.chats.list({prefix, reverse: true, limit: 1})]; + let queuedCalls = [...storage.pendingAgentCalls.list({prefix})].map(call => ({ + callId: call.callId, + methodName: call.methodName, + initiatorModelId: call.initiatorModelId, + argsSummary: call.argsSummary, + })); + let pendingActionIds = [...storage.actions.byHistoryFilter.get("pending")] + .filter(({caller}) => "chatId" in caller && caller.chatId === chatId) + .map(action => action.id); + let state: AdminChatState = { + turn: turn && { + modelId: turn.modelId, + initiator: turn.initiator, + callbackInitiated: turn.callbackInitiated, + resumeAttempts: turn.resumeAttempts ?? 0, + paused: meta.activeAgent === undefined, + }, + lastMessageAt: newest?.timestamp, + queuedCalls, + pendingActionIds, + }; + return access === "full" ? state : withholdChatStateContent(state); + }, chatStateHoldsContent); + } + + // The scan deleteChat() makes: a message's ChatAttachmentRef is the one record of an + // attachment's metadata. Each entry names the fields its type declares, since a stored + // reference also holds its path in the chat's scratch worktree, and can hold bytes. + async listChatAttachments(chatId: number): Promise { + return this.#call("listChatAttachments", {chatId}, () => { + let attachments: AdminChatAttachment[] = []; + for (let msg of this.#impl.storage.chats.list({prefix: chatKeyPrefix(chatId)})) { + if (msg.type !== "message") continue; + for (let {id, oid, mimeType, name, size} of msg.attachments ?? []) { + attachments.push({id, oid, mimeType, name, size, sequence: msg.sequence}); + } + } + return attachments; + }); + } + + async readChatAttachment(chatId: number, id: string): Promise { + return this.#call("readChatAttachment", {chatId, id}, + () => this.#impl.getChatAttachmentData(chatId, id, true), () => true); + } + + async listActions(options?: {beforeId?: number, filter?: ActionHistoryFilter}) + : Promise { + validateBeforeId(options?.beforeId); + return this.#call("listActions", {beforeId: options?.beforeId, filter: options?.filter}, + access => { + let page = actionHistoryPage(this.#impl.storage, options); + return access === "full" + ? page : {...page, entries: page.entries.map(withholdActionContent)}; + }, + page => page.entries.some(actionHoldsContent)); + } + + async listFiles(gadgetId: WorkpieceId): Promise { + return this.#call("listFiles", {gadgetId}, + () => this.#impl.gitCache.readLocalCommitTree(this.#committedHead(gadgetId))); + } + + async readFile(gadgetId: WorkpieceId, path: string): Promise { + return this.#call("readFile", {gadgetId, path}, + () => this.#impl.gitCache.readLocalFileAtCommit(this.#committedHead(gadgetId), path), + fileHoldsContent); + } + + // `limit`, the depth the validator answers with, bounds the walk: readCommitLog() stops only at + // a depth it is handed. The record keeps `depth` as the caller gave it. + async getCommitLog(fromCommit: string, depth?: number): Promise { + let limit = adminCommitLogDepth(depth); + return this.#call("getCommitLog", {fromCommit, depth}, + access => this.#impl.gitCache + .readCommitLog(this.#storedCommit(fromCommit), {depth: limit, stopAtMissing: true}) + .map(commit => access === "full" ? commit : withholdCommitContent(commit)), + log => log.some(commitHoldsContent)); + } + + async listFilesAtCommit(commitId: string): Promise { + return this.#call("listFilesAtCommit", {commitId}, + () => this.#impl.gitCache.readLocalCommitTree(this.#storedCommit(commitId))); + } + + async readFileAtCommit(commitId: string, path: string): Promise { + return this.#call("readFileAtCommit", {commitId, path}, + () => this.#impl.gitCache.readLocalFileAtCommit(this.#storedCommit(commitId), path), + fileHoldsContent); + } + + async listChangedPaths(fromCommit: string, toCommit: string): Promise { + return this.#call("listChangedPaths", {fromCommit, toCommit}, + async () => [...await this.#impl.gitCache.changedLocalFilePathsBetween( + this.#storedCommit(fromCommit), this.#storedCommit(toCommit))].toSorted()); + } + + // One entry per workpiece the chat's content holds and the workspace still has: a pin outlives + // its workpiece's removal and proposes nothing (see proposedChangeWorkpieceIds()). Each is + // compared with its pin's base, read only from stored objects. A gadget's content is its whole + // file map. A worktree's holds only the paths the chat touched, a removed one being absent from + // it, so the change stream names its paths and the base's tree says what each was. + async listPendingChanges(chatId: number): Promise { + return this.#call("listPendingChanges", {chatId}, async () => { + let impl = this.#impl; + // The fold settles one microtask before this goes on, and an accept's synchronous tail can + // run in that gap and clear the pins the content was folded from. So the chat's code base + // and message sequence are read on both sides of the fold, and a fold they moved under is + // taken again. + let content: CodeContent; + let codeBase: ChatCodeBase; + for (let attempt = 0; ; attempt++) { + let before = impl.chatCodeBase(impl.getChatMetaOrThrow(chatId)); + let token = impl.nextChatSequencePeek(chatId); + content = await this.#pendingContent(chatId); + codeBase = impl.chatCodeBase(impl.getChatMetaOrThrow(chatId)); + if (codeBase.generation === before.generation && codeBase.revision === before.revision && + impl.nextChatSequencePeek(chatId) === token) { + break; + } + if (attempt >= 4) throw new Error("The chat is changing too quickly; please retry."); + } + // Nothing is awaited from here on, so the pins and the change stream read below are those + // of the code base the content is at. The content may still be a cached fold that lacks + // one of those pins: a step that only commits in a worktree pins it without moving the + // generation or the revision, which are all the cache is keyed by. A worktree pinned that + // way and not edited since is then left out, where a fresh fold lists it with three empty + // lists. + let pins = new Map(codeBase.pins.map(pin => [pin.gadgetId, pin])); + let messages = [...impl.storage.chats.list({prefix: chatKeyPrefix(chatId)})]; + let touched = impl.worktreeTouchedPaths(messages, chatChangeStatuses(messages), + impl.listLiveChatChanges(chatId, codeBase.generation)); + return [...content] + .filter(([gadgetId]) => impl.storage.gadgets.get(gadgetId) !== undefined) + .toSorted(([a], [b]) => a - b) + .map(([gadgetId, files]) => { + let pin = pins.get(gadgetId); + let added: string[] = [], modified: string[] = [], removed: string[] = []; + if (pin === undefined) { + // A gadget the chat created: it has no commit, so every file is added. + added = [...files.keys()]; + } else if (impl.isWorktree(gadgetId)) { + for (let path of touched.get(gadgetId) ?? []) { + let file = files.get(path); + let entry = impl.gitCache.localPathEntryAtCommit(pin.baseCommit, path); + let base = entry?.kind === "dir" ? undefined : entry; + if (file === undefined) { + if (base !== undefined) removed.push(path); + } else if (base === undefined) { + added.push(path); + } else if (base.oid !== (typeof file === "string" ? blobOid(file) : file.oid)) { + modified.push(path); + } + } + } else { + let base = impl.gitCache.readCommitFiles(pin.baseCommit); + for (let [path, file] of files) { + if (!base.has(path)) added.push(path); + else if (base.get(path) !== file) modified.push(path); + } + removed = [...base.keys()].filter(path => !files.has(path)); + } + return { + gadgetId, + ...(pin && {baseCommit: pin.baseCommit, mergedCommit: pin.mergedCommit}), + added: added.toSorted(), modified: modified.toSorted(), removed: removed.toSorted(), + }; + }); + }); + } + + async readPendingFile(chatId: number, gadgetId: WorkpieceId, path: string) + : Promise { + return this.#call("readPendingFile", {chatId, gadgetId, path}, + async (): Promise => { + let impl = this.#impl; + let files = (await this.#pendingContent(chatId)).get(gadgetId); + if (files === undefined || impl.storage.gadgets.get(gadgetId) === undefined) { + throw new Error(`Chat ${chatId} holds no changes to workpiece ${gadgetId}.`); + } + let file = files.get(path); + if (file === undefined) return {kind: "absent"}; + if (typeof file !== "string") { + // Bytes are streamed from the stored blob, never read whole. + return { + kind: "stream", byteLength: file.byteLength, + content: blobByteStream(impl.gitCache.openBlobRef(file.oid, file.byteLength)), + }; + } + // The text's UTF-8 encoding, made once: its size decides how the file arrives, and a + // stream reads from it. + let encoded = new Blob([file]); + return encoded.size > MAX_BUFFERED_BLOB_SIZE + ? {kind: "stream", byteLength: encoded.size, content: encoded.stream()} + : {kind: "text", text: file}; + }, fileHoldsContent); + } + + async listGatekeepers(): Promise { + return this.#call("listGatekeepers", {}, access => { + let storage = this.#impl.storage; + let gadgets = [...storage.gadgets.list()].filter(record => record.type === "gadget"); + let hooks = boundHookInfos(this.#impl); + return [...storage.gatekeepers.list()].map(record => ({ + id: record.id, + resourceTitle: record.resourceTitle, + resourceUrl: record.resourceUrl, + vendorId: gatekeeperVendorId(record), + creationSpec: record.creationSpec, + createdBy: record.createdBy, + bindings: gadgets.flatMap(gadget => Object.entries(gadget.bindings) + .filter(([, edge]) => edge.target === record.id) + .map(([name, edge]) => ({ + gadgetId: gadget.id, name, ...(edge.pending ? {chatId: edge.pending.chatId} : {}), + }))), + hooks: hooks.filter(hook => hook.gatekeeperId === record.id) + .map(hook => access === "full" ? hook : withholdHookContent(hook)), + })); + }, gatekeepers => gatekeepers.some(gatekeeper => gatekeeper.hooks.some(hookHoldsContent))); + } + + // Each entry names the fields its type declares, since a stored record also holds the observer + // id, the handle connections know the collaborator by. A profile the collaborator table lacks + // is made of the id alone, as a share link's creator's is. Choices come in connection id + // order, as an object lists integer keys, and one outlives its connection, which then gives it + // no vendor. The listing is read to its end before the connections are read. + async listObservers(): Promise { + return this.#call("listObservers", {}, async () => { + let storage = this.#impl.storage; + let sharing = await this.#impl.getSharingManager(); + return [...storage.observers.list()].map((record): AdminObserver => ({ + profile: sharing.getCreatorProfile(record.profileId) ?? + {type: "user", id: record.profileId, name: record.profileId}, + accountChoices: Object.entries(record.accountChoices).map(([id, accountId]) => ({ + gatekeeperId: Number(id), + accountId, + vendorId: gatekeeperVendorId(storage.gatekeepers.get(Number(id))), + })), + })); + }); + } + + // Read from storage alone (see OverseerImpl.listObserverRequirements()). The record keeps the + // grant by name: "workspace", or the gadget's id. + async listObserverRequirements(grant: CollaboratorGrant): Promise { + return this.#call("listObserverRequirements", + grant === "workspace" ? {grant} : {gadgetId: grant.gadgetId}, + () => this.#impl.listObserverRequirements(grant)); + } + + // The owner's account is asked only when the owner created a link, as for a member's session. + async listShareLinks(): Promise { + return this.#call("listShareLinks", {}, + () => this.#impl.shareLinkInfos(() => this.#ownerProfile())); + } + + // The stored rules as they are, whether or not any can apply now (see AdminAutoApproval). What + // each connection could auto-approve is not read: that would ask every bound connection. + async listAutoApprovals(): Promise { + return this.#call("listAutoApprovals", {}, + () => [...this.#impl.storage.autoApproveTags.list()].map( + ({gatekeeperId, actionKind, enabledBy}) => ({gatekeeperId, actionKind, enabledBy}))); + } + + async listBlueprints(): Promise { + return this.#call("listBlueprints", {}, + access => this.#impl.blueprintSummaries() + .map(blueprint => access === "full" ? blueprint : withholdBlueprintContent(blueprint)), + blueprints => blueprints.some(blueprintHoldsContent)); + } + + async getDiagnostics(): Promise { + return this.#call("getDiagnostics", {}, async () => { + let impl = this.#impl; + let storage = impl.storage; + let hooks = [...storage.boundHooks.list()]; + let nextAlarm = await impl.ctx.storage.getAlarm(); + return { + storageVersion: storage.version.get(), + instanceStartedAt: new Date(impl.streamGeneration), + sessions: impl.liveSessionCounts(), + activeAgentChatIds: [...storage.activeAgents.list()].map(record => record.chatId), + pendingAgentCalls: [...storage.pendingAgentCalls.list()].length, + pendingActions: [...storage.actions.byHistoryFilter.get("pending")].length, + chats: [...storage.chatMeta.list()].length, + hooks: {total: hooks.length, enabled: hooks.filter(hook => hook.enabled).length}, + observers: [...storage.observers.list()].length, + storageBytes: impl.ctx.storage.sql.databaseSize, + present: impl.presenceSnapshot(), + nextAlarm: nextAlarm === null ? undefined : new Date(nextAlarm), + ownerRegistrationPending: storage.ownerRegistrationPending.get(), + }; + }); + } +} + // Capability representing one gadget workpiece, handed to "build"-role sessions via // Overseer.createGadget()/getGadget(). // diff --git a/packages/workshop-backend/src/storage-schema/admin-settings-storage.ts b/packages/workshop-backend/src/storage-schema/admin-settings-storage.ts index 666d83d74d..6b1b1faff8 100644 --- a/packages/workshop-backend/src/storage-schema/admin-settings-storage.ts +++ b/packages/workshop-backend/src/storage-schema/admin-settings-storage.ts @@ -9,9 +9,10 @@ import { collection, createTypedStorage } from "@gadgets/typed-storage"; import { - DEFAULT_ADMIN_AUDIT_RETENTION_DAYS, DEFAULT_BANNER_COLOR, + DEFAULT_ADMIN_AUDIT_RETENTION_DAYS, DEFAULT_ADMIN_WORKSPACE_ACCESS, DEFAULT_BANNER_COLOR, DEFAULT_UPDATE_CHECK_INTERVAL_MINUTES, DEFAULT_UPDATE_MINIMUM_AGE_HOURS, DEFAULT_UPDATE_NOTICE_SNOOZE_HOURS, + type AdminWorkspaceAccess, type AiModelProvider, type AmbientGatekeeperMode, type BannerConfig, type BlueprintOutput, type BlueprintPublicInfo, type GatewayModel, type GatewayModelMode, type GatewayModelSettings, type ReasoningLevel, @@ -145,6 +146,10 @@ export type AdminConfig = { updateMinimumAgeHours: number; /** How many hours a dismissed update notice stays hidden in the browser that dismissed it. */ updateNoticeSnoozeHours: number; + /** How much of a workspace admins chose to let inspectWorkspace() show. ADMIN_WORKSPACE_ACCESS_MAX + * holds it in wherever it is enforced (adminAccessPolicy() in admin-access.ts), so a compromised + * admin session can never raise access past what the deployment allows. */ + adminWorkspaceAccess: AdminWorkspaceAccess; /** How many days admins chose to keep admin audit records. Held at or above * ADMIN_AUDIT_RETENTION_MIN_DAYS wherever it is enforced. */ adminAuditRetentionDays: number; @@ -202,6 +207,7 @@ export const DEFAULT_ADMIN_CONFIG: AdminConfig = { updateCheckIntervalMinutes: DEFAULT_UPDATE_CHECK_INTERVAL_MINUTES, updateMinimumAgeHours: DEFAULT_UPDATE_MINIMUM_AGE_HOURS, updateNoticeSnoozeHours: DEFAULT_UPDATE_NOTICE_SNOOZE_HOURS, + adminWorkspaceAccess: DEFAULT_ADMIN_WORKSPACE_ACCESS, adminAuditRetentionDays: DEFAULT_ADMIN_AUDIT_RETENTION_DAYS, }; diff --git a/packages/workshop-backend/src/storage-schema/overseer-storage.ts b/packages/workshop-backend/src/storage-schema/overseer-storage.ts index 574f102635..3b98efa6d6 100644 --- a/packages/workshop-backend/src/storage-schema/overseer-storage.ts +++ b/packages/workshop-backend/src/storage-schema/overseer-storage.ts @@ -55,6 +55,14 @@ export type GatekeeperRecord = { /** Records how this gatekeeper was originally created, enabling blueprint metadata derivation. */ creationSpec?: GatekeeperCreationSpec; + /** + * Who created the connection: the member whose session made it, or the initiator of the agent + * turn that created its resource. Absent on an ambient capsule, which no person creates, and on + * a record whose creator was not recorded: one written before creators were recorded, or a + * resource created in a turn that has no record. + */ + createdBy?: AiChatAuthorInfo; + /** * Present on agent spawners only: the bindings given to each agent the spawner starts, keyed * by binding name. Seeded from the AgentSpawnerConfig.env the spawner was created with, then @@ -1441,6 +1449,19 @@ export function makeOverseerStorage(storage: DurableObjectStorage) { // project-specific part of the agent's system prompt, so nobody without that prompt can // probe a shared prompt cache for it (see runAgentPass). promptCacheSalt: undefined, + + // When the workspace was initialised; unset for one initialised before this was recorded, + // until the directory feed backfills it from the owner's record (OverseerImpl.syncDirectory). + createdAt: undefined, + + // The revision of the title the deployment directory is sent, bumped with every title + // write and at initialisation. Time-based (at least the epoch ms of the bump), so a + // workspace reinitialised under the same id outranks what its earlier life reported. + titleRev: 0, + + // The highest titleRev the directory has acknowledged; -1 until it acknowledges one, and + // again once the workspace leaves the directory (OverseerImpl.removeFromDirectory). + directoryRev: -1, }, collections: { diff --git a/packages/workshop-backend/src/storage-schema/user-storage.ts b/packages/workshop-backend/src/storage-schema/user-storage.ts index 235f540412..7e4f559003 100644 --- a/packages/workshop-backend/src/storage-schema/user-storage.ts +++ b/packages/workshop-backend/src/storage-schema/user-storage.ts @@ -340,6 +340,11 @@ export function makeUserStorage(storage: DurableObjectStorage) { // (-1 = never, which also lazily backfills users created before the // directory existed). See #syncDirectory(). directoryRev: -1, + // When the account was created (see #syncDirectory()); null for one created before this + // was recorded. + createdAt: null, + // Epoch ms of the session start the directory last took (0 = none). See #syncDirectory(). + directoryLoginAt: 0, } }); } diff --git a/packages/workshop-backend/src/user-directory.ts b/packages/workshop-backend/src/user-directory.ts index fe2b5015e0..4ec901d61d 100644 --- a/packages/workshop-backend/src/user-directory.ts +++ b/packages/workshop-backend/src/user-directory.ts @@ -1,15 +1,120 @@ -import type { UserDirectoryRecord } from "@gadgets/workshop-shared/api"; +import { + MAX_DIRECTORY_TEXT_LENGTH, type AdminUserRecord, type AdminWorkspacePage, + type AdminWorkspaceRecord, type UserDirectoryRecord, +} from "@gadgets/workshop-shared/api"; import { DurableObject } from "cloudflare:workers"; const SEARCH_RESULT_LIMIT = 10; // Every authenticated user can reach this one DO, so a search bounds what it is asked to scan. const MAX_QUERY_LENGTH = 1000; const MAX_EXCLUDE_IDS = 1000; +// A page stops early once its text would pass this many characters, which keeps it far below the +// RPC message limit whatever the directory holds, since ids are stored whole. +const MAX_PAGE_TEXT_LENGTH = 1_000_000; + +/** How often a user DO or an Overseer reports activity to the directory at most: an hour, the + * precision of AdminUserRecord.lastLogin and AdminWorkspaceRecord.lastActive. */ +export const DIRECTORY_ACTIVITY_INTERVAL_MS = 60 * 60_000; + +/** What a user DO reports of its account beside its profile (see syncUser()). */ +export type UserDirectoryActivity = { + /** When the account was created, if recorded; the first known value is kept. */ + created?: Date; + /** When the session being reported started; the stored value only moves forward. */ + lastLogin?: Date; +}; + +/** A workspace as its Overseer reports it to the directory (see syncWorkspace()). */ +export type WorkspaceDirectoryEntry = { + /** The workspace (Overseer Durable Object) id. */ + id: string; + /** Its title as of the sync's revision, whole; the directory keeps its first + * MAX_DIRECTORY_TEXT_LENGTH characters. */ + title: string; + /** The owner's canonical user id. */ + ownerId: string; + /** When it was initialised, if known. */ + created?: Date; + /** When it was last active, if this sync reports activity; the stored value only moves forward. */ + lastActive?: Date; +}; + +/** One page of UserDirectoryDurableObject.listUsers(): AdminUserRecord without `isAdmin`, which + * AdminApiImpl adds from ADMINS. */ +export type UserDirectoryPage = { users: Omit[]; nextCursor?: string }; + +// A display name or title as the directory keeps and lists it: its first MAX_DIRECTORY_TEXT_LENGTH +// characters, one fewer rather than half of a surrogate pair. +function clamp(text: string): string { + if (text.length <= MAX_DIRECTORY_TEXT_LENGTH) return text; + const last = text.charCodeAt(MAX_DIRECTORY_TEXT_LENGTH - 1); + const end = last >= 0xd800 && last <= 0xdbff + ? MAX_DIRECTORY_TEXT_LENGTH - 1 : MAX_DIRECTORY_TEXT_LENGTH; + return text.slice(0, end); +} + +// Name and id are joined by a newline in search_text, so a needle containing one could match +// across the two fields. +function validateQuery(query: string): void { + if (query.length > MAX_QUERY_LENGTH || /[\r\n]/.test(query)) { + throw new Error( + `Search query must be at most ${MAX_QUERY_LENGTH} characters with no line breaks.`); + } +} + +function validateLimit(limit: number): void { + if (!Number.isSafeInteger(limit) || limit < 1) { + throw new TypeError("A page limit must be a positive whole number."); + } +} + +// A time as the directory stores it: epoch ms, NULL for unknown. +function epochMs(date: Date | undefined): number | null { + return date ? date.getTime() : null; +} + +// Collects rows into one page in the order a query yields them: at most `limit` (the query asks +// for one more, to learn whether another page follows), and fewer once their text would pass +// MAX_PAGE_TEXT_LENGTH, though never none while any remain. Rows are read one at a time, so none +// past the first one left out is ever read. +function collectPage( + rows: Iterable, limit: number, toItem: (row: Row) => Item, + textLength: (item: Item) => number): { items: Item[]; nextCursor?: string } { + const items: Item[] = []; + let text = 0; + for (const row of rows) { + const item = toItem(row); + const length = textLength(item); + if (items.length >= limit || (items.length > 0 && text + length > MAX_PAGE_TEXT_LENGTH)) { + return { items, nextCursor: items[items.length - 1].id }; + } + items.push(item); + text += length; + } + return { items }; +} /** - * Deployment-wide directory of user profiles, so a user can find collaborators - * by name or id. Each user DO mirrors updates to its own profile here - * (`UserDurableObject.#syncDirectory`). + * Deployment-wide directory of users and workspaces: users find collaborators in it + * (AuthenticatedApi.searchUsers()), and admins list the deployment's users and workspaces + * (AdminApi.listUsers() and listWorkspaces()). Each user DO mirrors its profile and activity here + * (`UserDurableObject.#syncDirectory`), each Overseer mirrors its workspace + * (`OverseerImpl.syncDirectory`) and takes it out on deletion (`removeFromDirectory`), and + * AdminApiImpl removes a workspace it finds gone. Durable Objects cannot be enumerated, so the + * directory fills as it is used: an account is listed once it has started a session since the + * directory began recording accounts for user search (one recorded before the admin listings is + * listed without `lastLogin` until its next session), a workspace when it is next opened. + * + * The admin listings live here rather than in a store of their own because this is already the + * deployment-wide singleton (always `getByName("")`), every user DO already feeds it through a + * revision protocol that backfills lazily, and it is already migrated and bound in tests: a second + * store would duplicate the users table, the sync protocol and the per-session writes. + * + * It is a finding aid, not a record: display names and titles take any length, so they are kept + * and listed as their first MAX_DIRECTORY_TEXT_LENGTH characters, as boundAgentCatalog() clamps a + * catalog (the admin audit log marks its cuts instead, since a record must not pass a fragment off + * as the whole). Ids are identities and stay whole, and a username has no length limit, so a page + * is bounded by its text as well as by its count. */ export class UserDirectoryDurableObject extends DurableObject { constructor(ctx: DurableObjectState, env: Cloudflare.Env) { @@ -20,35 +125,108 @@ export class UserDirectoryDurableObject extends DurableObject { search_text TEXT NOT NULL, rev INTEGER NOT NULL ) STRICT`); + // Times are epoch ms; NULL means unknown. + ctx.storage.sql.exec(`CREATE TABLE IF NOT EXISTS user_activity ( + id TEXT PRIMARY KEY, + created INTEGER, + last_login INTEGER + ) STRICT`); + ctx.storage.sql.exec(`CREATE TABLE IF NOT EXISTS workspaces ( + id TEXT PRIMARY KEY, + title TEXT NOT NULL, + owner_id TEXT NOT NULL, + search_text TEXT NOT NULL, + created INTEGER, + last_active INTEGER, + rev INTEGER NOT NULL + ) STRICT`); + ctx.storage.sql.exec( + "CREATE INDEX IF NOT EXISTS workspaces_by_owner ON workspaces (owner_id, id)"); } /** - * Insert or update one user's record. `rev` is the user DO's profile revision: a record already - * at a higher revision is left alone, so syncs that arrive out of order still converge on the - * newest profile. + * Insert or update one user's record: the name clamped to MAX_DIRECTORY_TEXT_LENGTH, the search + * text built from the clamped name and the id's first MAX_DIRECTORY_TEXT_LENGTH characters, the + * id whole. `rev` is the user DO's profile revision: a record already at a higher revision is + * left alone, so syncs that arrive out of order still converge on the newest profile. + * `activity` is merged whatever `rev` is, as UserDirectoryActivity says. */ - syncUser(record: UserDirectoryRecord, rev: number): void { + syncUser(record: UserDirectoryRecord, rev: number, activity?: UserDirectoryActivity): void { + const name = clamp(record.name); this.ctx.storage.sql.exec( `INSERT INTO users (id, name, search_text, rev) VALUES (?, ?, ?, ?) ON CONFLICT (id) DO UPDATE SET name = excluded.name, search_text = excluded.search_text, rev = excluded.rev WHERE excluded.rev > users.rev`, - record.id, record.name, `${record.id}\n${record.name}`.toLowerCase(), rev); + record.id, name, `${clamp(record.id)}\n${name}`.toLowerCase(), rev); + if (activity?.created || activity?.lastLogin) { + this.ctx.storage.sql.exec( + `INSERT INTO user_activity (id, created, last_login) VALUES (?, ?, ?) + ON CONFLICT (id) DO UPDATE SET + created = coalesce(user_activity.created, excluded.created), + last_login = max(coalesce(user_activity.last_login, excluded.last_login), + coalesce(excluded.last_login, user_activity.last_login))`, + record.id, epochMs(activity.created), epochMs(activity.lastLogin)); + } + } + + /** + * Insert or update one workspace, its title clamped and its search text bounded as syncUser()'s + * are. `rev` is the Overseer's time-based directory revision, so a workspace reinitialised under + * the same id outranks what its earlier life reported: a higher `rev` replaces the title, the + * owner and (when the sync carries one) `created`; a lower or equal one changes none of them but + * fills a `created` the row lacks. `lastActive` only moves forward. + */ + syncWorkspace(entry: WorkspaceDirectoryEntry, rev: number): void { + const title = clamp(entry.title); + // Every expression in the SET list reads the row as it was before the update. + this.ctx.storage.sql.exec( + `INSERT INTO workspaces (id, title, owner_id, search_text, created, last_active, rev) + VALUES (?, ?, ?, ?, ?, ?, ?) + ON CONFLICT (id) DO UPDATE SET + title = iif(excluded.rev > workspaces.rev, excluded.title, workspaces.title), + owner_id = iif(excluded.rev > workspaces.rev, excluded.owner_id, workspaces.owner_id), + search_text = iif(excluded.rev > workspaces.rev, excluded.search_text, + workspaces.search_text), + created = iif(excluded.rev > workspaces.rev, + coalesce(excluded.created, workspaces.created), + coalesce(workspaces.created, excluded.created)), + last_active = max(coalesce(workspaces.last_active, excluded.last_active), + coalesce(excluded.last_active, workspaces.last_active)), + rev = max(workspaces.rev, excluded.rev)`, + entry.id, title, entry.ownerId, + `${entry.id}\n${title}\n${clamp(entry.ownerId)}`.toLowerCase(), + epochMs(entry.created), epochMs(entry.lastActive), rev); + } + + /** Forget a workspace: its Overseer, deleting it, or AdminApiImpl on finding it gone. */ + removeWorkspace(id: string): void { + this.ctx.storage.sql.exec("DELETE FROM workspaces WHERE id = ?", id); + } + + /** + * The title (clamped) and owner the directory lists for a workspace, or null when it lists none: + * what AdminApiImpl records, the title cut, for an open refused before reaching the workspace. + * Rejects an id over 1000 characters (workspace ids are 64). + */ + workspaceListing(id: string): { title: string; ownerId: string } | null { + if (id.length > MAX_QUERY_LENGTH) { + throw new Error(`Workspace id must be at most ${MAX_QUERY_LENGTH} characters.`); + } + const row = this.ctx.storage.sql.exec<{ title: string; owner_id: string }>( + "SELECT title, owner_id FROM workspaces WHERE id = ?", id).toArray()[0]; + return row ? { title: clamp(row.title), ownerId: row.owner_id } : null; } /** * Case-insensitive substring search over name and id, excluding every id in `excludeIds`. * The id is indexed first so an exact id starts at position one; ties sort by id then name. * Rejects a query over `MAX_QUERY_LENGTH` characters or containing a line break, and more than - * `MAX_EXCLUDE_IDS` distinct exclusions. + * `MAX_EXCLUDE_IDS` distinct exclusions. Names are clamped as every read clamps them, since a + * row stored before the directory clamped may hold more. */ searchUsers(query: string, excludeIds: string[]): UserDirectoryRecord[] { - // Name and id are joined by a newline in search_text, so a needle containing one could match - // across the two fields. - if (query.length > MAX_QUERY_LENGTH || /[\r\n]/.test(query)) { - throw new Error( - `Search query must be at most ${MAX_QUERY_LENGTH} characters with no line breaks.`); - } + validateQuery(query); const excluded = [...new Set(excludeIds)]; if (excluded.length > MAX_EXCLUDE_IDS) { throw new Error(`At most ${MAX_EXCLUDE_IDS} users can be excluded from a search.`); @@ -61,6 +239,94 @@ export class UserDirectoryDurableObject extends DurableObject { ORDER BY instr(search_text, ?), id, name COLLATE NOCASE LIMIT ${SEARCH_RESULT_LIMIT}`, JSON.stringify(excluded), needle, needle, - ).toArray(); + ).toArray().map(({ id, name }) => ({ id, name: clamp(name) })); + } + + /** + * Users in id order after `cursor` (the previous page's last id, of any length), matching + * `query` as searchUsers() matches (blank matches all), with their activity and clamped names: + * at most `limit`, and fewer once the page's ids and names would pass a million characters + * (never none while any remain). Rejects a `query` over 1000 characters or with a line break; + * `limit` must be a positive whole number (AdminApiImpl bounds it). + */ + listUsers(options: { query?: string; cursor?: string; limit: number }): UserDirectoryPage { + const { query = "", cursor, limit } = options; + validateQuery(query); + validateLimit(limit); + const needle = query.trim().toLowerCase(); + const conditions: string[] = []; + const bindings: string[] = []; + if (cursor !== undefined) { + conditions.push("users.id > ?"); + bindings.push(cursor); + } + if (needle !== "") { + conditions.push("instr(users.search_text, ?) > 0"); + bindings.push(needle); + } + const rows = this.ctx.storage.sql.exec<{ + id: string; name: string; created: number | null; last_login: number | null; + }>( + `SELECT users.id AS id, users.name AS name, user_activity.created AS created, + user_activity.last_login AS last_login + FROM users LEFT JOIN user_activity ON user_activity.id = users.id + ${conditions.length > 0 ? `WHERE ${conditions.join(" AND ")}` : ""} + ORDER BY users.id LIMIT ?`, + ...bindings, limit + 1); + const { items, nextCursor } = collectPage(rows, limit, row => ({ + id: row.id, + name: clamp(row.name), + ...row.created !== null && { created: new Date(row.created) }, + ...row.last_login !== null && { lastLogin: new Date(row.last_login) }, + }), user => user.id.length + user.name.length); + return nextCursor === undefined ? { users: items } : { users: items, nextCursor }; + } + + /** + * Workspaces in id order after `cursor`, of `owner` (matched exactly, any length) if given, + * matching `query` against id, title and owner id, each owner's display name joined from the + * users table and clamped (their id, clamped, while it has none); paged and bounded as + * listUsers() is, counting ids, titles and owners. Rejects what listUsers() rejects. + */ + listWorkspaces(options: { owner?: string; query?: string; cursor?: string; limit: number }) + : AdminWorkspacePage { + const { owner, query = "", cursor, limit } = options; + validateQuery(query); + validateLimit(limit); + const needle = query.trim().toLowerCase(); + const conditions: string[] = []; + const bindings: string[] = []; + if (owner !== undefined) { + conditions.push("workspaces.owner_id = ?"); + bindings.push(owner); + } + if (cursor !== undefined) { + conditions.push("workspaces.id > ?"); + bindings.push(cursor); + } + if (needle !== "") { + conditions.push("instr(workspaces.search_text, ?) > 0"); + bindings.push(needle); + } + const rows = this.ctx.storage.sql.exec<{ + id: string; title: string; owner_id: string; owner_name: string | null; + created: number | null; last_active: number | null; + }>( + `SELECT workspaces.id AS id, workspaces.title AS title, workspaces.owner_id AS owner_id, + users.name AS owner_name, workspaces.created AS created, + workspaces.last_active AS last_active + FROM workspaces LEFT JOIN users ON users.id = workspaces.owner_id + ${conditions.length > 0 ? `WHERE ${conditions.join(" AND ")}` : ""} + ORDER BY workspaces.id LIMIT ?`, + ...bindings, limit + 1); + const { items, nextCursor } = collectPage(rows, limit, (row): AdminWorkspaceRecord => ({ + id: row.id, + title: clamp(row.title), + owner: { id: row.owner_id, name: clamp(row.owner_name ?? row.owner_id) }, + ...row.created !== null && { created: new Date(row.created) }, + ...row.last_active !== null && { lastActive: new Date(row.last_active) }, + }), workspace => workspace.id.length + workspace.title.length + workspace.owner.id.length + + workspace.owner.name.length); + return nextCursor === undefined ? { workspaces: items } : { workspaces: items, nextCursor }; } } diff --git a/packages/workshop-backend/src/user.ts b/packages/workshop-backend/src/user.ts index 5c38eec3c0..6a6238b1d3 100644 --- a/packages/workshop-backend/src/user.ts +++ b/packages/workshop-backend/src/user.ts @@ -1,6 +1,6 @@ import { RpcStub } from "capnweb"; import type { ChatReadState, ChatTimelineHead, UnreadStateScope, UnreadStateSubscriber, UserUnreadState, UnreadThread, UserUnreadSummary, WorkspaceUnreadSummary } from "@gadgets/workshop-shared/api"; -import { WorkspaceMetadataWithTimestamps, AiChatAuthorInfo, AiModelConfig, RedactedAiModelConfig, ModelKind, modelKind, CollaboratorRole, ConnectedAccountsSubscriber, ConnectedAccountsFilter, GatekeeperVendorFilter, WorkspaceMetadata, BlueprintMetadata, BlueprintLibrarySummary, BlueprintSource, BlueprintUserSummary, BLUEPRINT_SCREENSHOT_R2_PREFIX, GatekeeperVendorInfo, OutputSummary, ListOutputsResult, AUTH_ERROR_CODES, createAuthError, ConnectFlowStart, validateCommitEmail, NotificationSubscriber, NotificationMute, UserNotification, type DeviceSessionHandoffStart, type WebPushSubscription } from '@gadgets/workshop-shared/api'; +import { WorkspaceMetadataWithTimestamps, AiChatAuthorInfo, AiModelConfig, RedactedAiModelConfig, ModelKind, modelKind, CollaboratorRole, ConnectedAccountsSubscriber, ConnectedAccountsFilter, GatekeeperVendorFilter, BlueprintMetadata, BlueprintLibrarySummary, BlueprintSource, BlueprintUserSummary, BLUEPRINT_SCREENSHOT_R2_PREFIX, GatekeeperVendorInfo, OutputSummary, ListOutputsResult, AUTH_ERROR_CODES, createAuthError, ConnectFlowStart, validateCommitEmail, NotificationSubscriber, NotificationMute, UserNotification, type DeviceSessionHandoffStart, type WebPushSubscription } from '@gadgets/workshop-shared/api'; import { ActionDescription, Gatekeeper, GatekeeperUser, GatekeeperUserVerifier, GatekeeperVendor, AccountDescription, VendorDescription, GatekeeperConnectCallback, ConnectHandoff, SupportedResource, ResourceConfiguratorFrame, AppUiContext, GatekeeperUiFrame, GatekeeperUserSelection, GatekeeperUserSelectionExchange, RedeemedGatekeeperUserSelection } from "@gadgets/workshop-shared/gatekeeper"; import { shouldAutoProvisionAccount, ambientGatekeeperMode } from "./provisioning-policy.js"; import { CloudflareGatekeeperUser } from "@gadgets/workshop-shared/cloudflare-gatekeeper"; @@ -32,6 +32,7 @@ import { DEVICE_HANDOFF_LIFETIME_MS, isDeviceHandoffState, sealDeviceSessionHandoff, type DeviceSessionCredential, type DeviceSessionHandoff, } from "./auth/device-session-handoff.js"; +import { DIRECTORY_ACTIVITY_INTERVAL_MS, type UserDirectoryActivity } from "./user-directory.js"; const logger = createWorkshopLogger("workshop.user"); @@ -375,21 +376,35 @@ export class UserDurableObject extends DurableObject { } } - // Mirrors the profile into the deployment-wide user directory. Best-effort - // and does not block the caller. The `syncUser` call opens this DO's input - // gate, so a rename can start a second sync while one is in flight, and the - // two can reach the directory in either order. Each sync carries its - // `profileRev` and the directory keeps the highest, so no ordering is needed - // here. The revision counter only increases, and a failed sync leaves it - // behind so the next authentication retries. - #syncDirectory(): void { + // Mirrors the profile into the deployment-wide user directory, with when the + // account was created and, when `session` says a session is starting, its + // start, at most once per DIRECTORY_ACTIVITY_INTERVAL_MS. Best-effort and + // does not block the caller. The `syncUser` call opens this DO's input gate, + // so a rename can start a second sync while one is in flight, and the two can + // reach the directory in either order. Each sync carries its `profileRev` and + // the directory keeps the highest, and it keeps the latest login time it is + // sent: the profile converges by revision and the login time by maximum, so + // neither needs ordering here. `directoryRev` and `directoryLoginAt` only + // advance, and a failed sync leaves both behind so the next session retries. + #syncDirectory(session: boolean): void { const rev = this.storage.profileRev.get(); - if (rev === this.storage.directoryRev.get()) return; + const now = Date.now(); + const reportLogin = + session && now - this.storage.directoryLoginAt.get() >= DIRECTORY_ACTIVITY_INTERVAL_MS; + if (rev === this.storage.directoryRev.get() && !reportLogin) return; const profile = this.storage.profile.get(); + const createdAt = this.storage.createdAt.get(); + const activity: UserDirectoryActivity = { + ...createdAt && { created: createdAt }, + ...reportLogin && { lastLogin: new Date(now) }, + }; this.ctx.exports.UserDirectoryDurableObject.getByName("") - .syncUser({ id: profile.id, name: profile.name }, rev) + .syncUser({ id: profile.id, name: profile.name }, rev, activity) .then(() => { if (rev > this.storage.directoryRev.get()) this.storage.directoryRev.put(rev); + if (reportLogin && now > this.storage.directoryLoginAt.get()) { + this.storage.directoryLoginAt.put(now); + } }, (error: unknown) => { logger.warn("failed to sync user directory record", { event: "user.directory.sync.failed", @@ -422,7 +437,7 @@ export class UserDurableObject extends DurableObject { if (session.pendingUntil) { throw createAuthError(AUTH_ERROR_CODES.invalidSessionToken); } - this.#syncDirectory(); + this.#syncDirectory(true); } /** @@ -438,13 +453,14 @@ export class UserDurableObject extends DurableObject { } // Create on first use. this.storage.created.put(true); + this.storage.createdAt.put(new Date()); this.storage.profile.put({ type: "user", name: email.split("@")[0], id: email, }); } - this.#syncDirectory(); + this.#syncDirectory(true); return isNew; } @@ -577,6 +593,7 @@ export class UserDurableObject extends DurableObject { } this.storage.created.put(true); + this.storage.createdAt.put(new Date()); this.storage.profile.put({ type: "user", name: displayName, @@ -607,6 +624,7 @@ export class UserDurableObject extends DurableObject { if (!this.storage.created.get()) { if (!allowCreate) return null; this.storage.created.put(true); + this.storage.createdAt.put(new Date()); this.storage.profile.put({ type: "user", name: email.split("@")[0], @@ -1308,7 +1326,7 @@ export class UserDurableObject extends DurableObject { profile.name = name; this.storage.profile.put(profile); this.storage.profileRev.put(this.storage.profileRev.get() + 1); - this.#syncDirectory(); + this.#syncDirectory(false); } async setOwnCommitEmail(email: string | null): Promise { @@ -1624,7 +1642,9 @@ export class UserDurableObject extends DurableObject { this.storage.gadgets.put(record); } - async getWorkspace(id: string): Promise { + /** The stored record of one of the user's workspaces, whose `created` the Overseer reads once to + * date a workspace older than the directory. */ + async getWorkspace(id: string): Promise { return this.storage.gadgets.get(id) || null; } diff --git a/packages/workshop-backend/vitest.config.ts b/packages/workshop-backend/vitest.config.ts index 61c666fa17..dd1bdfdf2f 100644 --- a/packages/workshop-backend/vitest.config.ts +++ b/packages/workshop-backend/vitest.config.ts @@ -89,6 +89,8 @@ export default defineConfig({ // @gadgets/scripts/property-runs) through `nodejs_compat`, which copies string bindings there. bindings: { PUBLIC_BASE_URL: 'https://workshop.example/', + // The deployment admin of the admin-view suites. + ADMINS: ['deployment-admin@example.test'], ...process.env.FAST_CHECK_SEED ? { FAST_CHECK_SEED: process.env.FAST_CHECK_SEED } : {}, }, // The overseer loads gadget code through this, so a test can run a real gadget facet. @@ -105,7 +107,7 @@ export default defineConfig({ TEST_USER_DIRECTORY: { className: 'UserDirectoryDurableObject', useSQLite: true }, // Likewise only reached through `ctx.exports`, by the overseer deleting a blueprint. TEST_ADMIN_SETTINGS: { className: 'AdminSettings', useSQLite: true }, - // AdminSettings also reaches it through `ctx.exports`. + // The Overseer and AdminSettings also reach it through `ctx.exports`. TEST_ADMIN_AUDIT: { className: 'AdminAuditDurableObject', useSQLite: true }, TEST_GATEKEEPER_USER_PICKER: { className: 'GatekeeperUserPickerTestHooks', useSQLite: true }, }, diff --git a/packages/workshop-frontend/src/AdminPage.tsx b/packages/workshop-frontend/src/AdminPage.tsx index b33e9425ad..1055a1eb60 100644 --- a/packages/workshop-frontend/src/AdminPage.tsx +++ b/packages/workshop-frontend/src/AdminPage.tsx @@ -105,8 +105,8 @@ export default function AdminPage() { // saved only once that later one has applied. const [reloadUpdateStatus] = useState(() => latestReads(setUpdateStatus)) - // The audit retention, null while the backend is older than the admin audit log and has no Audit - // tab. + // The admin access level and audit retention, null while the backend is older than the admin + // audit log and has no Audit tab. const [auditSettings, setAuditSettings] = useState(null) // Only the latest re-read applies (see latestReads). const [reloadAuditSettings] = useState(() => @@ -510,7 +510,7 @@ export default function AdminPage() { ) )} - {/* Admin audit log and its retention */} + {/* Admin audit log, workspace access and retention */} {activeTab === 'audit' && auditSettings !== null && ( * The tab to show for the one requested in the URL. The Updates tab exists only for a deployment * the deploy flow installed, which is known once the update status has loaded: until then a request * for it stands, so a link to it does not flash another tab first. The Audit tab exists only while - * the backend's settings carry the audit retention (a backend older than the admin audit log + * the backend's settings carry the admin access settings (a backend older than the admin audit log * sends none); the settings load before any tab shows, so its availability is never pending. */ export const resolveAdminTab = ( diff --git a/packages/workshop-frontend/src/features/admin-audit/AdminAuditPanel.test.tsx b/packages/workshop-frontend/src/features/admin-audit/AdminAuditPanel.test.tsx index 1eb1c906cf..c78d6c9f70 100644 --- a/packages/workshop-frontend/src/features/admin-audit/AdminAuditPanel.test.tsx +++ b/packages/workshop-frontend/src/features/admin-audit/AdminAuditPanel.test.tsx @@ -7,8 +7,10 @@ import { afterAll, afterEach, beforeEach, describe, expect, it, vi } from 'vites import type { RpcStub } from 'capnweb' import { MAX_ADMIN_AUDIT_RETENTION_DAYS } from '@gadgets/workshop-shared/api' import type { + AdminAccessRecord, AdminApi, AdminAuditPage, + AdminAuditRecord, AdminAuditStats, AdminSettingRecord, } from '@gadgets/workshop-shared/api' @@ -49,8 +51,10 @@ function deferred() { const RETENTION_LABEL = 'Keep audit records for (days)' const settingsWith = (patch: { + access?: Partial retention?: Partial } = {}): AuditSettings => ({ + adminWorkspaceAccess: { effective: 'metadata', chosen: 'metadata', max: 'full', ...patch.access }, adminAuditRetentionDays: { effective: 90, chosen: 90, min: 90, ...patch.retention }, }) @@ -66,11 +70,32 @@ const testStats = (patch: Partial = {}): AdminAuditStats => ({ { day: '2026-10-20', count: 0 }, { day: '2026-10-21', count: 2 }, ], - topAdmins: [{ admin: 'admin', count: 4 }, { admin: 'second-admin', count: 1 }], + topAdmins: [{ admin: 'admin', count: 4, contentReads: 2 }, { admin: 'second-admin', count: 1, contentReads: 0 }], + contentReads: 2, ...patch, }) const SETTING_AT = new Date('2026-10-21T08:00:00Z') +const ACCESS_AT = new Date('2026-10-21T09:15:00Z') + +const accessRecord = (patch: Partial = {}): AdminAccessRecord => ({ + kind: 'workspace', + at: ACCESS_AT, + admin: 'admin', + sessionId: 'session-1', + via: 'cli', + workspaceId: 'ws-titled', + workspaceOwner: 'owner@example.test', + workspaceTitle: 'Quarterly report', + method: 'getChat', + args: { chatId: 3 }, + reason: 'Ticket 42: agent stuck', + requested: 'full', + access: 'metadata', + exposed: 'metadata', + refused: 'ADMIN_CONTENT_WITHHELD', + ...patch, +}) const settingRecord = (patch: Partial = {}): AdminSettingRecord => ({ kind: 'setting', @@ -83,12 +108,13 @@ const settingRecord = (patch: Partial = {}): AdminSettingRec ...patch, }) -const testPage = (records = [settingRecord()], nextCursor?: string): AdminAuditPage => +const testPage = (records: AdminAuditRecord[] = [settingRecord()], nextCursor?: string): AdminAuditPage => nextCursor === undefined ? { records } : { records, nextCursor } const fakeAdmin = () => ({ getAdminAuditStats: vi.fn(async () => testStats()), listAdminAudit: vi.fn(async () => testPage()), + setAdminWorkspaceAccess: vi.fn(async () => {}), setAdminAuditRetentionDays: vi.fn(async () => {}), }) @@ -114,6 +140,35 @@ const button = (name: string) => { return element } +/** The options of the list `select` opens, which is left open. */ +const openOptions = async (select: HTMLElement) => { + if (select.getAttribute('aria-expanded') !== 'true') await click(select) + const list = document.getElementById(select.getAttribute('aria-controls') ?? '') + return [...list?.querySelectorAll('[role="option"]') ?? []] +} + +const enabledOptions = async (select: HTMLElement) => (await openOptions(select)) + .filter((option) => option.getAttribute('aria-disabled') !== 'true' && !option.hasAttribute('data-disabled')) + .map((option) => option.textContent) + +// By keyboard. The option turns Enter into a click it builds as a PointerEvent, which jsdom +// lacks, so the window has a stand-in for as long as the choice takes. +const choose = async (select: HTMLElement, label: string) => { + const option = (await openOptions(select)).find((element) => element.textContent === label) + if (!option) throw new Error(`no option ${label}`) + const view: { PointerEvent?: typeof MouseEvent } = window + const previous = view.PointerEvent + view.PointerEvent = MouseEvent + try { + await act(async () => { option.focus() }) + await act(async () => { + option.dispatchEvent(new KeyboardEvent('keydown', { key: 'Enter', bubbles: true, cancelable: true })) + }) + } finally { + view.PointerEvent = previous + } +} + // The column widths the log has stored, and its resize handles and the events that move them. const stored = () => JSON.parse(localStorage.getItem(AUDIT_COLUMN_WIDTHS_KEY) ?? 'null') as unknown const handle = (column: string) => button(`Resize the ${column} column`) @@ -175,6 +230,21 @@ describe('AdminAuditPanel', () => { return heading.parentElement } + // The select a card holds, by its label. + const selectIn = (title: string) => { + const element = card(title).querySelector('[role="combobox"]') + if (!element) throw new Error(`no select in ${title}`) + return element + } + const accessSelect = () => selectIn('Workspace access') + // The log's one select; its other comboboxes are text fields. + const sensitivitySelect = () => { + const element = [...card('Activity log').querySelectorAll('[role="combobox"]')] + .find((combobox) => !(combobox instanceof HTMLInputElement)) + if (!element) throw new Error('no sensitivity select') + return element + } + const retentionInput = () => { const labelElement = [...container.querySelectorAll('label')] .find((element) => element.textContent?.startsWith(RETENTION_LABEL)) @@ -204,12 +274,13 @@ describe('AdminAuditPanel', () => { act(() => root.unmount()) container.remove() document.body.innerHTML = '' - localStorage.clear() addToast.mockReset() + localStorage.clear() vi.restoreAllMocks() }) - // The log's widths, toggle, Time, Admin, Action, Changes (none: it takes what is left), Via. + // The log's widths: toggle, Time, Admin, Action (none: it takes what is left), Sensitivity, + // Target user. const widths = () => [...card('Activity log').querySelectorAll('col')].map((col) => col.style.width) describe('the summary', () => { @@ -233,14 +304,25 @@ describe('AdminAuditPanel', () => { it('shows the top admins, and filters the log by the one chosen', async () => { const { listAdminAudit, onFilterChange } = await render() - expect([...card('Most active admins').querySelectorAll('tbody tr')].map((row) => row.textContent)) - .toEqual(['admin4', 'second-admin1']) + expect([...card('Most active admins').querySelectorAll('tbody tr')] + .map((row) => [...row.querySelectorAll('td')].map((cell) => cell.textContent))) + .toEqual([['admin', '4', '2'], ['second-admin', '1', '0']]) await click(within(card('Most active admins'), 'Show only second-admin’s activity')) expect(onFilterChange).toHaveBeenCalledExactlyOnceWith({ admin: 'second-admin' }) expect(listAdminAudit).toHaveBeenLastCalledWith(pageQuery({ admin: 'second-admin' })) }) + it('counts the week’s content reads, in all and for each top admin, setting apart those with some', async () => { + await render() + + expect(card('Admin actions, last 7 days').textContent).toContain('2 content reads') + const rows = [...card('Most active admins').querySelectorAll('tbody tr')] + const [some, none] = rows.map((row) => row.lastElementChild!) + expect(some.className).toContain('text-kumo-danger') + expect(none.className).not.toContain('text-kumo-danger') + }) + it('says when the window holds no activity', async () => { const admin = fakeAdmin() admin.getAdminAuditStats.mockResolvedValue(testStats({ totalEvents: 0, topAdmins: [] })) @@ -283,7 +365,7 @@ describe('AdminAuditPanel', () => { const cells = [...logRows()[0].querySelectorAll('td')].map((cell) => cell.textContent) expect(cells.slice(1)).toEqual([ - formatFullTimestamp(SETTING_AT), 'second-admin', 'setSiteName', 'siteName', 'API', + formatFullTimestamp(SETTING_AT), 'second-admin', 'setSiteName', 'Setting change', '—Deployment-wide', ]) expect(logRows()[0].querySelector('time')?.getAttribute('dateTime')).toBe(SETTING_AT.toISOString()) }) @@ -299,19 +381,13 @@ describe('AdminAuditPanel', () => { expect(logRows().map((row) => row.classList.contains('bg-kumo-elevated'))).toEqual([false, true, false, true]) }) - it('summarizes several changes as the first and how many more', async () => { - const admin = fakeAdmin() - admin.listAdminAudit.mockResolvedValue(testPage([settingRecord({ - method: 'setBanner', - changes: [ - { setting: 'banner', key: 'message', after: 'Maintenance tonight' }, - { setting: 'banner', key: 'color', before: 'blue', after: 'red' }, - ], - })])) - - await render(settingsWith(), admin) + it('shows how a setting change came, and its action whole however long, from its row', async () => { + await render() - expect(logRows()[0].textContent).toContain('banner · message + 1 more') + const action = logRows()[0].querySelectorAll('td')[3].querySelector('span')! + expect(action.getAttribute('title')).toBe('setSiteName') + expect(action.className).toContain('truncate') + expect((await expand()).textContent).toContain('ViaAPI') }) it('shows a record’s changes, before and after, from a toggle', async () => { @@ -437,14 +513,31 @@ describe('AdminAuditPanel', () => { expect(listAdminAudit).toHaveBeenLastCalledWith(pageQuery({ method: 'setSignupsEnabled' })) }) - it('gives focus to the admin field once an admin is chosen in a row, whose button the filtered page replaces', async () => { - await render() + it('clears text typed and not applied in the admin and owner fields with every filter', async () => { + await render(settingsWith(), fakeAdmin(), { method: 'getChat' }) + const [admin, , owner] = card('Activity log').querySelectorAll('input[role="combobox"]') + await type(admin, 'alice') + await type(owner, 'bob') - await click(within(logRows()[0], 'Show only second-admin’s activity')) + await click(button('Clear filters')) - const field = card('Activity log').querySelector('input[role="combobox"]')! - expect(document.activeElement).toBe(field) - expect(field.value).toBe('second-admin') + expect(admin.value).toBe('') + expect(owner.value).toBe('') + }) + + it('gives focus to the field showing a name chosen in a row, whose button the filtered page replaces', async () => { + const admin = fakeAdmin() + admin.listAdminAudit.mockResolvedValue(testPage([accessRecord(), settingRecord()])) + await render(settingsWith(), admin) + const [adminField, , ownerField] = card('Activity log').querySelectorAll('input[role="combobox"]') + + await click(within(logRows()[1], 'Show only second-admin’s activity')) + expect(document.activeElement).toBe(adminField) + expect(adminField.value).toBe('second-admin') + + await click(within(logRows()[0], 'Show only calls into workspaces owner@example.test owns')) + expect(document.activeElement).toBe(ownerField) + expect(ownerField.value).toBe('owner@example.test') }) it('abandons what was typed on Escape, showing the admin filter still applied', async () => { @@ -452,12 +545,8 @@ describe('AdminAuditPanel', () => { const field = card('Activity log').querySelector('input[role="combobox"]')! await type(field, 'half-typed') - await act(async () => { - field.dispatchEvent(new KeyboardEvent('keydown', { key: 'Escape', bubbles: true, cancelable: true })) - }) - await act(async () => { - field.dispatchEvent(new KeyboardEvent('keydown', { key: 'Escape', bubbles: true, cancelable: true })) - }) + await key(field, { key: 'Escape' }) + await key(field, { key: 'Escape' }) expect(field.value).toBe('admin') expect(onFilterChange).not.toHaveBeenCalled() @@ -475,13 +564,9 @@ describe('AdminAuditPanel', () => { await click(button('Clear filters')) await type(field, 'adm') - await act(async () => { - field.dispatchEvent(new KeyboardEvent('keydown', { key: 'ArrowDown', bubbles: true, cancelable: true })) - }) + await key(field, { key: 'ArrowDown' }) expect(field.getAttribute('aria-activedescendant')).toBeTruthy() - await act(async () => { - field.dispatchEvent(new KeyboardEvent('keydown', { key: 'Enter', bubbles: true, cancelable: true })) - }) + await key(field, { key: 'Enter' }) expect(onFilterChange).toHaveBeenLastCalledWith({ admin: 'admin' }) expect(onFilterChange).not.toHaveBeenCalledWith({ admin: 'adm' }) }) @@ -624,6 +709,104 @@ describe('AdminAuditPanel', () => { expect(card('Activity log').querySelector('nav')).toBeNull() }) + it('shows a call into a workspace: its sensitivity and refusal, its workspace, and in its details the owner, reason, access and arguments', async () => { + const admin = fakeAdmin() + admin.listAdminAudit.mockResolvedValue(testPage([accessRecord()])) + await render(settingsWith(), admin) + + const cells = [...logRows()[0].querySelectorAll('td')].map((cell) => cell.textContent) + expect(cells.slice(1)).toEqual([ + formatFullTimestamp(ACCESS_AT), 'admin', 'getChat', 'Workspace metadataRefused', 'owner@example.test', + ]) + const details = await expand() + expect(details.textContent).toContain('Quarterly report') + expect(details.textContent).toContain('ws-titled') + expect(details.textContent).toContain('ViaCLI') + expect(details.textContent).toContain('Ticket 42: agent stuck') + expect(details.textContent).toContain('Metadata only (Full requested) · refused: Content withheld') + expect(details.textContent).toContain('chatId: 3') + }) + + it('marks a content read as one, and names an untitled workspace in its details', async () => { + const admin = fakeAdmin() + admin.listAdminAudit.mockResolvedValue(testPage([ + accessRecord({ exposed: 'content', access: 'full', refused: undefined, workspaceId: 'ws-untitled', workspaceTitle: '' }), + ])) + await render(settingsWith(), admin) + + const cells = [...logRows()[0].querySelectorAll('td')].map((cell) => cell.textContent) + expect(cells.slice(4)).toEqual(['Content read', 'owner@example.test']) + const details = await expand() + expect(details.textContent).toContain('Untitled') + expect(details.textContent).toContain('ws-untitled') + }) + + it('filters by sensitivity: a content read by what it exposed, a setting change by its kind', async () => { + const { listAdminAudit, onFilterChange } = await render() + + await choose(sensitivitySelect(), 'Content read') + expect(onFilterChange).toHaveBeenLastCalledWith({ sensitivity: 'content' }) + expect(listAdminAudit).toHaveBeenLastCalledWith(pageQuery({ exposed: 'content' })) + + await choose(sensitivitySelect(), 'Setting change') + expect(listAdminAudit).toHaveBeenLastCalledWith(pageQuery({ kind: 'setting' })) + }) + + it('filters by a workspace chosen in a record’s details, named in a chip that removes it', async () => { + const admin = fakeAdmin() + admin.listAdminAudit.mockResolvedValue(testPage([accessRecord()])) + const { listAdminAudit, onFilterChange } = await render(settingsWith(), admin) + + await click(within(await expand(), 'Show only calls into Quarterly report')) + expect(onFilterChange).toHaveBeenLastCalledWith({ workspace: 'ws-titled' }) + expect(listAdminAudit).toHaveBeenLastCalledWith(pageQuery({ workspace: 'ws-titled' })) + + const chip = button('Show every workspace, not only Quarterly report') + expect(chip.textContent).toBe('Workspace: Quarterly report') + await click(chip) + expect(onFilterChange).toHaveBeenLastCalledWith({ workspace: undefined }) + // The chip removed itself, so focus goes to the admin field rather than the page. + expect(document.activeElement).toBe(card('Activity log').querySelector('input[role="combobox"]')) + }) + + it('filters by the target user a row names, offering the page’s target users in the filter', async () => { + const admin = fakeAdmin() + admin.listAdminAudit.mockResolvedValue(testPage([ + accessRecord({ workspaceOwner: 'grace' }), accessRecord({ workspaceOwner: 'linus' }), + accessRecord({ workspaceOwner: 'grace' }), settingRecord(), + ])) + const { listAdminAudit, onFilterChange } = await render(settingsWith(), admin) + + await click(within(logRows()[1], 'Show only calls into workspaces linus owns')) + expect(onFilterChange).toHaveBeenLastCalledWith({ owner: 'linus' }) + expect(listAdminAudit).toHaveBeenLastCalledWith(pageQuery({ owner: 'linus' })) + + const target = [...card('Activity log').querySelectorAll('input[role="combobox"]')] + .find((input) => input.placeholder.startsWith('Any user'))! + await type(target, '') + await act(async () => { target.dispatchEvent(new KeyboardEvent('keydown', { key: 'ArrowDown', bubbles: true })) }) + expect([...document.body.querySelectorAll('[role="option"]')].map((option) => option.textContent)) + .toEqual(expect.arrayContaining(['grace', 'linus'])) + }) + + it('filters by the owner and the workspace a record’s details name, and by the owner typed', async () => { + const admin = fakeAdmin() + admin.listAdminAudit.mockResolvedValue(testPage([accessRecord()])) + const { listAdminAudit, onFilterChange } = await render(settingsWith(), admin) + await click(within(logRows()[0], 'Show only calls into workspaces owner@example.test owns')) + expect(onFilterChange).toHaveBeenLastCalledWith({ owner: 'owner@example.test' }) + expect(listAdminAudit).toHaveBeenLastCalledWith(pageQuery({ owner: 'owner@example.test' })) + + const owner = [...card('Activity log').querySelectorAll('input[role="combobox"]')] + .find((input) => input.placeholder.startsWith('Any user'))! + expect(owner.value).toBe('owner@example.test') + await type(owner, 'grace') + await act(async () => { + owner.dispatchEvent(new KeyboardEvent('keydown', { key: 'Enter', bubbles: true, cancelable: true })) + }) + expect(onFilterChange).toHaveBeenLastCalledWith({ owner: 'grace' }) + }) + it('loads the counts and the page again on Refresh, keeping the page shown until then', async () => { const { getAdminAuditStats, listAdminAudit } = await render() const reloaded = deferred() @@ -673,11 +856,111 @@ describe('AdminAuditPanel', () => { }) }) + describe('workspace access', () => { + it('shows the level enforced', async () => { + await render(settingsWith({ access: { effective: 'metadata', chosen: 'full', max: 'metadata' } })) + + expect(accessSelect().textContent).toContain('Metadata only') + expect(card('Workspace access').textContent) + .toContain('The level set here is Full, which that maximum limits to Metadata only. Choose ' + + 'Metadata only to set it here, so that a higher maximum does not raise it.') + }) + + it('says what Metadata only withholds and what it still shows', async () => { + await render() + + expect(card('Workspace access').textContent).toContain( + 'Metadata only withholds chat messages, attachments, file contents, commit messages, and the ' + + 'descriptions of actions, observations and hooks; titles, repository file names, commit ' + + 'authors, and the names and hashes of attachments stay, and titles and names can echo ' + + 'content.') + }) + + it('disables the levels above the deployment’s maximum, naming it', async () => { + await render(settingsWith({ access: { max: 'metadata' } })) + + expect(await enabledOptions(accessSelect())).toEqual(['Off', 'Metadata only']) + expect(card('Workspace access').textContent).toContain('This deployment allows at most Metadata only') + }) + + it('leaves only Off when the deployment allows no inspection', async () => { + await render(settingsWith({ access: { effective: 'off', chosen: 'metadata', max: 'off' } })) + + expect(await enabledOptions(accessSelect())).toEqual(['Off']) + expect(card('Workspace access').textContent).toContain('ADMIN_WORKSPACE_ACCESS_MAX allows no inspection') + }) + + it('sets the level chosen, then re-reads the settings and the log', async () => { + const { setAdminWorkspaceAccess, onSettingsChanged, listAdminAudit } = await render() + + await choose(accessSelect(), 'Full') + + expect(setAdminWorkspaceAccess).toHaveBeenCalledExactlyOnceWith('full') + expect(onSettingsChanged).toHaveBeenCalledOnce() + expect(listAdminAudit).toHaveBeenCalledTimes(2) + }) + + it('stores the level shown as the admins’ choice while the maximum holds a higher one down, and sends nothing for the choice itself', async () => { + const held = await render(settingsWith({ access: { effective: 'metadata', chosen: 'full', max: 'metadata' } })) + await choose(accessSelect(), 'Metadata only') + expect(held.setAdminWorkspaceAccess).toHaveBeenCalledExactlyOnceWith('metadata') + + act(() => root.unmount()) + root = createRoot(container) + const own = await render() + await choose(accessSelect(), 'Metadata only') + expect(own.setAdminWorkspaceAccess).not.toHaveBeenCalled() + }) + + it('is disabled while its change is in flight', async () => { + const admin = fakeAdmin() + const call = deferred() + admin.setAdminWorkspaceAccess.mockReturnValueOnce(call.promise) + await render(settingsWith(), admin) + + await choose(accessSelect(), 'Full') + expect(accessSelect().hasAttribute('data-disabled') || accessSelect().hasAttribute('disabled')).toBe(true) + + await act(async () => call.resolve()) + expect(accessSelect().hasAttribute('data-disabled') || accessSelect().hasAttribute('disabled')).toBe(false) + }) + + // Chosen by keyboard, from an option of the popup, which is gone once the save settles. + it('gives focus back to its trigger once a delayed change settles', async () => { + const admin = fakeAdmin() + const call = deferred() + admin.setAdminWorkspaceAccess.mockReturnValueOnce(call.promise) + await render(settingsWith(), admin) + + await choose(accessSelect(), 'Full') + ;(document.activeElement as HTMLElement | null)?.blur() + await act(async () => call.resolve()) + + expect(document.activeElement).toBe(accessSelect()) + }) + + it('reports a failed change and keeps the server’s level', async () => { + const admin = fakeAdmin() + admin.setAdminWorkspaceAccess.mockRejectedValueOnce(new Error('ADMIN_WORKSPACE_ACCESS_MAX allows at most "metadata"')) + const { onSettingsChanged } = await render(settingsWith(), admin) + + await choose(accessSelect(), 'Full') + + expect(addToast).toHaveBeenCalledExactlyOnceWith({ + title: 'Couldn’t update “Workspace access”', + description: 'ADMIN_WORKSPACE_ACCESS_MAX allows at most "metadata"', + variant: 'error', + }) + expect(onSettingsChanged).not.toHaveBeenCalled() + expect(accessSelect().textContent).toContain('Metadata only') + }) + }) + describe('the log’s column widths', () => { - it('starts from the defaults, the Changes column taking what is left', async () => { + it('starts from the defaults, the Action column taking what is left', async () => { await render() - expect(widths()).toEqual(['48px', '180px', '160px', '220px', '', '64px']) + expect(widths()).toEqual(['48px', '170px', '140px', '', '240px', '140px']) expect(stored()).toBeNull() }) @@ -686,44 +969,44 @@ describe('AdminAuditPanel', () => { handle('Time').focus() await key(handle('Time'), { key: 'ArrowRight' }) - expect(widths()[1]).toBe('196px') + expect(widths()[1]).toBe('186px') await key(handle('Time'), { key: 'ArrowLeft', shiftKey: true }) - expect(widths()[1]).toBe('132px') + expect(widths()[1]).toBe('122px') - expect(stored()).toEqual({ time: 132, admin: 160, action: 220, via: 64 }) + expect(stored()).toEqual({ time: 122, admin: 140, sensitivity: 240, target: 140 }) }) it('resizes a column by dragging its edge, storing the width once the drag ends', async () => { await render() - await pointer('pointerdown', handle('Action'), 500) - await pointer('pointermove', handle('Action'), 560) - expect(widths()[3]).toBe('280px') + await pointer('pointerdown', handle('Admin'), 500) + await pointer('pointermove', handle('Admin'), 560) + expect(widths()[2]).toBe('200px') expect(stored()).toBeNull() - await pointer('pointerup', handle('Action'), 580) + await pointer('pointerup', handle('Admin'), 580) - expect(widths()[3]).toBe('300px') - expect(stored()).toEqual({ time: 180, admin: 160, action: 300, via: 64 }) + expect(widths()[2]).toBe('220px') + expect(stored()).toEqual({ time: 170, admin: 220, sensitivity: 240, target: 140 }) }) it('puts back the width a drag began from when the drag is cancelled, storing nothing', async () => { await render() - await pointer('pointerdown', handle('Action'), 500) - await pointer('pointermove', handle('Action'), 600) - expect(widths()[3]).toBe('320px') + await pointer('pointerdown', handle('Admin'), 500) + await pointer('pointermove', handle('Admin'), 600) + expect(widths()[2]).toBe('240px') // A cancel may report any position: 0 here, which would narrow the column to its least. - await pointer('pointercancel', handle('Action'), 0) + await pointer('pointercancel', handle('Admin'), 0) - expect(widths()[3]).toBe('220px') + expect(widths()[2]).toBe('140px') expect(stored()).toBeNull() }) it('keeps a column within its bounds', async () => { await render() - await pointer('pointerdown', handle('Via'), 500) - await pointer('pointerup', handle('Via'), 0) + await pointer('pointerdown', handle('Target user'), 500) + await pointer('pointerup', handle('Target user'), 0) expect(widths()[5]).toBe('64px') await pointer('pointerdown', handle('Admin'), 0) await pointer('pointerup', handle('Admin'), 5000) @@ -736,16 +1019,16 @@ describe('AdminAuditPanel', () => { await act(async () => { handle('Admin').dispatchEvent(new MouseEvent('dblclick', { bubbles: true })) }) - expect(widths()[2]).toBe('160px') - expect(stored()).toEqual({ time: 180, admin: 160, action: 220, via: 64 }) + expect(widths()[2]).toBe('140px') + expect(stored()).toEqual({ time: 170, admin: 140, sensitivity: 240, target: 140 }) }) it('starts from the widths stored, bounded, and the defaults for any missing or malformed', async () => { - localStorage.setItem(AUDIT_COLUMN_WIDTHS_KEY, JSON.stringify({ admin: 333, time: 5, via: 'wide' })) + localStorage.setItem(AUDIT_COLUMN_WIDTHS_KEY, JSON.stringify({ admin: 333, time: 5, target: 'wide' })) await render() - expect(widths()).toEqual(['48px', '64px', '333px', '220px', '', '64px']) + expect(widths()).toEqual(['48px', '64px', '333px', '', '240px', '140px']) }) it('ignores stored widths that are not JSON', async () => { @@ -753,7 +1036,7 @@ describe('AdminAuditPanel', () => { await render() - expect(widths()).toEqual(['48px', '180px', '160px', '220px', '', '64px']) + expect(widths()).toEqual(['48px', '170px', '140px', '', '240px', '140px']) }) }) diff --git a/packages/workshop-frontend/src/features/admin-audit/AdminAuditPanel.tsx b/packages/workshop-frontend/src/features/admin-audit/AdminAuditPanel.tsx index 52f732ead2..36ad18ee63 100644 --- a/packages/workshop-frontend/src/features/admin-audit/AdminAuditPanel.tsx +++ b/packages/workshop-frontend/src/features/admin-audit/AdminAuditPanel.tsx @@ -1,11 +1,12 @@ -import { useEffect, useId, useState } from 'react' -import { Button, Loader, Table, useKumoToastManager } from '@cloudflare/kumo' +import { useEffect, useId, useRef, useState } from 'react' +import { Button, Loader, Select, Table, useKumoToastManager } from '@cloudflare/kumo' import { ArrowsClockwise } from '@phosphor-icons/react' import type { RpcStub } from 'capnweb' -import { MAX_ADMIN_AUDIT_RETENTION_DAYS } from '@gadgets/workshop-shared/api' -import type { AdminApi, AdminAuditStats } from '@gadgets/workshop-shared/api' +import { ADMIN_WORKSPACE_ACCESS_LEVELS, MAX_ADMIN_AUDIT_RETENTION_DAYS } from '@gadgets/workshop-shared/api' +import type { AdminApi, AdminAuditStats, AdminWorkspaceAccess } from '@gadgets/workshop-shared/api' import { DurationSetting } from '../../components/DurationSetting' import { logRpcFailure, rpcFailureDescription } from '../../rpcErrors' +import { ACCESS_LEVEL_NAMES } from './accessNames' import type { AuditFilter } from './auditFilter' import { AuditLog } from './AuditLog' import type { AuditSettings } from './auditSettings' @@ -25,6 +26,7 @@ type AdminAuditPanelProps = { } const CARD = 'bg-kumo-elevated border border-kumo-line rounded-xl p-6' +const ACCESS_LABEL = 'Workspace access' const RETENTION_LABEL = 'Keep audit records for (days)' const STATS_DAYS = 7 @@ -40,16 +42,27 @@ const dayName = (day: string) => { // The answer to one load of the statistics, for the value of `loads` it answered. type StatsAnswer = { loads: number } & ({ stats: AdminAuditStats } | { failed: true }) -const StatsCards = ({ stats, onAdminSelect }: { +// A count of content reads, set apart once there is one, since those are what an admin should look +// at first. +const ContentReads = ({ count }: { count: number }) => ( + 0 ? ' font-semibold text-kumo-danger' : ' text-kumo-subtle'}`}> + {NUMBER.format(count)} + +) + +const StatsCards = ({ stats, onFilterBy }: { stats: AdminAuditStats - onAdminSelect: (admin: string) => void + onFilterBy: (filter: AuditFilter) => void }) => ( -
+

Admin actions, last {stats.days} days

{NUMBER.format(stats.totalEvents)} {' '}{stats.totalEvents === 1 ? 'action' : 'actions'}, by UTC day + + {NUMBER.format(stats.contentReads)} {stats.contentReads === 1 ? 'content read' : 'content reads'} +

@@ -72,7 +85,8 @@ const StatsCards = ({ stats, onAdminSelect }: {

Most active admins

- By actions in the last {stats.days} days. Choose an admin to see only their activity. + By actions in the last {stats.days} days, with the content reads among them. Choose an + admin to see only their activity.

{stats.topAdmins.length === 0 ?

No admin activity in this window.

@@ -82,10 +96,11 @@ const StatsCards = ({ stats, onAdminSelect }: { Admin Actions + Content reads - {stats.topAdmins.map(({ admin, count }) => ( + {stats.topAdmins.map(({ admin, count, contentReads }) => ( {NUMBER.format(count)} + ))} @@ -108,18 +124,47 @@ const StatsCards = ({ stats, onAdminSelect }: {
) +const accessHelp = ({ chosen, effective, max }: AuditSettings['adminWorkspaceAccess']): string[] => { + const help = [ + 'The most of a workspace an admin may see when inspecting it: each inspection asks for ' + + 'Metadata only (the default) or Full, and gets the lower of that and this level. Metadata ' + + 'only withholds chat messages, attachments, file contents, commit messages, and the ' + + 'descriptions of actions, observations and hooks; titles, repository file names, commit ' + + 'authors, and the names and hashes of attachments stay, and titles and names can echo ' + + 'content. Off refuses inspection. Lowering it reaches views already open within about a ' + + 'minute.', + ] + if (max === 'off') { + help.push('This deployment’s ADMIN_WORKSPACE_ACCESS_MAX allows no inspection (a value that is ' + + 'not a level counts as off).') + } else { + help.push(`This deployment allows at most ${ACCESS_LEVEL_NAMES[max]}, which only a change to ` + + 'its ADMIN_WORKSPACE_ACCESS_MAX variable can raise.') + } + if (chosen !== effective) { + help.push(`The level set here is ${ACCESS_LEVEL_NAMES[chosen]}, which that maximum limits to ` + + `${ACCESS_LEVEL_NAMES[effective]}. Choose ${ACCESS_LEVEL_NAMES[effective]} to set it here, ` + + 'so that a higher maximum does not raise it.') + } + return help +} + +const accessRank = (level: AdminWorkspaceAccess) => ADMIN_WORKSPACE_ACCESS_LEVELS.indexOf(level) + /** * The admin Audit tab: what the deployment-wide admin audit log holds for the last week, the log - * itself, filtered and a page at a time, and the log's retention. + * itself, filtered and a page at a time, and the settings that bound admin access to workspaces and + * the log's retention. */ export const AdminAuditPanel = ({ admin, settings, onSettingsChanged, filter, onFilterChange }: AdminAuditPanelProps) => { const toasts = useKumoToastManager() // Each load is one value of this; a new one supersedes those before it. const [loads, setLoads] = useState(0) const [statsAnswer, setStatsAnswer] = useState(null) + const [savingAccess, setSavingAccess] = useState(false) const activityHeading = useId() const logHeading = useId() - const retention = settings.adminAuditRetentionDays + const { adminWorkspaceAccess: access, adminAuditRetentionDays: retention } = settings useEffect(() => { let current = true @@ -142,6 +187,21 @@ export const AdminAuditPanel = ({ admin, settings, onSettingsChanged, filter, on // Each settings change is one value of this, which sends the log back to its newest page. const [changes, setChanges] = useState(0) + // The access select is disabled while its change is in flight, and a browser takes focus from a + // control that becomes disabled without giving it back. So its trigger gets focus again once the + // change settles, unless focus has gone elsewhere since. The trigger, not the control focused + // when the change began: a choice by keyboard is made from an option of the popup, which is gone + // by then. Kumo's Select hands out no ref to its trigger, so it is found through this wrapper. + const accessField = useRef(null) + const refocusAccess = useRef(false) + useEffect(() => { + if (savingAccess || !refocusAccess.current) return + refocusAccess.current = false + if (document.activeElement === document.body) { + accessField.current?.querySelector('[role="combobox"]')?.focus() + } + }, [savingAccess]) + const reportFailure = (title: string, err: unknown) => { logRpcFailure(`${title}:`, err) toasts.add({ title, description: rpcFailureDescription(err), variant: 'error' }) @@ -167,7 +227,21 @@ export const AdminAuditPanel = ({ admin, settings, onSettingsChanged, filter, on } } - const selectAdmin = (selected: string) => onFilterChange({ ...filter, admin: selected }) + const changeAccess = async (level: AdminWorkspaceAccess) => { + // Against the stored choice, not the level shown: picking the shown level again while the + // maximum holds a higher choice down makes it the admins' choice, which a later rise of + // ADMIN_WORKSPACE_ACCESS_MAX then cannot raise. + if (level === access.chosen) return + refocusAccess.current = true + setSavingAccess(true) + try { + await change(`Couldn’t update “${ACCESS_LABEL}”`, () => admin.setAdminWorkspaceAccess(level)) + } finally { + setSavingAccess(false) + } + } + + const filterBy = (added: AuditFilter) => onFilterChange({ ...filter, ...added }) return (
@@ -176,8 +250,8 @@ export const AdminAuditPanel = ({ admin, settings, onSettingsChanged, filter, on

Admin activity

- Every change an admin makes to the deployment’s settings, kept - for {NUMBER.format(retention.effective)} days. + Every change an admin makes to the deployment’s settings, and every admin call into a + workspace, kept for {NUMBER.format(retention.effective)} days.

)} - {stats !== null && !statsFailed && } + {stats !== null && !statsFailed && }

Activity log

@@ -218,6 +292,30 @@ export const AdminAuditPanel = ({ admin, settings, onSettingsChanged, filter, on
+
+

{ACCESS_LABEL}

+
+ + label="Access level" + value={access.effective} + disabled={savingAccess} + onValueChange={(level) => { if (level !== null) void changeAccess(level) }} + renderValue={(level) => ACCESS_LEVEL_NAMES[level]} + description={( + + {accessHelp(access).map((text) => {text})} + + )} + > + {ADMIN_WORKSPACE_ACCESS_LEVELS.map((level) => ( + accessRank(access.max)}> + {ACCESS_LEVEL_NAMES[level]} + + ))} + +
+
+

Audit retention

{retention.chosen < retention.min && ( diff --git a/packages/workshop-frontend/src/features/admin-audit/AuditFilters.tsx b/packages/workshop-frontend/src/features/admin-audit/AuditFilters.tsx index ceae1678ef..41954539e7 100644 --- a/packages/workshop-frontend/src/features/admin-audit/AuditFilters.tsx +++ b/packages/workshop-frontend/src/features/admin-audit/AuditFilters.tsx @@ -1,87 +1,71 @@ -import { useRef, useState, type KeyboardEvent } from 'react' -import { Autocomplete, Button, Combobox } from '@cloudflare/kumo' +import { useRef, useState } from 'react' +import { Button, Combobox, Select } from '@cloudflare/kumo' import { X } from '@phosphor-icons/react' -import { AUDIT_METHOD_GROUPS, type AuditFilter, type AuditMethod } from './auditFilter' +import { + AUDIT_METHOD_GROUPS, + AUDIT_SENSITIVITIES, + isFiltered, + type AuditFilter, + type AuditMethod, + type AuditSensitivity, +} from './auditFilter' +import { SENSITIVITY_NAMES } from './SensitivityBadge' +import { TypedFilterField } from './TypedFilterField' // The Action filter's choices: each group's methods, in alphabetical order, as Base UI groups them. type ActionGroup = { value: string; items: AuditMethod[] } const ACTION_GROUPS: ActionGroup[] = AUDIT_METHOD_GROUPS.map(({ label, methods }) => ({ value: label, items: methods.toSorted() })) +const ANY_SENSITIVITY = 'any' +type SensitivityChoice = AuditSensitivity | typeof ANY_SENSITIVITY + /** - * The Audit tab's filters. The admin is typed (the deployment's admins are listed nowhere), with - * `adminSuggestions` offered as the field is typed in, and applies on Enter or once a suggestion is - * picked. The action is searched by typing, and applies as soon as it is chosen; clearing it shows - * every action. + * The Audit tab's filters. The admin and the target user (the owner of a workspace an admin called + * into) are typed, with `adminSuggestions` and `targetSuggestions` offered, and apply on Enter or + * once a suggestion is picked. The action is + * searched by typing, and the sensitivity chosen; both apply at once. The workspace filter is set + * from a record's details in the log, and shows as a chip named `workspaceLabel` that removes + * it. */ -export const AuditFilters = ({ filter, adminSuggestions, onFilterChange }: { +export const AuditFilters = ({ filter, adminSuggestions, targetSuggestions, workspaceLabel, onFilterChange }: { filter: AuditFilter adminSuggestions: string[] + targetSuggestions: string[] + /** What to call the filtered workspace: its title, or its id when no title is known. */ + workspaceLabel: string | undefined onFilterChange: (filter: AuditFilter) => void }) => { - // What is typed in the admin field, for the admin filter it was typed over. Dropped as soon as the - // filter changes from elsewhere (an admin's name picked in the log), during that render, so the - // field shows the new filter and coming back to the old one does not bring back what was typed. - const [typed, setTyped] = useState({ over: filter.admin, text: filter.admin ?? '' }) - if (typed.over !== filter.admin) setTyped({ over: filter.admin, text: filter.admin ?? '' }) - const adminText = typed.over === filter.admin ? typed.text : (filter.admin ?? '') - - const applyAdmin = (text: string) => { - const admin = text.trim() - setTyped({ over: admin || undefined, text: admin }) - onFilterChange({ ...filter, admin: admin || undefined }) - } - // Enter applies what is typed. Base UI gives the Autocomplete and the Combobox hidden text inputs - // of their own, and a form of several text inputs and no submit button is never submitted by Enter, - // so the key is handled here. While a suggestion is highlighted, Enter picks it instead. - const applyOnEnter = (event: KeyboardEvent) => { - if (event.key !== 'Enter' || event.nativeEvent.isComposing) return - if (event.target instanceof HTMLElement && event.target.getAttribute('aria-activedescendant')) return - event.preventDefault() - applyAdmin(adminText) - } - const filtered = filter.admin !== undefined || filter.method !== undefined - // Kumo's Autocomplete hands out no ref to its input, so it is found through this wrapper: Clear - // filters removes itself, and gives focus to the admin field rather than drop it. + // Kumo's Autocomplete hands out no ref to its input, so it is found through this wrapper: the + // workspace chip and Clear filters remove themselves, and give focus to the admin field rather + // than drop it. const adminField = useRef(null) + // How many times every filter was cleared: text typed in the admin and owner fields and not + // applied is cleared with them. + const [clears, setClears] = useState(0) + const focusAdminField = () => + adminField.current?.querySelector('[role="combobox"]')?.focus() return ( -
-
- - label="Admin" - items={adminSuggestions} - value={adminText} - onValueChange={(text, { reason }) => { - // Escape, with the list closed, asks to empty the field: it abandons what was typed - // instead, showing the admin filter applied, so the field never says "any admin" while - // the log is filtered by one. - if (reason === 'escape-key') { - setTyped({ over: filter.admin, text: filter.admin ?? '' }) - return - } - setTyped({ over: filter.admin, text }) - if (reason === 'item-press') applyAdmin(text) - }} - > - - - - {(admin: string) => ( - {admin} - )} - - - -
-
+
+
+
+ onFilterChange({ ...filter, admin })} + /> +
onFilterChange({ ...filter, method: (method as AuditMethod | null) ?? undefined })} > - + No action matches. @@ -100,19 +84,61 @@ export const AuditFilters = ({ filter, adminSuggestions, onFilterChange }: { -
- {filtered && ( - + Any sensitivity + {AUDIT_SENSITIVITIES.map((sensitivity) => ( + {SENSITIVITY_NAMES[sensitivity]} + ))} + +
+ onFilterChange({ ...filter, owner })} + /> +
+
+ {isFiltered(filter) && ( +
+ {filter.workspace !== undefined && ( + + )} + +
)}
) diff --git a/packages/workshop-frontend/src/features/admin-audit/AuditLog.tsx b/packages/workshop-frontend/src/features/admin-audit/AuditLog.tsx index 5cd94e1db7..3710b437d4 100644 --- a/packages/workshop-frontend/src/features/admin-audit/AuditLog.tsx +++ b/packages/workshop-frontend/src/features/admin-audit/AuditLog.tsx @@ -4,7 +4,7 @@ import { CaretLeft, CaretRight } from '@phosphor-icons/react' import type { RpcStub } from 'capnweb' import type { AdminApi, AdminAuditPage } from '@gadgets/workshop-shared/api' import { logRpcFailure } from '../../rpcErrors' -import type { AuditFilter } from './auditFilter' +import { auditQueryOf, isFiltered, type AuditFilter } from './auditFilter' import { AuditFilters } from './AuditFilters' import { AUDIT_LOG_COLUMNS, AuditLogRow } from './AuditLogRow' import { ColumnResizeHandle } from './ColumnResizeHandle' @@ -17,20 +17,24 @@ export const AUDIT_PAGE_SIZE = 50 export const AUDIT_COLUMN_WIDTHS_KEY = 'gadgets:workshop:adminAuditColumnWidths' // The log's resizable columns, by the name each is shown and announced with, and their widths -// before any is resized, in pixels. Changes is not among them: it takes the width left over, at -// least CHANGES_MIN_WIDTH, so widening one column never narrows another. -const RESIZABLE_COLUMNS = { time: 'Time', admin: 'Admin', action: 'Action', via: 'Via' } as const +// before any is resized, in pixels. Action is not among them: it takes the width left over, at +// least ACTION_MIN_WIDTH, so that it is the column a narrow log cuts, never a name. +const RESIZABLE_COLUMNS = { + time: 'Time', admin: 'Admin', sensitivity: 'Sensitivity', target: 'Target user', +} as const type ResizableColumn = keyof typeof RESIZABLE_COLUMNS -// Sized so that, with Changes at its least, the log fits the Audit tab at a 1440 px window. -const DEFAULT_WIDTHS: Record = { time: 180, admin: 160, action: 220, via: 64 } +// Sized so that, with Action at its least, the log fits the Audit tab at a 1440 px window, and a +// refused call's two badges share a line. +const DEFAULT_WIDTHS: Record = { time: 170, admin: 140, sensitivity: 240, target: 140 } const TOGGLE_WIDTH = 48 -const CHANGES_MIN_WIDTH = 160 +const ACTION_MIN_WIDTH = 140 // The answer to one load, for the request it answered (see `request` below). type Answer = { request: string } & ({ page: AdminAuditPage } | { failed: true }) // Identifies a filter, so that the pages turned under one are forgotten once it changes. -const filterKey = ({ admin, method }: AuditFilter) => JSON.stringify([admin ?? null, method ?? null]) +const filterKey = ({ admin, method, sensitivity, workspace, owner }: AuditFilter) => + JSON.stringify([admin, method, sensitivity, workspace, owner].map((value) => value ?? null)) /** * The Audit tab's log, filtered as `filter` says, one page at a time, newest first. Older pages @@ -65,10 +69,11 @@ export const AuditLog = ({ admin, filter, onFilterChange, adminSuggestions, relo // Everything a load depends on, so that an answer to an earlier one is told apart. const request = JSON.stringify([key, cursor ?? null, reloads, retries]) - const { admin: byAdmin, method } = filter + const { admin: byAdmin, method, sensitivity, workspace, owner } = filter useEffect(() => { let current = true - admin.listAdminAudit({ admin: byAdmin, method, cursor, limit: AUDIT_PAGE_SIZE }).then( + const query = auditQueryOf({ admin: byAdmin, method, sensitivity, workspace, owner }) + admin.listAdminAudit({ ...query, cursor, limit: AUDIT_PAGE_SIZE }).then( (page) => { if (current) setAnswer({ request, page }) }, (err: unknown) => { if (!current) return @@ -77,7 +82,7 @@ export const AuditLog = ({ admin, filter, onFilterChange, adminSuggestions, relo }, ) return () => { current = false } - }, [admin, byAdmin, method, cursor, request]) + }, [admin, byAdmin, method, sensitivity, workspace, owner, cursor, request]) // A page stays shown, dimmed, until the load that replaces it answers. const page = answer !== null && 'page' in answer ? answer.page : null @@ -85,6 +90,22 @@ export const AuditLog = ({ admin, filter, onFilterChange, adminSuggestions, relo const failed = answer !== null && 'failed' in answer && !busy const nextCursor = page?.nextCursor const pageNumber = cursors.length + 1 + // A row's name or workspace button is gone once the filtered page replaces the row, so focus goes + // to the filter field that now shows the choice (the admin field for a workspace, whose chip comes + // and goes) rather than drop to the page. + const filters = useRef(null) + const filterBy = (added: AuditFilter) => { + onFilterChange({ ...filter, ...added }) + const field = added.owner !== undefined ? 'owner' : 'admin' + filters.current?.querySelector(`[data-filter-field="${field}"] input[role="combobox"]`) + ?.focus() + } + const targetSuggestions = [...new Set(page?.records.flatMap((record) => + record.kind === 'workspace' && record.workspaceOwner ? [record.workspaceOwner] : []))] + const workspaceRecord = page?.records.find((record) => + record.kind === 'workspace' && record.workspaceId === workspace && record.workspaceTitle) + const workspaceLabel = workspace === undefined ? undefined + : (workspaceRecord?.kind === 'workspace' ? workspaceRecord.workspaceTitle : undefined) // A page button is disabled on the first or the last page, and a browser drops the focus of a // control that becomes disabled without giving it back. So once a turn lands there, focus moves @@ -108,10 +129,10 @@ export const AuditLog = ({ admin, filter, onFilterChange, adminSuggestions, relo }, [busy, pageNumber]) const columns = useColumnWidths(AUDIT_COLUMN_WIDTHS_KEY, DEFAULT_WIDTHS) - const tableWidth = TOGGLE_WIDTH + CHANGES_MIN_WIDTH + + const tableWidth = TOGGLE_WIDTH + ACTION_MIN_WIDTH + Object.values(columns.widths).reduce((sum, width) => sum + width, 0) const head = (column: ResizableColumn) => ( - + {RESIZABLE_COLUMNS[column]} ) - // A row's admin button is gone once the page filtered by that admin replaces the row, so focus - // goes to the admin field, which now shows the admin chosen, rather than drop to the page. - const filters = useRef(null) - const selectAdmin = (selected: string) => { - onFilterChange({ ...filter, admin: selected }) - filters.current?.querySelector('input[role="combobox"]')?.focus() - } - return (
- +
{page === null && !failed && ( @@ -158,18 +177,18 @@ export const AuditLog = ({ admin, filter, onFilterChange, adminSuggestions, relo
- - + + - Details + Expand {head('time')} {head('admin')} - {head('action')} - Changes - {head('via')} + Action + {head('sensitivity')} + {head('target')} @@ -179,7 +198,7 @@ export const AuditLog = ({ admin, filter, onFilterChange, adminSuggestions, relo {pageNumber > 1 ? 'No older admin activity.' - : filter.admin !== undefined || filter.method !== undefined + : isFiltered(filter) ? 'No admin activity matches these filters.' : 'No admin activity is recorded yet.'} @@ -190,7 +209,7 @@ export const AuditLog = ({ admin, filter, onFilterChange, adminSuggestions, relo key={`${cursor ?? ''}:${index}:${record.at.getTime()}`} record={record} shaded={index % 2 === 1} - onAdminSelect={selectAdmin} + onFilterBy={filterBy} /> ))} diff --git a/packages/workshop-frontend/src/features/admin-audit/AuditLogRow.tsx b/packages/workshop-frontend/src/features/admin-audit/AuditLogRow.tsx index 85b714eb0c..871294a58a 100644 --- a/packages/workshop-frontend/src/features/admin-audit/AuditLogRow.tsx +++ b/packages/workshop-frontend/src/features/admin-audit/AuditLogRow.tsx @@ -1,9 +1,17 @@ -import { useId, useState } from 'react' -import { Button, Table } from '@cloudflare/kumo' +import { useId, useState, type ReactNode } from 'react' +import { Badge, Button, Table } from '@cloudflare/kumo' import { CaretRight } from '@phosphor-icons/react' -import type { AdminAuditRecord, AdminSettingChange, AdminVia } from '@gadgets/workshop-shared/api' +import type { + AdminAccessRecord, + AdminAuditRecord, + AdminSettingChange, + AdminVia, +} from '@gadgets/workshop-shared/api' import { formatFullTimestamp } from '../../utils/formatTimestamp' +import { ACCESS_LEVEL_NAMES, REFUSAL_NAMES } from './accessNames' +import { sensitivityOf, type AuditFilter } from './auditFilter' import { AuditValue } from './AuditValue' +import { SensitivityBadge } from './SensitivityBadge' /** How many columns an AuditLogRow fills, for the table's empty and detail rows. */ export const AUDIT_LOG_COLUMNS = 6 @@ -63,22 +71,79 @@ const SettingChanges = ({ changes }: { changes: AdminSettingChange[] }) => ( ) -const changesSummary = (changes: AdminSettingChange[]) => { - if (changes.length === 0) return '' - const more = changes.length - 1 - return more === 0 ? changeName(changes[0]) : `${changeName(changes[0])} + ${NUMBER.format(more)} more` +// A user's name that filters the log by them, shown whole: one longer than its column wraps, +// since the action is the column cut. +const NameFilter = ({ name, label, onFilter }: { name: string; label: string; onFilter: () => void }) => ( + +) + +const Detail = ({ term, children }: { term: string; children: ReactNode }) => ( +
+
{term}
+
{children}
+
+) + +const AccessDetails = ({ record, onFilterBy }: { + record: AdminAccessRecord + onFilterBy: (filter: AuditFilter) => void +}) => { + const args = Object.entries(record.args) + return ( +
+ + + {record.workspaceId} + + {record.reason} + + {ACCESS_LEVEL_NAMES[record.access]} + {record.requested !== record.access && ` (${ACCESS_LEVEL_NAMES[record.requested]} requested)`} + {record.refused && <> · refused: {REFUSAL_NAMES[record.refused]}} + + {args.length > 0 && ( + + + {args.map(([name, value]) => `${name}: ${String(value)}`).join(' · ')} + + + )} + {VIA_NAMES[record.via]} +
+ ) } /** - * One record of the Audit tab's log: a summary row, with a toggle that shows what the record holds - * in full in a row below it. The admin's name filters the log by that admin. `shaded` stripes the - * summary row as Kumo stripes a table's even rows: Kumo counts the details row below each one, hidden - * or not, so every summary row is an odd row to it and would never be striped. + * One record of the Audit tab's log: a summary row (time, admin, action, sensitivity, target user), + * with a toggle that shows the rest in a row below it: how the call came, and a setting record's + * changes or an access record's workspace, reason, level and arguments. An access record's target + * user is its workspace's owner, the user the call reached. The admin's name and the target user, + * and in an access record's details its workspace, filter the log by `onFilterBy`. Names are never + * cut: an action too long for the space left is, whole in its tooltip. `shaded` stripes the summary + * row as Kumo stripes a table's even rows: Kumo counts the details row below each one, hidden or + * not, so every summary row is an odd row to it and would never be striped. */ -export const AuditLogRow = ({ record, shaded, onAdminSelect }: { +export const AuditLogRow = ({ record, shaded, onFilterBy }: { record: AdminAuditRecord shaded: boolean - onAdminSelect: (admin: string) => void + /** Adds a filter to those set. */ + onFilterBy: (filter: AuditFilter) => void }) => { const [expanded, setExpanded] = useState(false) const detailsId = useId() @@ -86,7 +151,7 @@ export const AuditLogRow = ({ record, shaded, onAdminSelect }: { return ( <> - + + + onFilterBy({ admin: record.admin })} + /> + + + {record.method} - {record.method} - - - {changesSummary(record.changes)} + + + + {record.kind === 'workspace' && record.refused && ( + Refused + )} - {VIA_NAMES[record.via]} + + {record.kind === 'setting' + ? ( + // A setting change reaches every user, not one: no user is its target. + + + Deployment-wide + + ) + : record.workspaceOwner + ? ( + onFilterBy({ owner: record.workspaceOwner })} + /> + ) + : Unknown} + diff --git a/packages/workshop-frontend/src/features/admin-audit/SensitivityBadge.tsx b/packages/workshop-frontend/src/features/admin-audit/SensitivityBadge.tsx new file mode 100644 index 0000000000..bb45dedb07 --- /dev/null +++ b/packages/workshop-frontend/src/features/admin-audit/SensitivityBadge.tsx @@ -0,0 +1,16 @@ +import { Badge } from '@cloudflare/kumo' +import type { AuditSensitivity } from './auditFilter' + +/** What the Audit tab calls each sensitivity. */ +export const SENSITIVITY_NAMES: Record = { + content: 'Content read', + metadata: 'Workspace metadata', + setting: 'Setting change', +} + +const VARIANTS = { content: 'error', metadata: 'neutral', setting: 'outline' } as const + +/** A record's sensitivity, as a badge: only a content read's is colored, so content reads stand out. */ +export const SensitivityBadge = ({ sensitivity }: { sensitivity: AuditSensitivity }) => ( + {SENSITIVITY_NAMES[sensitivity]} +) diff --git a/packages/workshop-frontend/src/features/admin-audit/TypedFilterField.tsx b/packages/workshop-frontend/src/features/admin-audit/TypedFilterField.tsx new file mode 100644 index 0000000000..f093c43d22 --- /dev/null +++ b/packages/workshop-frontend/src/features/admin-audit/TypedFilterField.tsx @@ -0,0 +1,68 @@ +import { useState, type KeyboardEvent } from 'react' +import { Autocomplete } from '@cloudflare/kumo' + +/** + * A filter typed as a user id, with `suggestions` offered as it is typed in. It applies on Enter or + * once a suggestion is picked; applying nothing clears it, and Escape abandons what was typed. Once `value` changes from elsewhere (a + * name picked in the log), or `clears` grows (every filter cleared), the field shows `value`, + * whatever was typed and not applied. + */ +export const TypedFilterField = ({ label, placeholder, value, clears, suggestions, onApply }: { + label: string + placeholder: string + value: string | undefined + clears: number + suggestions: string[] + onApply: (value: string | undefined) => void +}) => { + // What is typed, for the value and the clear it was typed over. Dropped as soon as either changes, + // during that render, so coming back to an earlier value does not bring back what was typed. + const [typed, setTyped] = useState({ over: value, clears, text: value ?? '' }) + if (typed.over !== value || typed.clears !== clears) setTyped({ over: value, clears, text: value ?? '' }) + const shown = typed.over === value && typed.clears === clears ? typed.text : (value ?? '') + + const apply = (text: string) => { + const applied = text.trim() || undefined + setTyped({ over: applied, clears, text: applied ?? '' }) + onApply(applied) + } + // Base UI gives the Autocomplete a hidden text input of its own, and a form of several text + // inputs and no submit button is never submitted by Enter, so the key is handled here. While a + // suggestion is highlighted, Enter picks it instead. + const applyOnEnter = (event: KeyboardEvent) => { + if (event.key !== 'Enter' || event.nativeEvent.isComposing) return + if (event.target instanceof HTMLElement && event.target.getAttribute('aria-activedescendant')) return + event.preventDefault() + apply(shown) + } + + return ( +
+ + label={label} + items={suggestions} + value={shown} + onValueChange={(text, { reason }) => { + // Escape, with the list closed, asks to empty the field: it abandons what was typed + // instead, showing the filter applied, so the field never says "any" while the log is + // filtered. + if (reason === 'escape-key') { + setTyped({ over: value, clears, text: value ?? '' }) + return + } + setTyped({ over: value, clears, text }) + if (reason === 'item-press') apply(text) + }} + > + + + + {(suggestion: string) => ( + {suggestion} + )} + + + +
+ ) +} diff --git a/packages/workshop-frontend/src/features/admin-audit/accessNames.ts b/packages/workshop-frontend/src/features/admin-audit/accessNames.ts new file mode 100644 index 0000000000..185b707af8 --- /dev/null +++ b/packages/workshop-frontend/src/features/admin-audit/accessNames.ts @@ -0,0 +1,14 @@ +import { ADMIN_ACCESS_ERROR_CODES, type AdminAccessRecord, type AdminWorkspaceAccess } from '@gadgets/workshop-shared/api' + +/** What the Audit tab calls each level of admin access to a workspace. */ +export const ACCESS_LEVEL_NAMES: Record = { + off: 'Off', + metadata: 'Metadata only', + full: 'Full', +} + +/** What the Audit tab calls each reason an admin call into a workspace was refused. */ +export const REFUSAL_NAMES: Record, string> = { + [ADMIN_ACCESS_ERROR_CODES.contentWithheld]: 'Content withheld', + [ADMIN_ACCESS_ERROR_CODES.accessOff]: 'Access off', +} diff --git a/packages/workshop-frontend/src/features/admin-audit/auditFilter.ts b/packages/workshop-frontend/src/features/admin-audit/auditFilter.ts index f0eb957530..7337af8419 100644 --- a/packages/workshop-frontend/src/features/admin-audit/auditFilter.ts +++ b/packages/workshop-frontend/src/features/admin-audit/auditFilter.ts @@ -1,29 +1,77 @@ -import { ADMIN_SETTING_METHODS, type AdminAuditRecord } from '@gadgets/workshop-shared/api' +import { + ADMIN_SETTING_METHODS, + ADMIN_WORKSPACE_METHODS, + type AdminAuditQuery, + type AdminAuditRecord, +} from '@gadgets/workshop-shared/api' /** A method an audit record can name, which the Audit tab can filter by. */ export type AuditMethod = AdminAuditRecord['method'] +/** + * How sensitive a record is, by what it exposed: "content" for a call into a workspace whose answer + * held content (AdminAccessRecord.exposed), "metadata" for any other call into a workspace, and + * "setting" for a change to the deployment's settings. + */ +export type AuditSensitivity = 'content' | 'metadata' | 'setting' + +/** The sensitivities, most sensitive first. */ +export const AUDIT_SENSITIVITIES: readonly AuditSensitivity[] = ['content', 'metadata', 'setting'] + +/** How sensitive `record` is (see AuditSensitivity). */ +export const sensitivityOf = (record: AdminAuditRecord): AuditSensitivity => + record.kind === 'setting' ? 'setting' : record.exposed + /** * The Audit tab's filters, kept in the admin page's URL as search parameters of these names, so a - * filtered view can be linked. Each is passed to AdminApi.listAdminAudit() as is. + * filtered view can be linked. */ export type AuditFilter = { /** Only this admin's records, by canonical user id. */ admin?: string /** Only records of calls to this method. */ method?: AuditMethod + /** Only records this sensitive. */ + sensitivity?: AuditSensitivity + /** Only records of access to this workspace, by id. */ + workspace?: string + /** Only records of access to workspaces this user owns, by canonical user id. */ + owner?: string } -/** The methods the Audit tab offers as filters, grouped as its Action combobox lists them. */ +/** The methods the Audit tab offers as filters, grouped as its Action filter lists them. */ export const AUDIT_METHOD_GROUPS: { label: string; methods: readonly AuditMethod[] }[] = [ + { label: 'Workspace calls', methods: ADMIN_WORKSPACE_METHODS }, { label: 'Settings changes', methods: ADMIN_SETTING_METHODS }, ] const isAuditMethod = (value: unknown): value is AuditMethod => AUDIT_METHOD_GROUPS.some(({ methods }) => (methods as readonly unknown[]).includes(value)) +const isSensitivity = (value: unknown): value is AuditSensitivity => + (AUDIT_SENSITIVITIES as readonly unknown[]).includes(value) + +const text = (value: unknown) => typeof value === 'string' && value !== '' ? value : undefined + /** The audit filters `search` holds, leaving out any that is malformed. */ -export const auditFilterOf = (search: Record): AuditFilter => ({ - ...(typeof search.admin === 'string' && search.admin !== '' ? { admin: search.admin } : {}), - ...(isAuditMethod(search.method) ? { method: search.method } : {}), +export const auditFilterOf = (search: Record): AuditFilter => { + const filter: AuditFilter = {} + const admin = text(search.admin) + if (admin !== undefined) filter.admin = admin + if (isAuditMethod(search.method)) filter.method = search.method + if (isSensitivity(search.sensitivity)) filter.sensitivity = search.sensitivity + const workspace = text(search.workspace) + if (workspace !== undefined) filter.workspace = workspace + const owner = text(search.owner) + if (owner !== undefined) filter.owner = owner + return filter +} + +/** Whether any filter is set. */ +export const isFiltered = (filter: AuditFilter) => Object.values(filter).some((value) => value !== undefined) + +/** The listAdminAudit() query for `filter`: a sensitivity is what the records exposed, or their kind. */ +export const auditQueryOf = ({ sensitivity, ...filter }: AuditFilter): AdminAuditQuery => ({ + ...filter, + ...(sensitivity === 'setting' ? { kind: 'setting' } : sensitivity ? { exposed: sensitivity } : {}), }) diff --git a/packages/workshop-frontend/src/features/admin-audit/auditSettings.test.ts b/packages/workshop-frontend/src/features/admin-audit/auditSettings.test.ts index c654a5eb69..14bf5ab48e 100644 --- a/packages/workshop-frontend/src/features/admin-audit/auditSettings.test.ts +++ b/packages/workshop-frontend/src/features/admin-audit/auditSettings.test.ts @@ -2,21 +2,36 @@ import { describe, expect, it } from 'vitest' import { auditSettingsOf, type AuditSettings } from './auditSettings' const complete = (): AuditSettings => ({ + adminWorkspaceAccess: { effective: 'metadata', chosen: 'full', max: 'metadata' }, adminAuditRetentionDays: { effective: 120, chosen: 30, min: 120 }, }) describe('auditSettingsOf', () => { - it('gives the retention from a view that carries it', () => { + it('gives the pair from a view that carries both', () => { expect(auditSettingsOf({ ...complete(), signupsEnabled: true } as Partial)) .toEqual(complete()) }) // What a backend older than the admin audit log sends. - it('gives null for a view lacking it', () => { + it('gives null for a view lacking both', () => { expect(auditSettingsOf({})).toBeNull() }) + it.each(['adminWorkspaceAccess', 'adminAuditRetentionDays'] as const)( + 'gives null for a view lacking %s', (field) => { + const view: Partial = complete() + delete view[field] + expect(auditSettingsOf(view)).toBeNull() + }) + it.each([ + ['an access level that is not an object', { adminWorkspaceAccess: 'full' }], + ['an access level that is not a level', { + adminWorkspaceAccess: { effective: 'metadata', chosen: 'everything', max: 'metadata' }, + }], + ['an access level without its maximum', { + adminWorkspaceAccess: { effective: 'metadata', chosen: 'metadata' }, + }], ['a retention that is not an object', { adminAuditRetentionDays: 90 }], ['a retention whose minimum is not a number', { adminAuditRetentionDays: { effective: 90, chosen: 90, min: '90' }, diff --git a/packages/workshop-frontend/src/features/admin-audit/auditSettings.ts b/packages/workshop-frontend/src/features/admin-audit/auditSettings.ts index aa2bc3ca0e..4ed1ac8fc3 100644 --- a/packages/workshop-frontend/src/features/admin-audit/auditSettings.ts +++ b/packages/workshop-frontend/src/features/admin-audit/auditSettings.ts @@ -1,21 +1,26 @@ -import type { AdminSettingsView } from '@gadgets/workshop-shared/api' +import { isAdminWorkspaceAccess, type AdminSettingsView } from '@gadgets/workshop-shared/api' -/** The deployment settings the admin Audit tab shows and changes: the audit retention. */ -export type AuditSettings = Pick +/** The deployment settings the admin Audit tab shows and changes: the access level and retention. */ +export type AuditSettings = Pick const isRecord = (value: unknown): value is Record => typeof value === 'object' && value !== null /** * The audit settings a settings view carries, or null when it lacks them or holds them malformed. - * A backend older than the admin audit log sends none, and has none of the methods the Audit tab - * calls either, so null means the deployment has no Audit tab yet. + * A backend older than the admin audit log sends neither, and has none of the methods the Audit + * tab calls either, so null means the deployment has no Audit tab yet. */ export const auditSettingsOf = (view: Partial): AuditSettings | null => { + const access: unknown = view.adminWorkspaceAccess const retention: unknown = view.adminAuditRetentionDays - if (!isRecord(retention)) return null + if (!isRecord(access) || !isRecord(retention)) return null + if (![access.effective, access.chosen, access.max].every(isAdminWorkspaceAccess)) return null if (![retention.effective, retention.chosen, retention.min].every((days) => typeof days === 'number')) { return null } - return { adminAuditRetentionDays: view.adminAuditRetentionDays! } + return { + adminWorkspaceAccess: view.adminWorkspaceAccess!, + adminAuditRetentionDays: view.adminAuditRetentionDays!, + } } diff --git a/packages/workshop-shared/src/api.ts b/packages/workshop-shared/src/api.ts index 24f029923b..f27c568256 100644 --- a/packages/workshop-shared/src/api.ts +++ b/packages/workshop-shared/src/api.ts @@ -615,8 +615,51 @@ export const createGatekeeperError = gatekeeperErrors.create; export const getGatekeeperErrorCode = gatekeeperErrors.getCode; /** - * One user as listed in the deployment-wide user directory (see - * `AuthenticatedApi.searchUsers`). + * Stable error codes attached to expected refusals of AdminApi.inspectWorkspace() and of + * AdminWorkspaceView calls. + */ +export const ADMIN_ACCESS_ERROR_CODES = { + workspaceNotFound: "ADMIN_WORKSPACE_NOT_FOUND", + accessOff: "ADMIN_WORKSPACE_ACCESS_OFF", + contentWithheld: "ADMIN_CONTENT_WITHHELD", + viewExpired: "ADMIN_VIEW_EXPIRED", + notAdmin: "ADMIN_NOT_ADMIN", +} as const; + +/** An expected refusal code from ADMIN_ACCESS_ERROR_CODES. */ +export type AdminAccessErrorCode = + typeof ADMIN_ACCESS_ERROR_CODES[keyof typeof ADMIN_ACCESS_ERROR_CODES]; + +const adminAccessErrors = codedErrorFamily({ + [ADMIN_ACCESS_ERROR_CODES.workspaceNotFound]: "Workspace not found.", + [ADMIN_ACCESS_ERROR_CODES.accessOff]: + "Admin access to workspaces is turned off on this deployment.", + [ADMIN_ACCESS_ERROR_CODES.contentWithheld]: + "This view is limited to workspace metadata, so this content is withheld. Content needs a " + + "view whose inspection asked for \"full\" access, on a deployment that allows it: " + + "describe() reports this view's level, and getSettings() the deployment's.", + [ADMIN_ACCESS_ERROR_CODES.viewExpired]: + "This admin view has expired. Inspect the workspace again.", + [ADMIN_ACCESS_ERROR_CODES.notAdmin]: "Only a deployment admin can inspect a workspace.", +}); + +/** Creates an expected admin-access refusal with a machine-readable code. */ +export const createAdminAccessError = adminAccessErrors.create; + +/** Reads the machine-readable code from an expected admin-access refusal. */ +export const getAdminAccessErrorCode = adminAccessErrors.getCode; + +/** + * The most characters of a display name or a workspace title the deployment directory keeps (see + * AuthenticatedApi.searchUsers(), AdminApi.listUsers() and AdminApi.listWorkspaces()): it lists a + * longer one as its first MAX_DIRECTORY_TEXT_LENGTH characters, since nothing else bounds either. + * Ids are kept whole. + */ +export const MAX_DIRECTORY_TEXT_LENGTH = 1000; + +/** + * One user as listed in the deployment-wide user directory (see `AuthenticatedApi.searchUsers` + * and `AdminApi.listUsers`). */ export type UserDirectoryRecord = { /** @@ -625,7 +668,7 @@ export type UserDirectoryRecord = { */ id: string; - /** The user's current display name. */ + /** The user's current display name: its first MAX_DIRECTORY_TEXT_LENGTH characters when longer. */ name: string; }; @@ -1227,11 +1270,12 @@ export interface AuthenticatedApi extends RpcTarget { amIAdmin(): Promise; /** - * Returns a capability for managing deployment-wide admin settings, or null when the caller is not - * an admin. The access check happens once here, so the returned stub's methods need no per-call - * checks. Every audit record the capability makes is labelled with an AdminVia derived from the - * client's User-Agent. (Authentication config — sign-in providers, password login — is - * intentionally not managed here; it stays env-var driven.) + * Returns a capability for managing and investigating the deployment as an admin, or null when + * the caller is not one. The access check happens here, once, for every AdminApi method but + * workspace inspection: AdminApi.inspectWorkspace() and each call on the view it mints check + * again (see AdminWorkspaceView). Every audit record the capability makes is labelled with an + * AdminVia derived from the client's User-Agent. (Authentication config — sign-in providers, + * password login — is intentionally not managed here; it stays env-var driven.) * * @nullWhen the caller is not a deployment admin; amIAdmin() says whether it is */ @@ -1425,6 +1469,23 @@ export type AdminSettingsView = { signupsEnabled: boolean; /** Whether users may search the user directory to find collaborators. */ userSearchEnabled: boolean; + /** + * How much of a workspace an admin may inspect (see AdminApi.setAdminWorkspaceAccess): + * inspectWorkspace() opens each view at the lower of `effective` and the level the inspection + * asks for (AdminInspectOptions.access). + */ + adminWorkspaceAccess: { + /** The level enforced: the lower of `chosen` and `max`. */ + effective: AdminWorkspaceAccess; + /** The level an admin set, DEFAULT_ADMIN_WORKSPACE_ACCESS until one does. Kept, but not + * enforced, while it is above `max`. */ + chosen: AdminWorkspaceAccess; + /** The highest level the deployment allows, which its ADMIN_WORKSPACE_ACCESS_MAX environment + * variable sets: "metadata" while that is unset or blank, and "off" while it holds anything + * that is not a level, so a misconfiguration never grants access. Only a deployment config + * change can raise it. */ + max: AdminWorkspaceAccess; + }; /** * How many days the admin audit log keeps a record (see AdminApi.setAdminAuditRetentionDays): * `effective`. @@ -1606,13 +1667,122 @@ export type DeploymentUpdateStatus = { */ export type AdminVia = "api" | "cli"; -/** How many records one page of AdminApi.listAdminAudit() holds when the caller gives no - * `limit`. */ +/** How many records one page of an admin listing (AdminApi.listUsers(), listWorkspaces(), + * listAdminAudit()) holds when the caller gives no `limit`. */ export const ADMIN_LIST_DEFAULT_LIMIT = 50; -/** The most records one page of AdminApi.listAdminAudit() may hold. */ +/** The most records one page of an admin listing may hold. */ export const ADMIN_LIST_MAX_LIMIT = 200; +/** + * One account of this deployment, as AdminApi.listUsers() lists it. `id` is the username of a + * password account and the email of a sign-in or Cloudflare Access account, kept whole; `name` is + * the display name, at most MAX_DIRECTORY_TEXT_LENGTH characters (see UserDirectoryRecord). + */ +export type AdminUserRecord = UserDirectoryRecord & { + /** When the account was created. Absent for one created before the deployment recorded it. */ + created?: Date; + /** When the user last started a session (signed in, or connected with a stored session), to + * within an hour. Absent until their first session after the deployment began recording it. */ + lastLogin?: Date; + /** Whether the user is a deployment admin now: named in the deployment's ADMINS variable. */ + isAdmin: boolean; +}; + +/** Options for AdminApi.listUsers(). */ +export type AdminUserQuery = { + /** Only users whose id or display name contains this, case-insensitively. At most 1000 + * characters, with no line break; absent or blank lists everyone. */ + query?: string; + /** The previous page's `nextCursor`; absent for the first page. */ + cursor?: string; + /** The page size: a whole number from 1 to ADMIN_LIST_MAX_LIMIT, ADMIN_LIST_DEFAULT_LIMIT when + * absent. */ + limit?: number; +}; + +/** One page of AdminApi.listUsers(). */ +export type AdminUserPage = { + /** The page's users, in id order: at most the query's `limit`, and fewer when their ids and + * names are long (see AdminApi.listUsers()). */ + users: AdminUserRecord[]; + /** Pass as `cursor` for the next page; absent once there is none. */ + nextCursor?: string; +}; + +/** One workspace of this deployment, as AdminApi.listWorkspaces() lists it. */ +export type AdminWorkspaceRecord = { + /** The workspace id, as AdminApi.inspectWorkspace() and AuthenticatedApi.openGadget() take it. */ + id: string; + /** The workspace title, as the workspace last reported it: its first MAX_DIRECTORY_TEXT_LENGTH + * characters when longer. */ + title: string; + /** The owner: their canonical user id, kept whole, and their display name as the directory last + * saw it (their id until they next start a session), at most MAX_DIRECTORY_TEXT_LENGTH + * characters. */ + owner: UserDirectoryRecord; + /** When the workspace was created. Absent until known: an older workspace's is read from its + * owner's records when it is first opened after the deployment began recording workspaces. */ + created?: Date; + /** When the workspace was last active (an agent turn, a code change, an action), to within an + * hour. Absent until it is next active. */ + lastActive?: Date; +}; + +/** Options for AdminApi.listWorkspaces(). */ +export type AdminWorkspaceQuery = { + /** Only workspaces this user owns, by canonical user id (AdminUserRecord.id), matched exactly. */ + owner?: string; + /** Only workspaces whose id, title or owner id contains this, case-insensitively. At most 1000 + * characters, with no line break; absent or blank matches every workspace. */ + query?: string; + /** The previous page's `nextCursor`; absent for the first page. */ + cursor?: string; + /** The page size: a whole number from 1 to ADMIN_LIST_MAX_LIMIT, ADMIN_LIST_DEFAULT_LIMIT when + * absent. */ + limit?: number; +}; + +/** One page of AdminApi.listWorkspaces(). */ +export type AdminWorkspacePage = { + /** The page's workspaces, in id order: at most the query's `limit`, and fewer when their ids, + * titles and owners are long (see AdminApi.listWorkspaces()). */ + workspaces: AdminWorkspaceRecord[]; + /** Pass as `cursor` for the next page; absent once there is none. */ + nextCursor?: string; +}; + +/** + * How much of a workspace a deployment admin may inspect through AdminApi.inspectWorkspace(), in + * increasing order: + * - "off": nothing; inspectWorkspace() is refused. + * - "metadata": its structure: titles, workpieces, collaborators, observers, connections, the + * chat and action lists, file names, commit ids and authors, the names, types, + * sizes and content hashes of attachments, the titles and descriptions of + * blueprints, share links' notes, and diagnostics. Chat messages, queued steers, + * attachments, file contents (at any commit, or pending in a chat), commit + * messages, the arguments of calls queued for an agent, which blueprint a gadget + * follows, the ids of the blueprints the workspace publishes, and the + * descriptions and fields of actions, observations and hooks are withheld. + * Titles, names, notes and blueprint descriptions stay, and can echo content. + * - "full": everything, what "metadata" withholds included. + */ +export const ADMIN_WORKSPACE_ACCESS_LEVELS = ["off", "metadata", "full"] as const; + +/** One of ADMIN_WORKSPACE_ACCESS_LEVELS. */ +export type AdminWorkspaceAccess = typeof ADMIN_WORKSPACE_ACCESS_LEVELS[number]; + +/** Whether `value` is an AdminWorkspaceAccess. */ +export function isAdminWorkspaceAccess(value: unknown): value is AdminWorkspaceAccess { + return ADMIN_WORKSPACE_ACCESS_LEVELS.includes(value as AdminWorkspaceAccess); +} + +/** The access level of a working AdminWorkspaceView: never "off", at which none works. */ +export type AdminViewAccess = Exclude; + +/** The access level a deployment has until an admin sets one. */ +export const DEFAULT_ADMIN_WORKSPACE_ACCESS: AdminWorkspaceAccess = "metadata"; + /** * How many days the admin audit log keeps a record until an admin sets otherwise, and the fewest * an admin may set unless the deployment's ADMIN_AUDIT_RETENTION_MIN_DAYS says otherwise. @@ -1622,11 +1792,59 @@ export const DEFAULT_ADMIN_AUDIT_RETENTION_DAYS = 90; /** The longest retention an admin may set, in days: ten years. */ export const MAX_ADMIN_AUDIT_RETENTION_DAYS = 3650; +/** How long an AdminWorkspaceView lasts unless AdminInspectOptions.lifetimeSeconds says: 15 + * minutes. */ +export const DEFAULT_ADMIN_VIEW_LIFETIME_SECONDS = 900; + +/** The longest an AdminWorkspaceView may last, in seconds: an hour. */ +export const MAX_ADMIN_VIEW_LIFETIME_SECONDS = 3600; + +/** The longest reason AdminApi.inspectWorkspace() takes, in characters, once trimmed. */ +export const MAX_ADMIN_ACCESS_REASON_LENGTH = 500; + +/** How many commits AdminWorkspaceView.getCommitLog() returns when the caller gives no `depth`. */ +export const DEFAULT_ADMIN_COMMIT_LOG_DEPTH = 20; + +/** The most commits one AdminWorkspaceView.getCommitLog() call returns. */ +export const MAX_ADMIN_COMMIT_LOG_DEPTH = 100; + +/** Options for AdminApi.inspectWorkspace(). */ +export type AdminInspectOptions = { + /** How long the view lasts: a whole number of seconds from 1 to + * MAX_ADMIN_VIEW_LIFETIME_SECONDS, DEFAULT_ADMIN_VIEW_LIFETIME_SECONDS when absent. */ + lifetimeSeconds?: number; + /** + * The level asked for: "metadata" when absent. The view gets the lower of this and the + * deployment's level (AdminSettingsView.adminWorkspaceAccess.effective). Asking for "full" where + * the deployment allows only "metadata" is not refused: the view opens at "metadata", and its + * describe() reports that. While the deployment's level is "off", every open is refused, whatever + * it asks for. Every record the view makes keeps the level asked for + * (AdminAccessRecord.requested). An investigation therefore starts at "metadata" and asks for + * "full" deliberately, in a new inspectWorkspace() call with a reason of its own, only to read + * content. The audit log shows when it did. + */ + access?: AdminViewAccess; +}; + +/** The workspace calls the audit log records: the open (AdminApi.inspectWorkspace()) and each + * AdminWorkspaceView method. */ +export const ADMIN_WORKSPACE_METHODS = [ + "inspectWorkspace", "describe", "listChats", "getChat", "getChatState", "listChatAttachments", + "readChatAttachment", "listActions", "listFiles", "readFile", "getCommitLog", + "listFilesAtCommit", "readFileAtCommit", "listChangedPaths", "listPendingChanges", + "readPendingFile", "listGatekeepers", "listObservers", "listObserverRequirements", + "listShareLinks", "listAutoApprovals", "listBlueprints", "getDiagnostics", +] as const satisfies readonly ("inspectWorkspace" | keyof AdminWorkspaceView)[]; + +/** One of ADMIN_WORKSPACE_METHODS. */ +export type AdminWorkspaceMethod = typeof ADMIN_WORKSPACE_METHODS[number]; + /** * The AdminApi methods that change a deployment setting, each recorded as an AdminSettingRecord - * before its change takes effect. The rest change no setting and are not recorded: the reads - * (getSettings, isBlueprintFeatured, getUpdateStatus, listAdminAudit, getAdminAuditStats), the - * gateway tests, and checkForUpdates. + * before its change takes effect. inspectWorkspace() is recorded as an AdminAccessRecord. The rest + * change no setting and are not recorded: the reads (getSettings, isBlueprintFeatured, + * getUpdateStatus, listUsers, listWorkspaces, listAdminAudit, getAdminAuditStats), the gateway + * tests, and checkForUpdates. */ export const ADMIN_SETTING_METHODS = [ "setSignupsEnabled", "setUserSearchEnabled", "setSiteName", "setSiteLogo", @@ -1637,14 +1855,16 @@ export const ADMIN_SETTING_METHODS = [ "setGatewayModelSettings", "setDefaultReasoning", "setGatewayProviderEnabled", "setUpdateChecksEnabled", "setUpdateCheckIntervalMinutes", "setUpdateMinimumAgeHours", "setUpdateNoticeSnoozeHours", "setRestrictedWebFetchHosts", "setRestrictedWebFetchApprovals", - "setAdminAuditRetentionDays", "setGatewayProviderOrder", "setGatewayModelOrder", + "setAdminWorkspaceAccess", "setAdminAuditRetentionDays", "setGatewayProviderOrder", + "setGatewayModelOrder", ] as const satisfies readonly (keyof AdminApi)[]; /** One of ADMIN_SETTING_METHODS. */ export type AdminSettingMethod = typeof ADMIN_SETTING_METHODS[number]; -/** The longest setting entry's key an audit record keeps, in characters: a longer one is cut to - * this, its last character replaced with "…". */ +/** The longest string an audit record keeps as a call's argument, a workspace's title or a + * setting entry's key, in characters: a longer one is cut to this, its last character replaced + * with "…". */ export const MAX_ADMIN_AUDIT_ARG_LENGTH = 256; /** The longest value an AdminSettingChange keeps whole, in characters: more than any bounded @@ -1653,6 +1873,98 @@ export const MAX_ADMIN_AUDIT_ARG_LENGTH = 256; * an excerpt (see AdminSettingChange.excerpt). */ export const MAX_ADMIN_AUDIT_VALUE_LENGTH = 10_000; +/** + * What one admin call into a workspace answered with (see AdminAccessRecord.exposed): "content" + * when its answer held something the "metadata" access level withholds (see + * ADMIN_WORKSPACE_ACCESS_LEVELS), "metadata" when it held none of it. + */ +export type AdminAccessExposure = "metadata" | "content"; + +/** + * One admin call into a workspace: the open that minted an AdminWorkspaceView, or a call made + * through one. It is recorded once the call has read what it answers with, and before any of that + * leaves the server, so that the record can say what the answer exposed. It is recorded in the + * deployment-wide log behind AdminApi.listAdminAudit(), never in the workspace, so nothing a + * workspace member can read ever shows it. A call the view itself stops + * is not recorded. That covers a call made after the view expired, after its admin stopped being an + * admin or after the workspace was deleted, and a call that any of these overtakes while it reads. + * A call the access level refuses is recorded, with `refused` set, and so is a call whose read + * fails, and so is an open refused before it reaches the workspace, because the deployment's access + * level is "off". + */ +export type AdminAccessRecord = { + /** Distinguishes this from an AdminSettingRecord. */ + kind: "workspace"; + /** When the call was recorded: once it had read its answer (or failed to), or as it was + * refused. */ + at: Date; + /** The admin who made it, by canonical user id. */ + admin: string; + /** Opaque. Shared by every record made through one AuthenticatedApi.getAdminApi() capability, + * so one admin session's records can be read together. */ + sessionId: string; + /** How the admin's calls reached the deployment (see AdminVia). */ + via: AdminVia; + /** The workspace. */ + workspaceId: string; + /** The workspace's owner, by canonical user id, as the workspace knew them when the view opened. + * A username has no length limit of its own, so an id over MAX_ADMIN_AUDIT_ARG_LENGTH characters + * is kept as its start, "…#" and a digest of the whole id, that many characters in all: two + * long ids stay apart. For an open refused before it reached the workspace, the owner the + * deployment directory lists, kept the same way, "" when it lists none. */ + workspaceOwner: string; + /** The workspace's title when the call was made, cut like an argument (see + * MAX_ADMIN_AUDIT_ARG_LENGTH), since a title has no length limit of its own. For an open refused + * before it reached the workspace, the title the deployment directory lists, cut the same way, + * "" when it lists none. */ + workspaceTitle: string; + /** "inspectWorkspace" for the open, else the view method called. */ + method: AdminWorkspaceMethod; + /** The call's arguments by name: only those it declares, an absent optional one left out, each + * string cut to MAX_ADMIN_AUDIT_ARG_LENGTH. The open records `expiresAt` (ISO 8601), and + * listObserverRequirements() its grant as `grant: "workspace"` or the gadget's `gadgetId`. No + * view method takes content, so none is recorded. */ + args: Record; + /** The reason the admin gave inspectWorkspace() for the view. */ + reason: string; + /** The level the view's open asked for (AdminInspectOptions.access; "metadata" when it named + * none). It is the same on every record of one view, so the log shows when an investigation asked + * for more. */ + requested: AdminViewAccess; + /** The level the call ran at: the view's own level (the lower of `requested` and the + * deployment's level when the view was opened), lowered to the deployment's setting at the time + * (see AdminWorkspaceView); "off" when the call was refused for that. Never above `requested`. */ + access: AdminWorkspaceAccess; + /** + * What the call's answer exposed. Only a call at "full" can record "content", and only when its + * answer held something the "metadata" level withholds: + * - getChat(): chat messages or a compaction summary; + * - readChatAttachment(): any attachment found; + * - readFile(), readFileAtCommit(), readPendingFile(): any file found at the path, + * `unreadable` ones included, because that message can quote the file (a symlink's quotes + * its target); + * - listChats(): a queued steer; + * - getChatState(): a queued call's argument summary; + * - getCommitLog(): a commit message; + * - describe(): the blueprint a gadget follows; + * - listBlueprints(): a blueprint's id; + * - listActions(), listGatekeepers(): a non-empty description, or a field, of an action, + * observation or hook. + * Otherwise "metadata": always for the open and for the calls whose answers hold nothing the + * level withholds (listChatAttachments(), listFiles(), listFilesAtCommit(), listChangedPaths(), + * listPendingChanges(), listObservers(), listObserverRequirements(), listShareLinks(), + * listAutoApprovals() and getDiagnostics()), for a call that was refused or whose read failed, + * and for a call at "full" whose answer held none of these. It is set from the answer itself, + * never from the level. An answer leaves the server only once its record exists: a call whose + * record could not be written returned nothing. + */ + exposed: AdminAccessExposure; + /** Present when the access level refused the call, which then read nothing: the code it was + * refused with. */ + refused?: typeof ADMIN_ACCESS_ERROR_CODES.accessOff + | typeof ADMIN_ACCESS_ERROR_CODES.contentWithheld; +}; + /** * One setting an AdminSettingRecord changed, with its value before and after. `setting` names a * field of the deployment's stored settings ("siteName", "modelModes", ...) or one of the two kept @@ -1667,10 +1979,10 @@ export const MAX_ADMIN_AUDIT_VALUE_LENGTH = 10_000; * * A string is recorded as itself, any other value as JSON with object keys sorted. A value is kept * whole up to MAX_ADMIN_AUDIT_VALUE_LENGTH characters, more than any bounded setting holds; past - * that both values are excerpts (see `excerpt`). A key is cut to MAX_ADMIN_AUDIT_ARG_LENGTH. No - * credential is a setting (provider keys and tokens live in the environment), but a free-text - * setting such as the agent instructions is recorded as an admin entered it. A logo is recorded - * as its size and SHA-256, never its bytes. + * that both values are excerpts (see `excerpt`). A key is cut like an argument (see + * MAX_ADMIN_AUDIT_ARG_LENGTH). No credential is a setting (provider keys and tokens live in the + * environment), but a free-text setting such as the agent instructions is recorded as an admin + * entered it. A logo is recorded as its size and SHA-256, never its bytes. */ export type AdminSettingChange = { /** The setting. */ @@ -1714,7 +2026,7 @@ export type AdminSettingRecord = { /** Opaque. Shared by every record made through one AuthenticatedApi.getAdminApi() capability, * so one admin session's records can be read together. */ sessionId: string; - /** How the admin's calls reached the deployment (see AdminVia). */ + /** How the admin's calls reach the deployment (see AdminVia). */ via: AdminVia; /** The AdminApi method called. */ method: AdminSettingMethod; @@ -1724,9 +2036,10 @@ export type AdminSettingRecord = { /** One record of the deployment-wide admin audit log, kept for the deployment's retention * (AdminSettingsView.adminAuditRetentionDays). */ -export type AdminAuditRecord = AdminSettingRecord; +export type AdminAuditRecord = AdminAccessRecord | AdminSettingRecord; -/** The kinds of AdminAuditRecord: "setting" for a change to the deployment's settings. */ +/** The kinds of AdminAuditRecord: "workspace" for access to a workspace, "setting" for a change to + * the deployment's settings. */ export type AdminAuditKind = AdminAuditRecord["kind"]; /** Options for AdminApi.listAdminAudit(). The filters combine. */ @@ -1737,6 +2050,15 @@ export type AdminAuditQuery = { admin?: string; /** Only records of calls to this method. */ method?: AdminAuditRecord["method"]; + /** Only records of access to this workspace. At most 1000 characters. */ + workspace?: string; + /** Only records of access to workspaces this user owns (see AdminAccessRecord.workspaceOwner), by + * canonical user id, whole or as the records keep it. At most 1000 characters. */ + owner?: string; + /** Only records of calls into a workspace whose answer exposed this (see + * AdminAccessRecord.exposed): "content" lists the content reads. Only access records carry it, + * so no setting record matches. */ + exposed?: AdminAccessExposure; /** Only records made at or after this time. */ since?: Date; /** The previous page's `nextCursor`; absent for the first page. */ @@ -1762,14 +2084,15 @@ export type AdminAuditStatsQuery = { /** The window: the last `days` UTC days, today included. A whole number from 1 to * MAX_ADMIN_AUDIT_STATS_DAYS, 7 when absent. */ days?: number; - /** Only records of this kind; every kind when absent. */ + /** Only records of this kind; both kinds when absent. */ kind?: AdminAuditKind; }; /** * What the admin audit log holds for one window, from AdminApi.getAdminAuditStats(). Every count - * covers the records of the queried kind (every kind when none is given). A record older than the - * retention is gone, so a window reaching past it counts nothing there. + * covers the records of the queried kind (both kinds when none is given). Only AdminAccessRecords + * can be content reads, so those are zero for the "setting" kind. A record older than the retention + * is gone, so a window reaching past it counts nothing there. */ export type AdminAuditStats = { /** The window's length in UTC days, today included. */ @@ -1779,18 +2102,25 @@ export type AdminAuditStats = { /** One entry per UTC day of the window, oldest first, days with no records included. `day` is * the date as YYYY-MM-DD. */ byDay: { day: string; count: number }[]; - /** The admins with the most records in the window, most first (ties by id), at most 10. */ - topAdmins: { admin: string; count: number }[]; + /** The admins with the most records in the window, most first (ties by id), at most 10, each + * with how many of those records are content reads. Every content read is listed by + * listAdminAudit({ exposed: "content" }), an admin's who is not among these included. */ + topAdmins: { admin: string; count: number; contentReads: number }[]; + /** The calls into a workspace in the window whose answer exposed content + * (AdminAccessRecord.exposed is "content"). */ + contentReads: number; }; /** - * Capability for managing deployment-wide admin settings, obtained via + * Capability for managing and investigating the deployment, obtained via * AuthenticatedApi.getAdminApi() (which is null for non-admins). The access check happens when the - * capability is minted, so these methods don't re-check. Covers branding, agent instructions, - * which gatekeeper connectors/resources are offered, and the models an AI Gateway deployment - * provides — NOT authentication config (that's env-var driven). Each setter throws on invalid - * input, and records the change it makes in the deployment's audit log before making it (see - * AdminSettingRecord). + * capability is minted, so these methods don't re-check, with one exception: inspectWorkspace() + * re-checks, and so does every call on the AdminWorkspaceView it mints, since a view is the one + * thing an admin holds that reads users' data. Covers branding, agent instructions, which + * gatekeeper connectors/resources are offered, the models an AI Gateway deployment provides, the + * user and workspace directory, and admin access to workspaces — NOT authentication config (that's + * env-var driven). Each setter throws on invalid input, and records the change it makes in the + * deployment's audit log before making it (see AdminSettingRecord). */ export interface AdminApi { /** Read all admin-managed settings for the admin UI in one call. */ @@ -2111,12 +2441,59 @@ export interface AdminApi { */ setUpdateNoticeSnoozeHours(hours: number): Promise; - // --- The audit log --- + // --- Users, workspaces and admin access --- + // + // listUsers() and listWorkspaces() read the deployment-wide directory that user search reads + // (see setUserSearchEnabled). Durable Objects cannot be enumerated, so the directory fills as + // accounts and workspaces are used. An account is listed once it has started a session since the + // directory began recording accounts for user search, so most are listed at once, without + // `lastLogin` until their next session; one with no session since then is listed at its next. A workspace is listed when it is next opened. One never used again is never listed. + + /** + * One page of the deployment's users, in id order: every account that has started a session + * since the directory began recording accounts. Unaffected by setUserSearchEnabled(). A page + * holds at most `limit` users and stops early, with a `nextCursor`, once their ids and names + * would pass a million characters: an id has no length limit of its own, so a count alone cannot + * bound a page. Rejects a `query` over 1000 characters or with a line break, and a `limit` that + * is not a whole number from 1 to ADMIN_LIST_MAX_LIMIT. + */ + listUsers(options?: AdminUserQuery): Promise; /** - * One page of the deployment-wide admin audit log, newest first, filtered as `options` says. - * Rejects a `limit` that is not a whole number - * from 1 to ADMIN_LIST_MAX_LIMIT, a filter over 1000 characters, and a malformed `cursor`. + * One page of the deployment's workspaces, in id order: every workspace opened since the + * directory began recording workspaces, deleted ones left out. A page stops early as + * listUsers()'s does, counting ids, titles and owners. Rejects what listUsers() rejects. + */ + listWorkspaces(options?: AdminWorkspaceQuery): Promise; + + /** + * Open a read-only view of a workspace to investigate it (see AdminWorkspaceView). The view runs + * at the lower of the level `options.access` asks for ("metadata" when absent) and the + * deployment's level (AdminSettingsView.adminWorkspaceAccess.effective). Asking for "full" where + * the deployment allows only "metadata" opens the view at "metadata", and its describe() reports + * that. `reason` is required: it is recorded with the open and with every call made through the + * view, and is rejected when blank or, once trimmed, longer than MAX_ADMIN_ACCESS_REASON_LENGTH. + * The open is recorded as an AdminAccessRecord ("inspectWorkspace") before the view is returned, + * so an open whose record cannot be written is refused. + * + * Refused with a code from ADMIN_ACCESS_ERROR_CODES, checked in this order: `notAdmin` once the + * caller is no longer a deployment admin; `workspaceNotFound` for a malformed id; `accessOff` + * while the access level is "off", recorded in the deployment-wide log without the workspace + * being reached; `workspaceNotFound` for an id that names no workspace (never initialised or + * since deleted), which this neither creates nor claims; `viewExpired` should the view's + * lifetime run out before the workspace answers. Only `accessOff` refusals are recorded. + * + * Admin access is a deliberate exception to observer verification (see docs/observers.md): the + * admin is never verified against the workspace's connections nor registered as an observer, and + * the audit log is the compensating control. + */ + inspectWorkspace(workspaceId: string, reason: string, options?: AdminInspectOptions) + : Promise>; + + /** + * One page of the deployment-wide admin audit log, newest first, filtered as `options` says. It + * holds access to workspaces and changes to settings. Rejects a `limit` that is not a whole + * number from 1 to ADMIN_LIST_MAX_LIMIT, a filter over 1000 characters, and a malformed `cursor`. */ listAdminAudit(options?: AdminAuditQuery): Promise; @@ -2124,6 +2501,15 @@ export interface AdminApi { * default). Throws unless `days` is a whole number from 1 to MAX_ADMIN_AUDIT_STATS_DAYS. */ getAdminAuditStats(options?: AdminAuditStatsQuery): Promise; + /** + * Set how much of a workspace inspectWorkspace() may show (see ADMIN_WORKSPACE_ACCESS_LEVELS), + * DEFAULT_ADMIN_WORKSPACE_ACCESS by default. Throws for a level above the deployment's maximum + * (AdminSettingsView.adminWorkspaceAccess.max), which only its ADMIN_WORKSPACE_ACCESS_MAX + * environment variable can raise. A view already minted runs at the lower of its own level and + * the new one from its next call once the setting reaches it, within about a minute. + */ + setAdminWorkspaceAccess(access: AdminWorkspaceAccess): Promise; + /** * Set how many days the admin audit log keeps a record, DEFAULT_ADMIN_AUDIT_RETENTION_DAYS by * default. A record older than that is no longer listed or counted, and is deleted as the log is @@ -2135,6 +2521,378 @@ export interface AdminApi { setAdminAuditRetentionDays(days: number): Promise; } +/** A workspace as AdminWorkspaceView.describe() presents it. */ +export type AdminWorkspaceDescription = Pick & { + /** The owner's profile. */ + owner: AiChatAuthorInfo; + /** Every workpiece summary a "build" session sees (see WorkpieceSummary): gadgets, pending ones + * included, and worktrees. Below "full" a gadget's `upstream` is left out: a blueprint id is a + * share link to code the view would otherwise withhold. */ + workpieces: WorkpieceSummary[]; + /** Everyone the workspace is shared with, as Overseer.listCollaborators() lists them. */ + collaborators: CollaboratorInfo[]; + /** The level this call ran at: the view's own level (the lower of the level its open asked for + * and the deployment's level then), lowered to the deployment's current setting (see + * AdminWorkspaceView). */ + access: AdminViewAccess; + /** When this view stops working. */ + expiresAt: Date; +}; + +/** One connection (gatekeeper workpiece) of a workspace, as AdminWorkspaceView.listGatekeepers() + * lists it. */ +export type AdminGatekeeperInfo = { + /** Its workpiece id. */ + id: WorkpieceId; + /** The connected resource's title, as the workspace records it. */ + resourceTitle?: string; + /** The connected resource's URL, when known. */ + resourceUrl?: string; + /** The vendor whose gatekeeper serves it. Absent for one with no vendor (a model, a spawner). */ + vendorId?: string; + /** How it was created. Absent for one created before this was recorded. */ + creationSpec?: GatekeeperCreationSpec; + /** Who created it: the member who added it, or whoever started the agent turn that created its + * resource. Absent for an ambient connection (`creationSpec.type` "ambient"), which the + * workspace takes from a singleton account its owner holds and no person creates, and for a + * connection whose creator was not recorded, as one made before creators were recorded. */ + createdBy?: AiChatAuthorInfo; + /** The gadget bindings that point at it. `chatId` is present while one is pending in that chat. */ + bindings: { gadgetId: WorkpieceId; name: string; chatId?: number }[]; + /** The hooks it delivers, as Overseer.listHooks() lists them; below "full" each has an empty + * `description.description`. */ + hooks: BoundHookInfo[]; +}; + +/** A workspace's operational state, as AdminWorkspaceView.getDiagnostics() reports it. None of it + * is content. */ +export type AdminWorkspaceDiagnostics = { + /** The version of the workspace's storage schema. */ + storageVersion: number; + /** When the workspace's current Durable Object instance started. */ + instanceStartedAt: Date; + /** The members' sessions open now, by kind ("owner", or a collaborator's role). Admin views are + * never counted. */ + sessions: { owner: number; build: number; use: number }; + /** The chats with an agent turn recorded, running or paused; getChatState() says which. */ + activeAgentChatIds: number[]; + /** Calls to callable agents recorded but not yet in their chat. */ + pendingAgentCalls: number; + /** Actions waiting for approval. */ + pendingActions: number; + /** The workspace's chats. */ + chats: number; + /** The workspace's hooks, and how many are enabled. */ + hooks: { total: number; enabled: number }; + /** How many collaborators hold an observer record (see AdminObserver, which says what one + * means); listObservers() lists them. */ + observers: number; + /** The size of the workspace's storage, in bytes. */ + storageBytes: number; + /** Who has the workspace open now, as its presence roster lists them (see + * Overseer.subscribeToPresence()). An admin view is never among them. */ + present: PresenceParticipant[]; + /** When the workspace's alarm next fires. Absent while none is set. */ + nextAlarm?: Date; + /** Whether an external message created the workspace before its owner's account recorded it. */ + ownerRegistrationPending: boolean; +}; + +/** One call made on a chat's callable agent that is not in the chat yet, as + * AdminChatState.queuedCalls lists it. */ +export type AdminQueuedAgentCall = { + /** Its id, unique in the workspace. A chat's calls reach it in id order. */ + callId: number; + /** The method called on the agent. */ + methodName: string; + /** The model the call's turn is to run on. Null when the agent's spawner names none. */ + initiatorModelId: string | null; + /** A short summary of the call's arguments. Its caller wrote them, so below "full" it is "". */ + argsSummary: string; +}; + +/** + * What a workspace records of one chat's agent turn and of what waits on the chat, as + * AdminWorkspaceView.getChatState() reports it: the state behind a chat that looks stuck. The + * chat's title, token counts, cost and code pins are in listChats(). + */ +export type AdminChatState = { + /** The agent turn recorded for the chat: one running, or one paused. Absent when no turn is + * recorded. */ + turn?: { + /** The model the turn runs on. */ + modelId: string; + /** Who started the turn: a user, or a gadget whose spawner or call started it. */ + initiator: AiChatAuthorInfo; + /** Whether a call to the agent started the turn, rather than a chat message. */ + callbackInitiated: boolean; + /** How many times the workspace has looked up the turn's model since it last prepared the + * turn to run. It grows while restarts keep finding the turn unfinished. */ + resumeAttempts: number; + /** Whether the turn is paused: recorded, while the chat names no agent as responding + * (AiChatMetadata.activeAgent), as when it waits for a decision on an action. */ + paused: boolean; + }; + /** When the newest entry of the chat's log was written (a message, an agent step, a change, an + * action): the only time the workspace keeps of a turn's progress. Absent for a chat with no + * entry. */ + lastMessageAt?: Date; + /** The calls made on the chat's agent that are not in the chat yet, oldest first. */ + queuedCalls: AdminQueuedAgentCall[]; + /** The ids of the actions waiting for approval that were submitted from the chat, oldest first; + * listActions() holds each. */ + pendingActionIds: number[]; +}; + +/** One attachment of a chat message, as AdminWorkspaceView.listChatAttachments() lists it: the + * message's ChatAttachmentRef without its bytes. Its `oid` is a hash of those bytes, so it tells + * whether two attachments are the same file; like the name, the type and the size, it shows at + * every level. */ +export type AdminChatAttachment = + Pick & { + /** The sequence of the message it is attached to (AiChatMessage.sequence). */ + sequence: number; +}; + +/** + * One gadget or worktree a chat holds changes to that are not accepted yet, as + * AdminWorkspaceView.listPendingChanges() lists it, with the paths the chat's current content + * changes against the commit those changes apply on top of. Paths are names: readPendingFile() + * reads a changed file, and readFileAtCommit() its base. + */ +export type AdminPendingChange = { + /** The gadget or worktree. */ + gadgetId: WorkpieceId; + /** The commit the chat's changes to it apply on top of (ChatGadgetPin.baseCommit). Absent for a + * gadget the chat created that is not accepted yet: it has no commit, and each of its files is + * in `added`. */ + baseCommit?: string; + /** The accepted code the chat's content includes (ChatGadgetPinState.mergedCommit). Absent + * with `baseCommit`. Accepting changes the workpiece from this commit to the chat's content, + * while the paths here compare that content with `baseCommit`. So where the two commits + * differ, as when the pin is rooted at a merge commit (after an update from mainline, or for a + * proposed blueprint merge), listChangedPaths(mergedCommit, baseCommit) is the rest of the + * comparison, and the three lists can be empty while there is a change to accept. */ + mergedCommit?: string; + /** The paths the chat's content holds that `baseCommit` lacks, sorted. */ + added: string[]; + /** The paths whose content differs from `baseCommit`'s, sorted. */ + modified: string[]; + /** The paths `baseCommit` holds that the chat's content lacks, sorted. */ + removed: string[]; +}; + +/** The account an observer chose for one connection, as AdminObserver.accountChoices lists it. */ +export type AdminObserverAccountChoice = ObserverAccountChoice & { + /** The vendor whose gatekeeper serves the connection, in lower case as + * AdminGatekeeperInfo.vendorId gives it (ObserverBindingNeed.vendorId keeps the vendor's own + * spelling). Absent once the connection is removed. */ + vendorId?: string; +}; + +/** + * One collaborator who holds an observer record, as AdminWorkspaceView.listObservers() lists + * them. A workspace reserves the record when a collaborator first opens what was shared with them + * while a connection it reaches needs an account, before anything is verified, and keeps it until + * they lose access, whether or not that open was admitted (see docs/observers.md). So a record + * does not say its holder was admitted: `accountChoices` says what was verified. + */ +export type AdminObserver = { + /** Who they are: their profile as the workspace's collaborator list keeps it, or one made of + * their id alone when it keeps none. */ + profile: AiChatAuthorInfo; + /** The account they chose for each connection, by connection id, each recorded once an + * admission verified it. Empty means nothing is verified yet: no admission of theirs has + * passed. An entry is what an admission verified then, and every open checks it again. An + * `accountId` names a connected account in their own user account, so the workspace knows its + * id and nothing else of it. */ + accountChoices: AdminObserverAccountChoice[]; +}; + +/** + * One auto-approval rule of a workspace, as AdminWorkspaceView.listAutoApprovals() lists it: a + * member's opt-in for actions of `actionKind` on the connection. A rule alone applies nothing. An + * action of that kind is applied without a manual approval only when its author also marked it + * `autoApprovable` (ActionDescription.autoApprovable), and never once the workspace has read + * restricted data (`containsRestrictedData`, which describe() reports): the rule then stays + * stored, and listed here, while every action waits for a manual approval. + */ +export type AdminAutoApproval = { + /** The connection the rule is on. */ + gatekeeperId: WorkpieceId; + /** The kind of action the rule covers. An action matches by its kind's `tag`. */ + actionKind: ActionKind; + /** Who turned the rule on. The actions it applies are applied under their authority. */ + enabledBy: AiChatAuthorInfo; +}; + +/** + * A deployment admin's read-only view of one workspace, minted by AdminApi.inspectWorkspace(). + * Nothing it does changes the workspace or is seen by its members: it joins no presence and no + * session count, and starts no agent or connection. + * + * Each call first passes three checks, refused unrecorded with a code from + * ADMIN_ACCESS_ERROR_CODES: the view has not expired (`viewExpired`), its admin is still a + * deployment admin (`notAdmin`), and the workspace still exists (`workspaceNotFound`). It then + * runs at an access level: the view's own, fixed when it was minted at the lower of the level its + * open asked for (AdminInspectOptions.access) and the deployment's, then lowered to the + * deployment's current setting. So lowering the setting reaches live views (within about a minute) + * and raising it does not. A call that level refuses is recorded and refused, having read nothing: + * `accessOff` once the setting is "off", `contentWithheld` below "full" for a method that reads + * only content (getChat(), readChatAttachment(), readFile(), readFileAtCommit(), + * readPendingFile()). Any other call reads its answer, never through a connection: what a + * connection supplied is read only as the workspace stores it (the commits of a worktree checked + * out through one are that repository's), on the level's terms, and the connection is not asked. + * It is then recorded as an + * AdminAccessRecord, with what that answer exposed, in the deployment-wide log. Only then is the + * call answered: nothing it read leaves the server before its record exists, and a call whose + * record cannot be written fails and returns nothing. The + * checks run again once the call has read the deployment's setting, and again once it has read its + * answer, just before it writes. So a call that expiry, a demotion or the workspace's deletion + * overtakes while it reads writes nothing. A call whose read fails is recorded as an attempt that + * exposed nothing, and fails with the read's error. + * + * A workspace restart (a revocation, a deletion, a redeploy) ends the view along with every other + * session on the workspace; inspect again. + */ +export interface AdminWorkspaceView extends RpcTarget { + /** The workspace's metadata, owner, workpieces and collaborators, with the level this call ran + * at and the view's expiry. */ + describe(): Promise; + + /** The workspace's chats, newest first, as Overseer.listChats() lists them; below "full", without + * `queuedSteers`, which hold message text. */ + listChats(): Promise; + + /** One page of a chat's messages, as Overseer.getChatHistory() pages them. Content: refused below + * "full". */ + getChat(chatId: number, beforeSequence?: number): Promise; + + /** What is recorded of a chat's agent turn, its newest entry, the calls queued for its agent + * and its pending actions (see AdminChatState). Below "full" each queued call's `argsSummary` + * is "". Throws for an id that names no chat. */ + getChatState(chatId: number): Promise; + + /** The attachments of a chat's messages, oldest message first, without their bytes. Their + * names, types, sizes and content hashes (`oid`) stay at every level, as titles do. Empty for + * an id that names no chat. */ + listChatAttachments(chatId: number): Promise; + + /** One attachment's bytes, as Overseer.getChatAttachmentContent() returns them, read only from + * what is stored. Content: refused below "full". Throws for an id that names no attachment of + * the chat. */ + readChatAttachment(chatId: number, id: string): Promise; + + /** + * One page of the workspace's action log, as Overseer.listActions() pages it. Admin access is + * never in it (see AdminAccessRecord). Below "full", every action, observation and hook entry has + * an empty `description.description` and no `description.fields`: titles, states, resources and + * times remain. + */ + listActions(options?: { beforeId?: number; filter?: ActionHistoryFilter }) + : Promise; + + /** + * The tree at a committed head, as Overseer.listTree() gives it, read only from the objects the + * workspace stores. It never pulls through a connection, and throws when an object is missing. + * Only a permanent gadget has a head to read: throws for a pending gadget, a worktree, or an id + * that names no gadget. + */ + listFiles(gadgetId: WorkpieceId): Promise; + + /** One file of a gadget's committed head, as Overseer.readFilesAtCommit() reports it, read as + * listFiles() reads and refused as it is. Content: refused below "full". */ + readFile(gadgetId: WorkpieceId, path: string): Promise; + + /** + * The history of a stored commit, as Overseer.getCommitLog() walks it: `fromCommit` first, then + * its ancestry, at most `depth` commits (a whole number from 1 to MAX_ADMIN_COMMIT_LOG_DEPTH, + * DEFAULT_ADMIN_COMMIT_LOG_DEPTH when absent). Any commit the workspace stores can start it: a + * gadget's head (WorkpieceSummary.commitId), a worktree's commits, a chat pin's base. Read only + * from the objects the workspace stores: a parent that is not stored, as below the commit a + * worktree was checked out at, ends that line of the walk and is never pulled. Below "full" + * each `message` is "": a commit message describes its change. + */ + getCommitLog(fromCommit: string, depth?: number): Promise; + + /** The tree at any stored commit, read as listFiles() reads a committed head. Any commit the + * workspace stores can be read: a version in a gadget's history, a worktree's commit, a chat + * pin's base. Throws for an id that names no stored commit, and when an object is missing. */ + listFilesAtCommit(commitId: string): Promise; + + /** One file at any stored commit, read as readFile() reads a committed head and refused as + * listFilesAtCommit() is. Content: refused below "full". */ + readFileAtCommit(commitId: string, path: string): Promise; + + /** The paths that differ between two stored commits, as Overseer.listChangedPaths() lists + * them, read only from stored objects. Paths are names: the difference itself is two + * readFileAtCommit() calls. */ + listChangedPaths(fromCommit: string, toCommit: string): Promise; + + /** + * The paths a chat's current content adds, modifies and removes against each pin's base, per + * workpiece it holds changes to (see AdminPendingChange), in workpiece id order. A workpiece + * here is a gadget or a worktree; a gadget the chat created that is not accepted yet has no + * pin, and lists every file as added. It is what the chat changed on top of each base, which is + * not always all that accepting would change (see AdminPendingChange.mergedCommit). Folded from + * the chat's content, edits not yet recorded in a message included, and the objects the + * workspace stores. Throws for an id that names no chat. + */ + listPendingChanges(chatId: number): Promise; + + /** + * One file of a chat's current content for a workpiece it holds changes to (one that + * listPendingChanges() lists), shaped as readFile()'s answer. Text arrives as `text`, or, over + * MAX_BUFFERED_BLOB_SIZE, as a `stream` of its UTF-8 bytes, as readFile() sends a large file; + * a file the chat holds as bytes arrives as a `stream` whatever its size. `absent` when the + * chat's content has no file at the path: one the chat removed, or in a worktree one it has + * not touched, which readFileAtCommit() reads at the change's `baseCommit`. Content: refused + * below "full". Throws for an id that names no chat, and when the chat holds no changes to the + * workpiece. + */ + readPendingFile(chatId: number, gadgetId: WorkpieceId, path: string): Promise; + + /** The workspace's connections, oldest first, with their creators, bindings and hooks. */ + listGatekeepers(): Promise; + + /** The collaborators holding an observer record, with the account each chose for each + * connection (see AdminObserver). A record with no choice is one whose admission has not + * passed: nothing is verified for that collaborator yet. */ + listObservers(): Promise; + + /** The connections a recipient of `grant` would have to pass before opening what it grants, as + * Overseer.listObserverRequirements() lists them. It is what sharing the workspace, or one + * gadget, asks of the recipient: an account that each of those connections verifies. Read from + * the workspace's own records: no connection is asked. Throws for a gadget that cannot be + * shared, and when a connection `grant` reaches was made before connections could verify + * collaborators: any such connection for "workspace", and for a gadget only one it reaches. */ + listObserverRequirements(grant: CollaboratorGrant): Promise; + + /** The active share links, as Overseer.listShareLinks() lists them. Each has its id, note, + * creator, role and gadget, and never a key that redeems it. */ + listShareLinks(): Promise; + + /** The auto-approval rules, each with its connection, its kind of action and who turned it on. + * A rule is a member's opt-in and applies nothing alone (see AdminAutoApproval): an action is + * applied without a manual approval only when its author also marked it `autoApprovable`, and + * never once the workspace has read restricted data (`containsRestrictedData`, which + * describe() reports), when every action waits whatever the rules say. listActions() shows + * each action's `description.autoApprovable` and `description.actionKind`, and `autoApproved` + * on one a rule applied. */ + listAutoApprovals(): Promise; + + /** The published blueprints, as Overseer.listBlueprints() lists them. Below "full" each `id` + * is "" and `screenshotUrl` is left out: a blueprint's id is a share link to its code. Titles + * and descriptions stay at every level, as a workspace's title does. */ + listBlueprints(): Promise; + + /** The workspace's operational state. Admin access records are not part of it: they are kept + * only in the deployment-wide log (AdminApi.listAdminAudit()). */ + getDiagnostics(): Promise; +} + /** A partial edit to one promoted format. Absent fields are left alone. */ export type AdminFormatPatch = { enabled?: boolean; @@ -4980,8 +5738,11 @@ export type ChatAttachmentRef = { id: string; /** Original immutable upload blob. Absent on legacy attachments. */ oid?: GitOid; + /** MIME type the sender declared for the attachment, normalized when sending. */ mimeType: string; + /** Display filename, sanitized when sending. Absent when the sender gave none. */ name?: string; + /** Size of the attachment in bytes. */ size: number; /** diff --git a/scripts/run-dev-server.ts b/scripts/run-dev-server.ts index 425db29731..e73c36daa7 100644 --- a/scripts/run-dev-server.ts +++ b/scripts/run-dev-server.ts @@ -593,8 +593,9 @@ for (const gk of gatekeepers) { // needs the token even when the binding is present. "CF_AI_GATEWAY", "CF_AI_GATEWAY_PROVIDERS", "CF_AI_GATEWAY_ACCOUNT_ID", "CF_AI_GATEWAY_API_TOKEN", "CF_AI_GATEWAY_USE_BINDING", - // The admin audit log's minimum retention (see docs/admin-access.md). - "ADMIN_AUDIT_RETENTION_MIN_DAYS", + // The bounds on admin access to workspaces (see docs/admin-access.md), e.g. + // `ADMIN_WORKSPACE_ACCESS_MAX=full pnpm dev-server` lets the dev admin try full inspection. + "ADMIN_WORKSPACE_ACCESS_MAX", "ADMIN_AUDIT_RETENTION_MIN_DAYS", ]; // OAuth app credentials (GOOGLE_/GITHUB_/CLOUDFLARE_OAUTH_*) are NOT passed to the backend anymore; // they are injected into the gatekeeper Workers (see SHARED_GATEKEEPER_CREDS below).