Skip to content

Commit 4a20345

Browse files
authored
Merge pull request #1094 from cipherstash/feat/go-stashgen
feat(golang)!: the Go SDK: stash tags, stashgen and generated Encrypt/Decrypt
2 parents 415b62c + c1bb0dd commit 4a20345

243 files changed

Lines changed: 20248 additions & 9614 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/dependabot.yml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -211,8 +211,8 @@ updates:
211211
- /packages/stack-auth/fuzz
212212
- /packages/stack-kms/fuzz
213213
- /packages/stack-encrypt/fuzz
214-
- /languages/golang/stackencrypt/guest
215-
- /languages/golang/stackauth/guest
214+
- /languages/golang/encrypt/guest
215+
- /languages/golang/auth/guest
216216
# Monthly, matching the other two cargo entries.
217217
schedule:
218218
interval: monthly

‎.github/workflows/tests-golang.yml‎

Lines changed: 44 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -8,8 +8,10 @@ name: Tests (Go)
88
# wasi-check the stack crates build for wasm32-wasip1 with no JS-host or
99
# native-HTTP dependencies, the no-http shape passes its tests
1010
# and docs, both guests pass lint and tests and are built with
11-
# their import surfaces checked, their sha256 is recorded, and
12-
# `go:test` runs against them.
11+
# their import surfaces checked (the stack-encrypt guest twice:
12+
# the real build and the deterministic-kms test build), their
13+
# sha256 is recorded, `go:test` runs against them, and
14+
# `go generate` leaves the tree unchanged.
1315
# go-lint golangci-lint, Linux only.
1416
# go-binding-cross
1517
# the same Go tests on macOS and Windows, against the guests
@@ -122,8 +124,8 @@ jobs:
122124
with:
123125
workspaces: |
124126
.
125-
languages/golang/stackencrypt/guest
126-
languages/golang/stackauth/guest
127+
languages/golang/encrypt/guest
128+
languages/golang/auth/guest
127129
128130
# The HTTP-free core compiles for wasm32-wasip1 with no JS-host backend
129131
# and no native HTTP/TLS stack in its graph: the invariant the wazero
@@ -146,6 +148,14 @@ jobs:
146148
- name: stack-encrypt guest release build and import-surface gate
147149
run: mise run wasm:guest:build
148150

151+
# The deterministic-kms TEST build, under encrypt/testdata where no
152+
# build or embed sees it: the Go tests open the record fixture Rust
153+
# sealed through it, and run round trips with no ZeroKMS. The tests
154+
# skip when it is absent, so a build that forgets this step would pass
155+
# with less coverage, which is why the job builds it unconditionally.
156+
- name: stack-encrypt guest deterministic test build
157+
run: mise run wasm:guest:build:deterministic
158+
149159
- name: Credential guest lint and tests
150160
run: mise run wasm:auth-guest:test
151161

@@ -157,7 +167,7 @@ jobs:
157167
# same bytes rather than a stale or rebuilt guest.
158168
- name: Record the guests' checksums
159169
run: |
160-
for guest in stackencrypt/wasm/stack_encrypt_guest.wasm stackauth/wasm/stack_auth_guest.wasm; do
170+
for guest in encrypt/wasm/stack_encrypt_guest.wasm encrypt/testdata/stack_encrypt_guest_deterministic.wasm auth/wasm/stack_auth_guest.wasm; do
161171
(cd languages/golang && openssl dgst -sha256 "$guest" | awk '{print $NF}' > "$guest.sha256" && echo "$guest sha256 $(cat "$guest.sha256")")
162172
done
163173
@@ -166,25 +176,45 @@ jobs:
166176
with:
167177
name: wasm-guests
168178
path: |
169-
languages/golang/stackencrypt/wasm/stack_encrypt_guest.wasm
170-
languages/golang/stackencrypt/wasm/stack_encrypt_guest.wasm.sha256
171-
languages/golang/stackauth/wasm/stack_auth_guest.wasm
172-
languages/golang/stackauth/wasm/stack_auth_guest.wasm.sha256
179+
languages/golang/encrypt/wasm/stack_encrypt_guest.wasm
180+
languages/golang/encrypt/wasm/stack_encrypt_guest.wasm.sha256
181+
languages/golang/encrypt/testdata/stack_encrypt_guest_deterministic.wasm
182+
languages/golang/encrypt/testdata/stack_encrypt_guest_deterministic.wasm.sha256
183+
languages/golang/auth/wasm/stack_auth_guest.wasm
184+
languages/golang/auth/wasm/stack_auth_guest.wasm.sha256
173185
if-no-files-found: error
174186
retention-days: 1
175187

176188
# Format, vet and hermetic tests on amd64 and 386. The guest's memory
177189
# lock is best effort, so its test skips where RLIMIT_MEMLOCK refuses
178-
# it; CI raises the limit and sets STACKENCRYPT_TESTS_REQUIRE_LOCK so
190+
# it; CI raises the limit and sets STACK_ENCRYPT_TESTS_REQUIRE_LOCK so
179191
# the skip is an error here.
180192
- name: Go binding
181193
env:
182-
STACKENCRYPT_TESTS_REQUIRE_LOCK: "1"
194+
STACK_ENCRYPT_TESTS_REQUIRE_LOCK: "1"
183195
run: |
184196
ulimit -l "$(ulimit -H -l)"
185197
echo "RLIMIT_MEMLOCK: $(ulimit -l) KiB"
186198
mise run go:test
187199
200+
# What is encrypted is fixed before the program ships: every generated
201+
# file is committed, and this fails when `go generate` would change
202+
# one. It runs the real stashgen against the guest just built, so it
203+
# also proves the generator and the engine agree on the module's own
204+
# examples. `git diff` sees only tracked files, so the status check
205+
# catches a generated file that was never committed.
206+
- name: Generated code is committed
207+
working-directory: languages/golang
208+
run: |
209+
CGO_ENABLED=0 go generate ./...
210+
git diff --exit-code -- .
211+
untracked="$(git status --porcelain -- .)"
212+
if [ -n "$untracked" ]; then
213+
echo "::error::go generate wrote files that are not committed:"
214+
echo "$untracked"
215+
exit 1
216+
fi
217+
188218
# Linux only: macOS and Windows would report the same findings. Needs no
189219
# guests: the packages embed a directory and compile without them.
190220
go-lint:
@@ -260,7 +290,7 @@ jobs:
260290
- name: The guests are the ones Linux built and checked
261291
run: |
262292
cd languages/golang
263-
for guest in stackencrypt/wasm/stack_encrypt_guest.wasm stackauth/wasm/stack_auth_guest.wasm; do
293+
for guest in encrypt/wasm/stack_encrypt_guest.wasm encrypt/testdata/stack_encrypt_guest_deterministic.wasm auth/wasm/stack_auth_guest.wasm; do
264294
want=$(cat "$guest.sha256")
265295
got=$(openssl dgst -sha256 "$guest" | awk '{print $NF}')
266296
if [ "$want" != "$got" ]; then
@@ -326,7 +356,7 @@ jobs:
326356
- name: The guests are the ones the wasi-check job built and checked
327357
run: |
328358
cd languages/golang
329-
for guest in stackencrypt/wasm/stack_encrypt_guest.wasm stackauth/wasm/stack_auth_guest.wasm; do
359+
for guest in encrypt/wasm/stack_encrypt_guest.wasm encrypt/testdata/stack_encrypt_guest_deterministic.wasm auth/wasm/stack_auth_guest.wasm; do
330360
want=$(cat "$guest.sha256")
331361
got=$(openssl dgst -sha256 "$guest" | awk '{print $NF}')
332362
if [ "$want" != "$got" ]; then
@@ -340,7 +370,7 @@ jobs:
340370
# call `liveClient`, and not all of them are named `TestLive*`.
341371
- name: Go live tests
342372
working-directory: languages/golang
343-
run: CGO_ENABLED=0 go test -v ./stackencrypt/...
373+
run: CGO_ENABLED=0 go test -v ./encrypt/...
344374

345375
- name: stack-encrypt examples
346376
run: |

‎.gitignore‎

Lines changed: 8 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -108,7 +108,11 @@ mutants.out/
108108

109109
# The Go module's embedded WASI guests: build outputs of `mise run
110110
# wasm:guest:build` and `mise run wasm:auth-guest:build`.
111-
languages/golang/stackencrypt/wasm/*.wasm
112-
languages/golang/stackauth/wasm/*.wasm
113-
languages/golang/stackencrypt/wasm/*.sha256
114-
languages/golang/stackauth/wasm/*.sha256
111+
languages/golang/encrypt/wasm/*.wasm
112+
languages/golang/auth/wasm/*.wasm
113+
languages/golang/encrypt/wasm/*.sha256
114+
languages/golang/auth/wasm/*.sha256
115+
# The deterministic-kms TEST build of the stack-encrypt guest, from `mise run
116+
# wasm:guest:build:deterministic`; under testdata so no binary embeds it.
117+
languages/golang/encrypt/testdata/*.wasm
118+
languages/golang/encrypt/testdata/*.sha256

‎AGENTS.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -92,7 +92,7 @@ Every npm package except EQL lives under `languages/typescript/`: packages in `l
9292
links are provenance only.
9393
- `packages/stack-auth`, `packages/stack-profile`, `packages/stack-kms`, `packages/stack-encrypt`, `packages/stack-encrypt-derive`, `packages/stack-guest-abi`: The Rust crates imported from `cipherstash/cipherstash-suite` with their history — `stack-auth` and `stack-profile` (published to crates.io, one version group), `stack-kms` (published to crates.io from 0.1.0, its own version group, re-exported by `stack-encrypt` as `stack_encrypt::kms`), `stack-encrypt` and `stack-encrypt-derive` (published to crates.io from 0.1.0, one version group; `eql-bindings`' `stack-encrypt` feature depends on them from the registry), and `stack-guest-abi` (`publish = false`). They are the members of the **root Cargo workspace**, with the three node binding crates below. See "Working on the Rust crates".
9494
- `languages/typescript/packages/auth`, `languages/typescript/packages/profile`, `languages/typescript/packages/stack-auth-wasm`: The node bindings of those crates. `@cipherstash/auth` (napi-rs v2) and its six `platforms/*` packages are published to npm from this repository by `release.yml` (`auth-artifacts`, `publish-auth`); a change to what it ships, the `stack-auth` crate included, needs an `@cipherstash/auth` changeset (`require-auth-npm-changeset.yml`). `@cipherstash/profile` and its platforms are private and never published; `@cipherstash/stack-auth-wasm` is private and builds the wasm that `@cipherstash/auth` ships. Their `build` and `test` scripts never invoke cargo; `build:native`, `build:debug` and `test:cargo` do.
95-
- `languages/golang`: The Go module (`stackencrypt`, `stackauth`, `internal`), a wazero host with no cgo. Its two WASI guests (`*/guest`) are detached Cargo workspaces built by `mise run wasm:guest:build` and `mise run wasm:auth-guest:build`; the `.wasm` files they embed are gitignored. There is no Go release process yet.
95+
- `languages/golang`: The Go module — the SDK `encrypt` with its generated-code support `encrypt/gensupport` and the policy packages `encrypt/policy` and `encrypt/policy/protosource`; the credential package `auth`; the generator `stashgen` and its command `cmd/stashgen`; and `internal` (the shared guest plumbing and the `record` wire model). A wazero host with no cgo. Its two WASI guests (`encrypt/guest`, `auth/guest`) are detached Cargo workspaces built by `mise run wasm:guest:build` and `mise run wasm:auth-guest:build`; `mise run wasm:guest:build:deterministic` builds the seeded TEST build the hermetic Go tests use; the `.wasm` files they embed are gitignored. Generated `*_stash.go` files are committed and CI fails when `go generate ./...` changes one. There is no Go release process yet.
9696
- `e2e/*`: Cross-package end-to-end tests (package managers, supply chain, Prisma example README)
9797
- `languages/typescript/examples/*`: Working apps (basic, prisma, supabase-worker)
9898
- `docs/plans/*`: Internal design plans. User-facing documentation lives at https://cipherstash.com/docs (not in this repo).

‎Cargo.lock‎

Lines changed: 0 additions & 1 deletion
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎Cargo.toml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -27,8 +27,8 @@ exclude = [
2727
"packages/stack-encrypt/fuzz",
2828
# WASI guests for the Go module, built through `wasm:guest:build` and
2929
# `wasm:auth-guest:build`.
30-
"languages/golang/stackencrypt/guest",
31-
"languages/golang/stackauth/guest",
30+
"languages/golang/encrypt/guest",
31+
"languages/golang/auth/guest",
3232
]
3333

3434
[workspace.package]

‎SECURITY.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -33,7 +33,7 @@ It also carries the source of five Rust crates published to crates.io,
3333
`packages/stack-profile`) and **`stack-kms`**, **`stack-encrypt`** and
3434
**`stack-encrypt-derive`** (`packages/stack-kms`, `packages/stack-encrypt`,
3535
`packages/stack-encrypt-derive`), and of the **Go module** at
36-
`languages/golang` (`stackencrypt` and `stackauth`, over WASI guests built
36+
`languages/golang` (`encrypt` and `auth`, over WASI guests built
3737
from the stack-* crates), which has no release yet. All of these are in scope
3838
for security reports on the same terms as the npm packages above.
3939

‎docs/plans/2026-10-04-plan-builder.md‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,13 @@ the builder itself, #1071), with a second round of decisions: decrypt is
1616
spelled `open`, the two starts and the three context sources, the typed
1717
verb and the picker, the derive narrowed before it emits the plan, and EQL
1818
types assembled per language with no registry.
19+
Amended 2026-10-07 (#1094): where this document says the typed parts "have
20+
no data form" and lists `context_field` among them ("Consolidation", and
21+
item 3 under "Why the first draft was dropped"), it is wrong about
22+
`context_field`. That verb is not typed, and the data grammar now spells it
23+
as a plan-level `"context_field"` key; the Go SDK spells it as the
24+
`context_field` tag word. The typed parts that have no data form are
25+
`encrypt_into`, the picker and the one-value start.
1926
**Date:** 2026-10-04
2027
**Issue:** #1046
2128
**Builds on:** #1050 (one context per column; `Label`, `Describe`), #971 (EQL v3

‎languages/golang/.gitattributes‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
# stashgen's golden test expectations. Marked `linguist-generated` so GitHub
2+
# collapses them in pull-request and commit diffs by default and excludes them
3+
# from repository language statistics. This is a GitHub display hint only:
4+
# nothing about Git, CI, or the build changes.
5+
#
6+
# The generated *_stash.go files are deliberately NOT marked. They are the
7+
# committed record of which fields are encrypted, with which indexes and under
8+
# which context, so a change to them must stay expanded for the reviewer to
9+
# read (docs/sdk-design-principles.md, Go principle 10).
10+
*.golden linguist-generated
Lines changed: 17 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,9 @@
1-
# stackauth
1+
# auth
22

33
The Go binding of the developer profile — the directory `stash auth login`
44
writes — read through the `stack-profile` Rust crate running inside a WASI
55
guest under [wazero], with `CGO_ENABLED=0`. It is the credential half of
6-
the Go SDK: it hands a [`stackencrypt`](../stackencrypt) client its client
6+
the Go SDK: it hands an [`encrypt`](../encrypt) client its client
77
key and its bearer token without either package re-deriving the profile's
88
layout, and without either importing the other.
99

@@ -16,30 +16,30 @@ cross-process refresh lock for device sessions.
1616

1717
## Use
1818

19-
Most applications never call this package directly: a `stackencrypt`
19+
Most applications never call this package directly: an `encrypt`
2020
client built with `NewClient(ctx)` and no options resolves its credentials with
21-
`stackencrypt.AutoCredentials`, which reads the environment first and then
21+
`encrypt.AutoCredentials`, which reads the environment first and then
2222
the profile, through this package. Use it directly to take the profile
2323
apart yourself:
2424

2525
```go
2626
import (
2727
"context"
2828

29-
"github.com/cipherstash/stack/languages/golang/stackauth"
30-
"github.com/cipherstash/stack/languages/golang/stackencrypt"
29+
"github.com/cipherstash/stack/languages/golang/auth"
30+
"github.com/cipherstash/stack/languages/golang/encrypt"
3131
)
3232

3333
func run(ctx context.Context) error {
34-
profile, err := stackauth.Resolve(ctx) // CS_CONFIG_PATH, else ~/.cipherstash
34+
profile, err := auth.Resolve(ctx) // CS_CONFIG_PATH, else ~/.cipherstash
3535
if err != nil {
3636
return err
3737
}
3838
defer profile.Close()
3939

4040
workspace, err := profile.CurrentWorkspaceStore(ctx)
4141
if err != nil {
42-
return err // stackauth.ErrNoCurrentWorkspace: run `stash auth login`
42+
return err // auth.ErrNoCurrentWorkspace: run `stash auth login`
4343
}
4444
clientID, clientKey, err := workspace.SecretKey(ctx)
4545
if err != nil {
@@ -50,9 +50,9 @@ func run(ctx context.Context) error {
5050
return err
5151
}
5252
defer source.Close()
53-
client, err := stackencrypt.NewClient(ctx,
53+
client, err := encrypt.NewClient(ctx,
5454
// The key is consumed and wiped by NewClient.
55-
stackencrypt.WithCredentials(stackencrypt.NewCredentials(clientID, clientKey, source)),
55+
encrypt.WithCredentials(encrypt.NewCredentials(clientID, clientKey, source)),
5656
)
5757
if err != nil {
5858
return err
@@ -63,32 +63,32 @@ func run(ctx context.Context) error {
6363
}
6464
```
6565

66-
`stackauth.ClientKey` and `stackencrypt.ClientKey` are one type, so the
66+
`auth.ClientKey` and `encrypt.ClientKey` are one type, so the
6767
key goes straight from the profile into the credentials. The profile and
6868
the strategy are the caller's: the client asks the strategy for a token on
6969
every request but never closes it, so both stay open until the client is
7070
closed (the deferred calls above run in that order).
7171

72-
A stackencrypt client takes its token only from a strategy, never a raw
72+
An encrypt client takes its token only from a strategy, never a raw
7373
string: a raw token cannot be refreshed when it expires, and would bypass
7474
the cross-process lock a device-session refresh holds with the `stash` CLI
7575
(the IdP revokes a whole refresh-token chain when one is used twice).
7676
`workspace.Token(ctx)` still reads the stored token, for inspection.
7777

7878
With no profile directory at all (CI, a container, a server authenticating
79-
by federation), `stackauth.OpenWithoutProfile(ctx)` runs the guest with
79+
by federation), `auth.OpenWithoutProfile(ctx)` runs the guest with
8080
nothing mounted: the access-key and OIDC strategies work, and every profile
8181
read is `ErrNoProfile`.
8282

8383
`profile.AccessKey(ctx, crn, key)`, `profile.OIDC(ctx, crn, provider)`, and
84-
`profile.Auto(ctx)` also return strategies that `stackencrypt.NewCredentials`
84+
`profile.Auto(ctx)` also return strategies that `encrypt.NewCredentials`
8585
takes. `Auto` checks `CS_CLIENT_ACCESS_KEY` and
8686
`CS_WORKSPACE_CRN` first, then the current workspace's stored device session.
8787
The OIDC provider is a one-method `Token(context.Context) (string, error)`
8888
interface, called on every token fetch for the JWT of the user the call is
8989
for; each distinct JWT is exchanged once while its CTS token lasts. Use
90-
`stackauth.OAuth2TokenSource(source)` to adapt a
91-
`golang.org/x/oauth2.TokenSource`. `WithAuthBaseURL(url)` overrides service
90+
`auth.OAuth2TokenSource(source)` to adapt a
91+
`golang.org/x/oauth2.TokenSource`. `WithBaseURL(url)` overrides service
9292
discovery for local tests or a custom CTS host; `WithCacheCapacity(n)` sets
9393
how many users' CTS tokens an OIDC strategy keeps (1024 unless set), sized to
9494
the users it serves within a CTS token's lifetime.
@@ -107,7 +107,7 @@ session refresh call, on the path `ProfileStore.LockPath` names. A fresh
107107
token is read without the lock; on refresh the guest re-reads
108108
auth.json after acquisition and saves refreshed tokens before release.
109109

110-
The crypto guest behind `stackencrypt` is not widened by this package
110+
The crypto guest behind `encrypt` is not widened by this package
111111
existing: it still has no filesystem and no environment.
112112

113113
## Build

0 commit comments

Comments
 (0)