Exporting `memhttp` / `memhttptest`

[oxisto]

Is your feature request related to a problem? Please describe.

We are currently exploring migrating our project from the gRPC Go implementation to connect-go. Since our project uses several micro services, we have a lot of tests were we launch other microservices (in a defined state) to interact with our test target (or the other way around). For that we heavily rely on the bufconn package to avoid a lot of unnecessary (even though local) network traffic and network port management in the tests.

It seems that the internal packages memhttp / memhttptest offer a similar functionality. I wonder if there is an intent to publish these packages as an official testing solution?

Describe the solution you'd like

Export the memhttp package.

[jhump]

@oxisto, the internal package here was largely based on https://github.com/akshayjshah/memhttp. We are in fact considering something like this as exported API available in the connectrpc GitHub organization, but until then you can use the above 3rd-party module.

[oxisto]

@oxisto, the internal package here was largely based on https://github.com/akshayjshah/memhttp. We are in fact considering something like this as exported API available in the connectrpc GitHub organization, but until then you can use the above 3rd-party module.

Ah excellent, thanks! Having this within the connectrpc org would be awesome, but this will do for the migration period.

[mfridman]

Would love to see a version of this library exposed. +1

This might also be interesting with the net/http improvements in https://go.dev/doc/go1.24#nethttppkgnethttp

[oxisto]

Any plans for actually doing this? ;)

[emcfarlane]

I've been looking into this. With a plan to propose a new memhttp-go repository (connectrpc.com/memhttp) following the same pattern as cors, authn and otelconnect packages.

Before committing to that, I want to flag golang/go#77362. This is a proposal to add testing/nettest to the standard library with in-memory net.Listener, net.Conn, and net.PacketConn. It covers the primitives memhttp is built on and would make a standalone module largely redundant. It's going to x/exp first, so no firm timeline yet, but it could be usable relatively soon.

I'm leaning toward waiting for that rather than publishing something to deprecate shortly after. But I'd like to hear from anyone following this if this package would still be desirable?

[oxisto]

I've been looking into this. With a plan to propose a new memhttp-go repository (connectrpc.com/memhttp) following the same pattern as cors, authn and otelconnect packages.

Before committing to that, I want to flag golang/go#77362. This is a proposal to add testing/nettest to the standard library with in-memory net.Listener, net.Conn, and net.PacketConn. It covers the primitives memhttp is built on and would make a standalone module largely redundant. It's going to x/exp first, so no firm timeline yet, but it could be usable relatively soon.

I'm leaning toward waiting for that rather than publishing something to deprecate shortly after. But I'd like to hear from anyone following this if this package would still be desirable?

Both approaches would probably work fine. I am also a big favor of using something in the standard library (or at least in /x) rather than having another external dependency. We have currently "abandoned" in-memory testing and are just testing with real HTTP connections with random port 0, but having something in memory would definitely be better for testing. Testing with real TCP connections is good, but also has some major drawbacks.

[emcfarlane]

The v2 proposal (#951) addresses this with a dedicated package rather than exporting memhttp. connectinprocess provides a Transport that dispatches a client directly to a connect.Server in the same process:

srv := connect.NewServer()
foov1connect.RegisterFooServiceHandler(srv, svc)
client := foov1connect.NewFooServiceClient(connect.NewClient(connectinprocess.New(srv)))

Interceptors run on both sides and metadata flows as usual. Less machinery than an in-memory HTTP server. Welcome your feedback on the proposal.