Skip to content

feat(http): add http.client.log for logging outgoing HTTP requests #4337

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
reinkrul wants to merge 4 commits into
base: master
Choose a base branch
from
Open

feat(http): add http.client.log for logging outgoing HTTP requests #4337

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
def43cd
feat(http): add http.client.log for logging outgoing HTTP requests
reinkrul Jun 8, 2026
80f0cda
fix(http): apply client request logging at request time
reinkrul Jun 8, 2026
8da80e0
refactor(http): simplify http.client.log header logging
reinkrul Jun 8, 2026
bd39e7a
refactor(http): move client request logging into http/client package
reinkrul Jun 9, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion docs/pages/deployment/server_options.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@
http.clientipheader X-Forwarded-For Case-sensitive HTTP Header that contains the client IP used for audit logs. For the X-Forwarded-For header only link-local, loopback, and private IPs are excluded. Switch to X-Real-IP or a custom header if you see your own proxy/infra in the logs.
http.log metadata What to log about HTTP requests. Options are 'nothing', 'metadata' (log request method, URI, IP and response code), and 'metadata-and-body' (log the request and response body, in addition to the metadata). When debug vebosity is set the authorization headers are also logged when the request is fully logged.
http.cache.maxbytes 10485760 HTTP client maximum size of the response cache in bytes. If 0, the HTTP client does not cache responses.
http.client.log nothing What to log about outgoing HTTP requests made by the node. Options are 'nothing', 'metadata' (log request method, URI and response code), and 'metadata-and-body' (log the request and response body, in addition to the metadata). When debug vebosity is set the request and response headers are also logged.
http.internal.address 127.0.0.1:8081 Address and port the server will be listening to for internal-facing endpoints.
http.internal.auth.audience Expected audience for JWT tokens (default: hostname)
http.internal.auth.authorizedkeyspath Path to an authorized_keys file for trusted JWT signers
Expand Down Expand Up @@ -70,5 +71,5 @@
tracing.servicename Service name reported to the tracing backend. Defaults to 'nuts-node'.
**policy**
policy.directory ./config/policy Directory to read policy files from. Policy files are JSON files that contain a scope to PresentationDefinition mapping.
policy.authzen.endpoint Base URL of the AuthZen PDP endpoint. Required when any credential profile uses scope_policy 'dynamic'.
policy.authzen.endpoint Base URL of the AuthZen PDP endpoint. Required when any credential profile uses scope_policy 'dynamic'; the node refuses to start if such a profile is configured but this flag is empty.
======================================== =================================================================================================================================================================================================================================================================================================================================================================================================================================================================== ============================================================================================================================================================================================================================================================================================================================================
29 changes: 26 additions & 3 deletions http/client/client.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,13 @@ func httpSpanName(_ string, r *http.Request) string {
// StrictMode is a flag that can be set to true to enable strict mode for the HTTP client.
var StrictMode bool

// RequestLogger wraps the transport of HTTP clients created by this package to log
// outgoing requests and their responses. It is left nil (no logging) by default.
// It is read at request time (not when the client is created), because clients are often created
// before the HTTP engine configures it: the HTTP engine is configured last, while other engines
// create their HTTP clients earlier.
var RequestLogger func(http.RoundTripper) http.RoundTripper

// DefaultMaxHttpResponseSize is a default maximum size of an HTTP response body that will be read.
// Very large or unbounded HTTP responses can cause denial-of-service, so it's good to limit how much data is read.
// This of course heavily depends on the use case, but 1MB is a reasonable default.
Expand All @@ -81,15 +88,31 @@ func New(timeout time.Duration) *StrictHTTPClient {
}
}

// getTransport wraps the given transport with OpenTelemetry instrumentation if tracing is enabled.
// getTransport wraps the given transport with request logging and OpenTelemetry
// instrumentation (if tracing is enabled).
func getTransport(base http.RoundTripper) http.RoundTripper {
// Always install the logging indirection so RequestLogger can be configured after the client
// is created: whether to log is decided per request, not when the client is created.
transport := http.RoundTripper(&loggingTransport{base: base})
if tracing.Enabled() {
return otelhttp.NewTransport(base,
return otelhttp.NewTransport(transport,
otelhttp.WithSpanNameFormatter(httpSpanName),
otelhttp.WithTracerProvider(tracing.GetTracerProvider()),
)
}
return base
return transport
}

// loggingTransport applies the configured RequestLogger (if any) at request time.
type loggingTransport struct {
base http.RoundTripper
}

func (l *loggingTransport) RoundTrip(request *http.Request) (*http.Response, error) {
if logger := RequestLogger; logger != nil {
return logger(l.base).RoundTrip(request)
}
return l.base.RoundTrip(request)
}

// NewWithCache creates a new HTTP client with the given timeout.
Expand Down
44 changes: 40 additions & 4 deletions http/client/client_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@ func TestStrictHTTPClient(t *testing.T) {
client := NewWithTLSConfig(time.Second, &tls.Config{
InsecureSkipVerify: true,
})
ts := client.client.Transport.(*http.Transport)
ts := client.client.Transport.(*loggingTransport).base.(*http.Transport)
assert.True(t, ts.TLSClientConfig.InsecureSkipVerify)
})
})
Expand Down Expand Up @@ -215,14 +215,16 @@ func TestGetTransport(t *testing.T) {
assert.NotEqual(t, SafeHttpTransport, transport)
})

t.Run("returns base transport when tracing disabled", func(t *testing.T) {
t.Run("wraps base in logging transport when tracing disabled", func(t *testing.T) {
original := tracing.Enabled()
tracing.SetEnabled(false)
t.Cleanup(func() { tracing.SetEnabled(original) })

transport := getTransport(SafeHttpTransport)

assert.Equal(t, SafeHttpTransport, transport)
logging, ok := transport.(*loggingTransport)
require.True(t, ok)
assert.Equal(t, SafeHttpTransport, logging.base)
})
}

Expand All @@ -245,6 +247,40 @@ func TestNew(t *testing.T) {

client := New(time.Second)

assert.Equal(t, SafeHttpTransport, client.client.Transport)
logging, ok := client.client.Transport.(*loggingTransport)
require.True(t, ok)
assert.Equal(t, SafeHttpTransport, logging.base)
})
}

func TestRequestLogger_setAfterClientCreation(t *testing.T) {
// Clients are created before the HTTP engine sets RequestLogger, so logging must take effect
// for clients that already exist when RequestLogger is configured.
originalTracing := tracing.Enabled()
tracing.SetEnabled(false)
t.Cleanup(func() { tracing.SetEnabled(originalTracing) })
t.Cleanup(func() { RequestLogger = nil })
StrictMode = false

server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusOK)
}))
t.Cleanup(server.Close)

// Create the client first, while RequestLogger is still nil.
RequestLogger = nil
client := New(time.Second)

// Now configure the logger, as the HTTP engine does later during startup.
var invoked atomic.Bool
RequestLogger = func(base http.RoundTripper) http.RoundTripper {
invoked.Store(true)
return base
}

httpRequest, _ := http.NewRequest(http.MethodGet, server.URL, nil)
_, err := client.Do(httpRequest)

require.NoError(t, err)
assert.True(t, invoked.Load(), "RequestLogger should be applied to clients created before it was set")
}
83 changes: 83 additions & 0 deletions http/clientlogger.go
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
/*
* Copyright (C) 2024 Nuts community
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*
*/

package http

import (
"bytes"
"io"
"net/http"

"github.com/sirupsen/logrus"
)

// clientRequestLogger is an http.RoundTripper that logs outgoing HTTP requests and their responses.
// It mirrors the server-side request/body logging: metadata is always logged, bodies only when logBody is set.
type clientRequestLogger struct {
transport http.RoundTripper
logger *logrus.Entry
logBody bool
}

func (c *clientRequestLogger) RoundTrip(request *http.Request) (*http.Response, error) {
fields := logrus.Fields{
"method": request.Method,
"uri": request.URL.String(),
}
if c.logger.Logger.Level >= logrus.DebugLevel {
fields["headers"] = request.Header
}
c.logger.WithFields(fields).Info("HTTP client request")

Check failure

Code scanning / CodeQL

Clear-text logging of sensitive information High

Sensitive data returned by HTTP request headers
Loading
flows to a logging call.
Comment thread
github-advanced-security[bot] marked this conversation as resolved.
Fixed

if c.logBody && request.Body != nil && isLoggableContentType(request.Header.Get("Content-Type")) {
body, err := io.ReadAll(request.Body)
_ = request.Body.Close()
if err != nil {
return nil, err
}
request.Body = io.NopCloser(bytes.NewReader(body))
c.logger.Infof("HTTP client request body: %s", string(body))
}

response, err := c.transport.RoundTrip(request)
if err != nil {
c.logger.WithFields(logrus.Fields{
"method": request.Method,
"uri": request.URL.String(),
}).WithError(err).Info("HTTP client request failed")
return nil, err
}

c.logger.WithFields(logrus.Fields{
"method": request.Method,
"uri": request.URL.String(),
"status": response.StatusCode,
}).Info("HTTP client response")

if c.logBody && response.Body != nil && isLoggableContentType(response.Header.Get("Content-Type")) {
body, err := io.ReadAll(response.Body)
_ = response.Body.Close()
if err != nil {
return nil, err
}
response.Body = io.NopCloser(bytes.NewReader(body))
c.logger.Infof("HTTP client response body: %s", string(body))
}

return response, nil
}
Loading
Loading