Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Changed

- `spark-http-proxy hosts describe` reads the container live: image, status and uptime, the port it is routed to, the backend and network, whether it answers through the proxy, its mounts, and its command with secrets redacted by flag name ([#164](https://github.com/sparkfabrik/http-proxy/issues/164))
- `tailscale-peers --refresh` replaces `tailscale-refresh-peers`, which is removed
- The peer table has two groups, `PROXY` and `EXCLUDED`, with the reason in a `STATUS` column
- The peer summary is one line: machines, how many run this proxy and forward what, how many are excluded
Expand Down
36 changes: 33 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -138,9 +138,39 @@ pg-workflows.githuman.sparkfabrik.loc local githuman-pg-workflows ~/webapp
macos.test.spark.loc Mac-Test - -
```

`hosts describe <hostname>` reports one host, including how it is routed. A remote
host shows the machine and no directory: local paths are not published to the
tailnet. `hosts --json` prints the local records as JSON. Hostnames served by other machines are not in it; `tailscale-peers --json` carries those.
`hosts describe <hostname>` reads the container behind one hostname live from
Docker and the proxy, so it answers "what is this container" without a
`docker inspect` by hand:

```console
$ spark-http-proxy hosts describe sparkdock.githuman.sparkfabrik.loc
sparkdock.githuman.sparkfabrik.loc
container githuman-sparkdock
image node:lts
status running, up 29 hours
directory ~/webapps/sparkfabrik/agents/tailcat-use-cases/sparkdock
routed by VIRTUAL_HOST, port 3847
backend http://172.17.0.3:3847
network bridge
reachable 200
mounts ~/webapps/sparkfabrik/agents/tailcat-use-cases/sparkdock -> same path (rw)
githuman-npm-cache -> /cache/npm (rw)
githuman-data -> /data/githuman (rw)
command docker-entrypoint.sh bash -c npx githuman@0.9.0 serve --host 0.0.0.0 --auth <redacted>
```

The port and backend are what Traefik routes to, read from its API, so they are
right for `VIRTUAL_HOST` and for native `traefik.*` labels alike. `reachable` is
the HTTP status of a request sent through the proxy with that `Host` header,
which is the path a browser takes; on Docker Desktop the container's own address
is inside the VM and would not answer from the host. Secrets in the command line
are redacted by flag name (`--auth`, `--token`, `--password` and similar, bare or
`--flag=value`), by assignment name (`*_TOKEN=`, `*_SECRET=`, ...) and in URL
userinfo. A value passed some other way is printed as is.

A remote host shows the machine and no directory: local paths are not published
to the tailnet. A record whose container Docker no longer has is reported as
such and the command fails. `hosts --json` prints the local records as JSON. Hostnames served by other machines are not in it; `tailscale-peers --json` carries those.

Directories come from the compose working directory, or the first bind mount for a
container started with `docker run`.
Expand Down
139 changes: 132 additions & 7 deletions bin/lib/hosts.sh
Original file line number Diff line number Diff line change
Expand Up @@ -127,8 +127,136 @@ hosts_list() {
done
}

# Redacts secret-bearing values in a command line: by flag name (bare, quoted
# or --flag=value), by assignment name, and the userinfo of a URL. Never by the
# shape of a value, so a secret passed some other way is still printed.
hosts_redact_command() {
local flags='auth|token|password|passwd|pass|secret|api-key|apikey|access-key|secret-key|client-secret|credentials|bearer'
sed -E \
-e "s#(--?(${flags})[= ])'[^']*'#\\1'<redacted>'#g" \
-e "s#(--?(${flags})[= ])\"[^\"]*\"#\\1\"<redacted>\"#g" \
-e "s#(--?(${flags})[= ])[^'\" ]+#\\1<redacted>#g" \
-e "s#([A-Za-z0-9_]*(TOKEN|SECRET|PASSWORD|PASSWD|API_KEY|APIKEY|ACCESS_KEY|CREDENTIALS)[A-Za-z0-9_]*=)'[^']*'#\\1'<redacted>'#g" \
-e "s#([A-Za-z0-9_]*(TOKEN|SECRET|PASSWORD|PASSWD|API_KEY|APIKEY|ACCESS_KEY|CREDENTIALS)[A-Za-z0-9_]*=)\"[^\"]*\"#\\1\"<redacted>\"#g" \
-e "s#([A-Za-z0-9_]*(TOKEN|SECRET|PASSWORD|PASSWD|API_KEY|APIKEY|ACCESS_KEY|CREDENTIALS)[A-Za-z0-9_]*=)[^'\" ]+#\\1<redacted>#g" \
-e 's#(://)[^/@ ]+:[^/@ ]+@#\1<redacted>@#g'
}

# The published host port of one proxy port, empty when the proxy is not up.
hosts_proxy_port() {
docker port http-proxy "$1/tcp" 2>/dev/null | head -n 1 | sed 's/.*://'
}

# The backend URL Traefik routes a hostname to, read from its API. Empty when
# the proxy is not running or nothing routes the hostname.
hosts_backend_url() {
local hostname="$1" api_port router service provider
api_port="$(hosts_proxy_port 8080)"
[[ -z "${api_port}" ]] && return 0
# One router per line, then the one whose rule names this exact host. A rule
# quotes the host with backticks or, escaped in the JSON, double quotes.
router="$(curl -s --max-time 5 "http://127.0.0.1:${api_port}/api/http/routers?search=${hostname}" 2>/dev/null |
sed 's/},{/}\n{/g' | grep -F -e "Host(\`${hostname}\`)" -e "Host(\\\"${hostname}\\\")" | head -n 1)"
[[ -z "${router}" ]] && return 0
service="$(grep -o '"service":"[^"]*"' <<<"${router}" | head -n 1 | cut -d'"' -f4)"
provider="$(grep -o '"provider":"[^"]*"' <<<"${router}" | head -n 1 | cut -d'"' -f4)"
[[ -z "${service}" || -z "${provider}" ]] && return 0
[[ "${service}" == *@* ]] || service="${service}@${provider}"
curl -s --max-time 5 "http://127.0.0.1:${api_port}/api/http/services/${service}" 2>/dev/null |
grep -o '"url":"[^"]*"' | head -n 1 | cut -d'"' -f4
}

# The record for a hostname served on this machine, read from Docker live.
hosts_describe_local() {
local hostname="$1" container="$2" directory="$3" routing="$4"
local info image state command networks uptime status backend port http_port code
local mounts line type name source destination rw first=true

echo "${hostname}"

if ! info="$(docker inspect --format '{{.Config.Image}}{{"\n"}}{{.State.Status}}{{"\n"}}{{range $k, $v := .NetworkSettings.Networks}}{{$k}} {{end}}{{"\n"}}{{.Path}} {{join .Args " "}}' "${container}" 2>/dev/null)"; then
echo " container ${container}, not found"
echo " directory $(hosts_abbreviate "${directory:--}")"
echo " routed by ${routing}"
log_error "The record names a container Docker no longer has; the proxy drops it on its next event"
return 1
fi
image="$(sed -n 1p <<<"${info}")"
state="$(sed -n 2p <<<"${info}")"
networks="$(sed -n 3p <<<"${info}" | sed 's/ $//; s/ /, /g')"
command="$(sed -n '4,$p' <<<"${info}" | hosts_redact_command)"

# docker ps renders the uptime, so no date arithmetic on either platform.
uptime="$(docker ps -a --filter "name=^${container}$" --format '{{.Status}}' 2>/dev/null | head -n 1)"
status="${state}"
if [[ -n "${uptime}" ]]; then
uptime="$(tr '[:upper:]' '[:lower:]' <<<"${uptime:0:1}")${uptime:1}"
[[ "${uptime}" == "${state}"* ]] && status="${uptime}" || status="${state}, ${uptime}"
fi

backend="$(hosts_backend_url "${hostname}")"
port=""
if [[ "${backend}" =~ ^[a-z]+://[^/]*:([0-9]+)(/|$) ]]; then
port="${BASH_REMATCH[1]}"
fi

echo " container ${container}"
echo " image ${image}"
echo " status ${status}"
echo " directory $(hosts_abbreviate "${directory:--}")"
case "${routing}" in
virtual-host) echo " routed by VIRTUAL_HOST${port:+, port ${port}}" ;;
traefik-labels) echo " routed by traefik.* labels${port:+, port ${port}}" ;;
*) echo " routed by ${routing}" ;;
esac

http_port="$(hosts_proxy_port 80)"
if [[ -z "${http_port}" ]]; then
echo " backend unknown, the proxy is not running"
echo " network ${networks:-none}"
echo " reachable unknown, the proxy is not running"
else
echo " backend ${backend:-none, the proxy has no route for this hostname}"
echo " network ${networks:-none}"
# Through the proxy, as a browser would go: on Docker Desktop the
# container's own address lives inside the VM and does not answer the host.
code="$(curl -s -o /dev/null -w '%{http_code}' --max-time 5 -H "Host: ${hostname}" "http://127.0.0.1:${http_port}/" 2>/dev/null)"
if [[ -z "${code}" || "${code}" == "000" ]]; then
echo " reachable no answer within 5s"
else
echo " reachable ${code}"
fi
fi

# A bind mount has no name, and read collapses adjacent tabs, so the fields
# are separated by a character that is not whitespace.
mounts="$(docker inspect --format '{{range .Mounts}}{{.Type}}{{"\x1f"}}{{.Name}}{{"\x1f"}}{{.Source}}{{"\x1f"}}{{.Destination}}{{"\x1f"}}{{.RW}}{{"\n"}}{{end}}' "${container}" 2>/dev/null)"
if [[ -z "${mounts}" ]]; then
echo " mounts none"
fi
while IFS=$'\x1f' read -r type name source destination rw; do
[[ -z "${destination}" ]] && continue
[[ "${rw}" == "true" ]] && rw="rw" || rw="ro"
if [[ "${type}" == "volume" && -n "${name}" ]]; then
line="${name} -> ${destination} (${rw})"
elif [[ "${source}" == "${destination}" ]]; then
line="$(hosts_abbreviate "${source}") -> same path (${rw})"
else
line="$(hosts_abbreviate "${source}") -> ${destination} (${rw})"
fi
if [[ "${first}" == "true" ]]; then
echo " mounts ${line}"
first=false
else
echo " ${line}"
fi
done <<<"${mounts}"

echo " command ${command}"
}

hosts_describe() {
local wanted="$1" hostname container directory routing machine found=false local_out line rest
local wanted="$1" hostname container directory routing machine found=false local_out line rest rc=0

if [[ -z "${wanted}" ]]; then
log_error "Which hostname? Usage: ${0} hosts describe <hostname>"
Expand All @@ -154,11 +282,7 @@ hosts_describe() {
routing="${rest#*$'\t'}"
[[ "${hostname}" != "${wanted}" ]] && continue
found=true
echo "${hostname}"
echo " served by this machine"
echo " container ${container}"
echo " directory $(hosts_abbreviate "${directory}")"
echo " routed by ${routing}"
hosts_describe_local "${hostname}" "${container}" "${directory}" "${routing}" || rc=1
done <<<"${local_out}"

while IFS=$'\t' read -r hostname machine; do
Expand All @@ -174,14 +298,15 @@ hosts_describe() {
log_info "See what does with: ${0} hosts"
return 1
fi
return "${rc}"
}

# One usage string, so the help and the error cannot drift apart.
hosts_usage() {
echo "Usage: ${0} hosts [list|describe <hostname>|--json]"
echo ""
echo " list Every hostname served, local and from peers (the default)"
echo " describe <hostname> One hostname: its container, directory and routing"
echo " describe <hostname> One hostname's container, read live: image, status, routing, backend, mounts, command"
echo " --json The local records, machine-readable"
echo ""
echo "Hostnames served by other machines carry no directory, and are listed in full by:"
Expand Down
94 changes: 90 additions & 4 deletions test/test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -2115,8 +2115,42 @@ test_hosts_command() {
printf 'labelled.spark.loc\tother-1\t\ttraefik-labels\n' >>"${dir}/hosts.tsv"
printf 'ok\n9 4\nMac-Test\tmacos.spark.loc,second.spark.loc\n' >"${dir}/summary"

# describe reads the container and the proxy live, so both are stubbed:
# docker answers inspect, ps and port for app-1 and other-1 and knows no
# other container; curl answers the Traefik API and the reachability probe.
mkdir -p "${dir}/stub"
cat >"${dir}/stub/docker" <<'DOCKER'
#!/usr/bin/env bash
case "$1 $2" in
"port http-proxy")
case "$3" in 80/tcp) echo "0.0.0.0:80" ;; 8080/tcp) echo "0.0.0.0:30000" ;; esac ;;
"ps -a")
echo "Up 3 hours" ;;
"inspect --format")
name="${*: -1}"
case "${name}" in app-1|other-1) ;; *) echo "Error: No such object: ${name}" >&2; exit 1 ;; esac
if [[ "$3" == *Mounts* ]]; then
printf 'bind\x1f\x1f%s/projects/app\x1f%s/projects/app\x1ftrue\n' "${HOME}" "${HOME}"
printf 'volume\x1fapp-cache\x1f/var/lib/docker/volumes/app-cache/_data\x1f/cache\x1ffalse\n'
else
printf 'node:lts\nrunning\nbridge \nnode server.js --auth s3cret --token=t0k3n API_KEY=k3y API_TOKEN='"'"'live secret'"'"' https://user:pw@db.spark.loc/x\n'
fi ;;
*) echo "unexpected docker call: $*" >&2; exit 1 ;;
esac
DOCKER
cat >"${dir}/stub/curl" <<'CURL'
#!/usr/bin/env bash
url="${*: -1}"
case "${url}" in
*/api/http/routers*) echo '[{"rule":"Host(`local.spark.loc`)","service":"app-1","provider":"file"},{"rule":"Host(\"labelled.spark.loc\")","service":"other-1@docker","provider":"docker"}]' ;;
*/api/http/services/*) echo '{"loadBalancer":{"servers":[{"url":"http://172.17.0.5:8080"}]}}' ;;
*) printf '200' ;;
esac
CURL
chmod +x "${dir}/stub/docker" "${dir}/stub/curl"

run_hosts() {
env HOSTS_STATE_FILE="$1" TAILSCALE_SUMMARY_FILE="${dir}/summary" \
env PATH="${dir}/stub:${PATH}" HOSTS_STATE_FILE="$1" TAILSCALE_SUMMARY_FILE="${dir}/summary" \
bash -c '
log_info(){ echo "$1"; }
log_warning(){ echo "$1" >&2; }
Expand Down Expand Up @@ -2175,11 +2209,63 @@ test_hosts_command() {
rc=0
out="$(run_hosts "${dir}/hosts.tsv" hosts_describe local.spark.loc)" || rc=$?
total=$((total + 1))
if [ "${rc}" -eq 0 ] && echo "${out}" | grep -q "app-1" && echo "${out}" | grep -q "virtual-host"; then
success "describe names the container and how it is routed"
if [ "${rc}" -eq 0 ] && echo "${out}" | grep -q "container app-1" &&
echo "${out}" | grep -q "image node:lts" &&
echo "${out}" | grep -q "status running, up 3 hours" &&
echo "${out}" | grep -q "routed by VIRTUAL_HOST, port 8080" &&
echo "${out}" | grep -q "backend http://172.17.0.5:8080" &&
echo "${out}" | grep -q "network bridge" &&
echo "${out}" | grep -q "reachable 200"; then
success "describe reads image, status, routing port, backend and reachability live"
passed=$((passed + 1))
else
error "describe did not report the container live: $(echo "${out}" | tr '\n' ' ')"
fi

total=$((total + 1))
if echo "${out}" | grep -q "mounts ~/projects/app -> same path (rw)" &&
echo "${out}" | grep -q "app-cache -> /cache (ro)"; then
success "describe renders a same-path bind and a named volume"
passed=$((passed + 1))
else
error "describe rendered the mounts wrong: $(echo "${out}" | grep -A2 mounts | tr '\n' ' ')"
fi

# Redaction by flag name, assignment name and URL userinfo, never by the
# shape of a value. Every planted secret must be gone; the flags must stay.
total=$((total + 1))
if ! echo "${out}" | grep -qE "s3cret|t0k3n|k3y|live|secret'|user:pw" &&
echo "${out}" | grep -q -- "--auth <redacted> --token=<redacted> API_KEY=<redacted> API_TOKEN='<redacted>' https://<redacted>@db.spark.loc/x"; then
success "describe redacts secrets in the command by flag, assignment and URL"
passed=$((passed + 1))
else
error "a secret survived in the command line: $(echo "${out}" | grep command)"
fi

# A container routed by native labels has no VIRTUAL_HOST to report. Its
# rule quotes the host with double quotes and its service is already
# provider-qualified, both of which Traefik's API can return.
rc=0
out="$(run_hosts "${dir}/hosts.tsv" hosts_describe labelled.spark.loc)" || rc=$?
total=$((total + 1))
if [ "${rc}" -eq 0 ] && echo "${out}" | grep -q "routed by traefik.\* labels, port 8080"; then
success "describe names native labels as the routing"
passed=$((passed + 1))
else
error "describe did not report the label-routed host: $(echo "${out}" | tr '\n' ' ')"
fi

# A record whose container Docker no longer has: say so, print what the
# record holds, and fail, rather than inventing a running container.
printf 'gone.spark.loc\tgone-1\t%s/projects/gone\tvirtual-host\n' "${HOME}" >>"${dir}/hosts.tsv"
rc=0
out="$(run_hosts "${dir}/hosts.tsv" hosts_describe gone.spark.loc)" || rc=$?
total=$((total + 1))
if [ "${rc}" -ne 0 ] && echo "${out}" | grep -q "gone-1, not found" && echo "${out}" | grep -q "~/projects/gone"; then
success "describe reports a container that is gone and fails"
passed=$((passed + 1))
else
error "describe did not report the local host: $(echo "${out}" | tr '\n' ' ')"
error "describe on a gone container: exit ${rc}, $(echo "${out}" | tr '\n' ' ')"
fi

rc=0
Expand Down
Loading