Add upstream proxy

[eraow]

Note: This PR was generated with AI assistance.

Summary

Adds support for routing httpjail's outbound requests through an upstream proxy using the standard HTTP_PROXY and HTTPS_PROXY environment variables.

Motivation

I want to run httpjail behind a corporate proxy. Without upstream proxy support, httpjail connects directly to destination hosts, which fails in environments where direct egress is blocked.

What this does

• Uses HTTP_PROXY for HTTP destinations.
• Uses HTTPS_PROXY for HTTPS destinations.
• Also accepts the lowercase http_proxy and https_proxy variants.
• Supports http://, https://, and bare host:port proxy addresses.
• Supports Basic authentication through proxy URLs.
• Uses a direct connection when no proxy is configured for the destination scheme.
• Sends HTTPS requests through a CONNECT tunnel and forwards plain HTTP requests in absolute-form.
• Applies timeouts to proxy setup operations without limiting established long-running connections such as WebSocket or gRPC.
• Prevents inherited proxy settings and credentials from leaking into strongly jailed commands or Docker.
• Keeps upstream proxy initialization logs at debug level so normal CLI output is unaffected.

Proxy configuration is environment-only; no additional command-line option is introduced. Behavior is unchanged when neither proxy environment variable is configured.

Usage

Route HTTPS requests through a corporate proxy:

  httpjail --js-file rules.js -- curl https://github.com

Route both HTTP and HTTPS requests through the same proxy:

HTTP_PROXY=http://proxy.example.com:8080 \
HTTPS_PROXY=http://proxy.example.com:8080 \
  httpjail --js-file rules.js -- ./my-app

Proxy credentials and HTTPS proxies are also supported:

HTTPS_PROXY=http://user:pass@proxy.example.com:8080 \
  httpjail --js-file rules.js -- curl https://github.com

HTTPS_PROXY=https://proxy.example.com:8443 \
  httpjail --js-file rules.js -- curl https://github.com

Manual verification

Given rules allowing only github.com:

const DOMAINS = [
  "github.com",
];

(function () {
  const h = (r.host || "").toLowerCase();
  for (const d of DOMAINS) {
    if (d[0] === ".") {
      if (h === d.slice(1) || h.endsWith(d)) return true;
    } else if (h === d) {
      return true;
    }
  }
  return false;
})();

Allowed request

$ sudo env -u SUDO_UID -u SUDO_GID \
    HTTPS_PROXY=http://proxy.example.com:8080 \
    target/debug/httpjail -vv \
    --js-file rules.js \
    --request-log jail.log \
    curl -s -o /dev/null -w "%{http_code}\n" https://github.com
...
DEBUG httpjail: Routing httpjail upstream requests through the proxy environment
DEBUG httpjail::proxy: Upstream client initialized to route through the upstream proxy
...
200
...

Blocked request

$ sudo env -u SUDO_UID -u SUDO_GID \
    HTTPS_PROXY=http://proxy.example.com:8080 \
    target/debug/httpjail -vv \
    --js-file rules.js \
    --request-log jail.log \
    curl -s -o /dev/null -w "%{http_code}\n" https://example.com
...
403
...

[ammario]

Why shouldn't httpjail itself respect the standard HTTP_PROXY variables? I think it's clear on its face it wouldn't pass that down to children (as that would invalidate the whole point of the jail).

[eraow]

Agreed. For the first release, I’ll remove --upstream-proxy and HTTPJAIL_UPSTREAM_PROXY and use the standard HTTP_PROXY / HTTPS_PROXY environment variables for httpjail’s own egress.
Adding a dedicated CLI option and httpjail-specific env var creates an extra configuration path before we have a concrete need for it. If we later need an explicit per-invocation override, we can add it in a follow-up change.

[eraow]

Done in 701d116.

[eraow]

@ammario following up on 701d116, which drops --upstream-proxy / HTTPJAIL_UPSTREAM_PROXY in favor of the standard HTTP_PROXY / HTTPS_PROXY for httpjail's own egress.

One question: did you intend for the dedicated flag and env var to be removed entirely, or kept as an explicit override alongside HTTP_PROXY? I went with full removal, but it's easy to bring back either way. Let me know how this looks.

[ammar-agent]

I reviewed this against the latest main (f66518b). The core approach—a Hyper connector that preserves streaming, pooling, and unbounded established connections—is reasonable, so I don't think the raw line count alone warrants a rewrite. I do think a few changes are needed before merging:

1. NO_PROXY / no_proxy is not honored. With HTTP_PROXY or HTTPS_PROXY set, every destination of that scheme is proxied, including internal hosts that a standard corporate proxy environment expects to connect to directly. Please either implement bypass matching or explicitly narrow the documented semantics.
2. Proxy configuration is silently process-global through HTTPS_CLIENT.get_or_init. If one ProxyServer initializes the client, a later call to the public new_with_upstream_proxies constructor can ignore its proxy argument. Keeping the client on ProxyServer / ProxyContext would make the configuration honest and remove hidden global state.
3. The most intricate path lacks an end-to-end test: HTTPS destination through an upstream CONNECT proxy, followed by destination TLS and an HTTP response. The existing tests separately cover plain HTTP proxying and the CONNECT exchange.
4. read_connect_status performs one async read per byte. That avoids swallowing tunnel payload, but is contrary to the project's minimal-latency goal. Please use buffered reads while preserving any bytes read beyond the CONNECT headers.

For complexity cleanup, the manual RunArgs Debug implementation appears to be residue from the removed dedicated proxy option and can return to a derive. Test-only/public helpers such as ProxyConnector::new and UpstreamProxies::all could also be narrowed. If TLS-to-proxy (https://proxy) is not a demonstrated requirement for the first release, deferring it would remove a meaningful portion of the stream-erasure and second-TLS-config machinery while still supporting HTTPS destinations through an ordinary HTTP proxy.

Local verification: formatting and cargo clippy --all-targets -- -D warnings pass; all unit tests pass. The full suite encountered the unchanged parallel fixed-port collision in weak_integration_max_tx_bytes.

— Codex, AI review agent