Skip to content

Latest commit

 

History

History
352 lines (265 loc) · 13.7 KB

api-proxy-sidecar.md

File metadata and controls

352 lines (265 loc) · 13.7 KB
title description
API Proxy Sidecar
Secure LLM API credential management using an isolated proxy sidecar container.

The AWF firewall supports an optional Node.js-based API proxy sidecar that securely holds LLM API credentials and automatically injects authentication headers while routing all traffic through Squid to respect domain whitelisting.

:::note For a deep dive into how AWF handles authentication tokens and credential isolation, see the Authentication Architecture guide. :::

Overview

When enabled, the API proxy sidecar:

  • Isolates credentials: API keys are never exposed to the agent container
  • Auto-authentication: Automatically injects Bearer tokens and API keys
  • Dual provider support: Supports both OpenAI (Codex) and Anthropic (Claude) APIs
  • Transparent proxying: Agent code uses standard SDK environment variables
  • Squid routing: All traffic routes through Squid to respect domain whitelisting

Architecture

┌─────────────────────────────────────────────────┐
│ AWF Network (172.30.0.0/24)                     │
│                                                  │
│  ┌──────────────┐       ┌─────────────────┐   │
│  │   Squid      │◄──────│  Node.js Proxy  │   │
│  │ 172.30.0.10  │       │  172.30.0.30    │   │
│  └──────┬───────┘       └─────────────────┘   │
│         │                        ▲              │
│         │  ┌──────────────────────────────┐    │
│         │  │      Agent Container         │    │
│         │  │      172.30.0.20             │    │
│         │  │  OPENAI_BASE_URL=            │    │
│         │  │   http://172.30.0.30:10000/v1│────┘
│         │  │  ANTHROPIC_BASE_URL=         │
│         │  │   http://172.30.0.30:10001   │
│         │  └──────────────────────────────┘
│         │
└─────────┼─────────────────────────────────────┘
          │ (Domain whitelist enforced)
          ↓
  api.openai.com or api.anthropic.com

Traffic flow:

  1. Agent makes a request to 172.30.0.30:10000 (OpenAI) or 172.30.0.30:10001 (Anthropic)
  2. API proxy strips any client-supplied auth headers and injects the real credentials
  3. API proxy routes the request through Squid via HTTP_PROXY/HTTPS_PROXY
  4. Squid enforces the domain whitelist (only allowed domains pass)
  5. Request reaches api.openai.com or api.anthropic.com

Usage

Basic usage

# Set API keys in environment
export OPENAI_API_KEY="sk-..."
export ANTHROPIC_API_KEY="sk-ant-..."

# Enable API proxy sidecar
sudo awf --enable-api-proxy \
  --allow-domains api.openai.com,api.anthropic.com \
  -- your-command

Codex (OpenAI) example

export OPENAI_API_KEY="sk-..."

sudo awf --enable-api-proxy \
  --allow-domains api.openai.com \
  -- npx @openai/codex -p "write a hello world function"

The agent container automatically uses http://172.30.0.30:10000/v1 as the OpenAI base URL.

Claude Code example

export ANTHROPIC_API_KEY="sk-ant-..."

sudo awf --enable-api-proxy \
  --allow-domains api.anthropic.com \
  -- claude-code "write a hello world function"

The agent container automatically uses http://172.30.0.30:10001 as the Anthropic base URL.

Both providers

export OPENAI_API_KEY="sk-..."
export ANTHROPIC_API_KEY="sk-ant-..."

sudo awf --enable-api-proxy \
  --allow-domains api.openai.com,api.anthropic.com \
  -- your-multi-llm-tool

Environment variables

AWF manages environment variables differently across the three containers (squid, api-proxy, agent) to ensure secure credential isolation.

Squid container

The Squid proxy container runs with minimal environment variables:

Variable Value Description
HTTP_PROXY Not set Squid is the proxy, not a client
HTTPS_PROXY Not set Squid is the proxy, not a client

API proxy container

The API proxy sidecar receives real credentials and routing configuration:

Variable Value When set Description
OPENAI_API_KEY Real API key --enable-api-proxy and env set OpenAI API key (injected into requests)
ANTHROPIC_API_KEY Real API key --enable-api-proxy and env set Anthropic API key (injected into requests)
COPILOT_GITHUB_TOKEN Real token --enable-api-proxy and env set GitHub Copilot token (injected into requests)
HTTP_PROXY http://172.30.0.10:3128 Always Routes through Squid for domain filtering
HTTPS_PROXY http://172.30.0.10:3128 Always Routes through Squid for domain filtering

:::danger[Real credentials in api-proxy] The api-proxy container holds real, unredacted credentials. These are used to authenticate requests to LLM providers. This container is isolated from the agent and has all capabilities dropped for security. :::

Agent container

The agent container receives redacted placeholders and proxy URLs:

Variable Value When set Description
OPENAI_BASE_URL http://172.30.0.30:10000/v1 OPENAI_API_KEY provided to host Redirects OpenAI SDK to proxy
ANTHROPIC_BASE_URL http://172.30.0.30:10001 ANTHROPIC_API_KEY provided to host Redirects Anthropic SDK to proxy
ANTHROPIC_AUTH_TOKEN placeholder-token-for-credential-isolation ANTHROPIC_API_KEY provided to host Placeholder token (real auth via BASE_URL)
CLAUDE_CODE_API_KEY_HELPER /usr/local/bin/get-claude-key.sh ANTHROPIC_API_KEY provided to host Helper script for Claude Code CLI
COPILOT_API_URL http://172.30.0.30:10002 COPILOT_GITHUB_TOKEN provided to host Redirects Copilot CLI to proxy
COPILOT_TOKEN placeholder-token-for-credential-isolation COPILOT_GITHUB_TOKEN provided to host Placeholder token (real auth via API_URL)
COPILOT_GITHUB_TOKEN placeholder-token-for-credential-isolation COPILOT_GITHUB_TOKEN provided to host Placeholder token protected by one-shot-token
OPENAI_API_KEY Not set --enable-api-proxy Excluded from agent (held in api-proxy)
ANTHROPIC_API_KEY Not set --enable-api-proxy Excluded from agent (held in api-proxy)
HTTP_PROXY http://172.30.0.10:3128 Always Routes through Squid proxy
HTTPS_PROXY http://172.30.0.10:3128 Always Routes through Squid proxy
NO_PROXY localhost,127.0.0.1,172.30.0.30 --enable-api-proxy Bypass proxy for localhost and api-proxy
AWF_API_PROXY_IP 172.30.0.30 --enable-api-proxy Used by iptables setup script
AWF_ONE_SHOT_TOKENS COPILOT_GITHUB_TOKEN,GITHUB_TOKEN,... Always Tokens protected by one-shot-token library

:::tip[Placeholder tokens] Token variables in the agent are set to placeholder-token-for-credential-isolation instead of real values. This ensures:

  • Agent code cannot exfiltrate credentials
  • CLI tools that check for token presence still work
  • Real authentication happens via the *_BASE_URL or *_API_URL environment variables
  • The one-shot-token library protects placeholder values from being read more than once :::

These environment variables are recognized by:

  • OpenAI Python SDK (openai)
  • OpenAI Node.js SDK (openai)
  • Anthropic Python SDK (anthropic)
  • Anthropic TypeScript SDK (@anthropic-ai/sdk)
  • GitHub Copilot CLI (@github/copilot)
  • Codex CLI
  • Claude Code CLI

:::tip You don't need to change any agent code. The SDKs automatically read *_BASE_URL environment variables and redirect API calls through the proxy. :::

Security benefits

Credential isolation

API keys are held in the sidecar container, not the agent:

  • Agent code cannot read API keys from environment variables
  • A compromised agent cannot exfiltrate credentials
  • Keys are not exposed in the agent container's stdout/stderr logs

:::danger[Protect host credentials] API keys are stored in the sidecar container's environment and in the Docker Compose configuration on disk. Protect the host filesystem and configuration accordingly. Only non-sensitive key prefixes are logged for debugging. :::

Network isolation

The proxy enforces domain-level egress control:

  • The agent can only reach the API proxy IP (172.30.0.30) for API calls
  • The sidecar routes all traffic through Squid proxy
  • Squid enforces the domain whitelist (L7 filtering)
  • iptables rules prevent the agent from bypassing the proxy

Resource limits

The sidecar has strict resource constraints:

  • 512 MB memory limit
  • 100 process limit
  • All capabilities dropped
  • no-new-privileges security option

How it works

1. Container startup

When you pass --enable-api-proxy:

  1. AWF starts a Node.js API proxy at 172.30.0.30
  2. API keys are passed to the sidecar via environment variables
  3. HTTP_PROXY/HTTPS_PROXY in the sidecar are configured to route through Squid
  4. The agent container waits for the sidecar health check to pass

2. Request flow

Agent Code
  ↓ (HTTP request to 172.30.0.30:10000/v1)
Node.js API Proxy
  ↓ (strips client auth headers)
  ↓ (injects Authorization: Bearer $OPENAI_API_KEY)
  ↓ (routes via HTTPS_PROXY to Squid)
Squid Proxy
  ↓ (enforces domain whitelist)
  ↓ (TLS connection to api.openai.com)
OpenAI API

3. Header injection

The Node.js proxy automatically:

  • Strips any client-supplied Authorization, x-api-key, Proxy-Authorization, and X-Forwarded-* headers
  • Injects the correct authentication headers:
    • OpenAI: Authorization: Bearer $OPENAI_API_KEY
    • Anthropic: x-api-key: $ANTHROPIC_API_KEY and anthropic-version: 2023-06-01 (if not already set by the client)

:::caution The proxy enforces a 10 MB request body size limit to prevent denial-of-service via large payloads. :::

4. Pre-flight health check

Before running the user command, the agent container runs a health check script (api-proxy-health-check.sh) that verifies:

  • API keys are not present in the agent environment (credential isolation working)
  • The API proxy is reachable and responding (connectivity established)

If either check fails, the agent exits immediately without running the user command.

Configuration reference

CLI options

sudo awf --enable-api-proxy [OPTIONS] -- COMMAND

Required environment variables (at least one):

  • OPENAI_API_KEY — OpenAI API key
  • ANTHROPIC_API_KEY — Anthropic API key

Recommended domain whitelist:

  • api.openai.com — for OpenAI/Codex
  • api.anthropic.com — for Anthropic/Claude

Optional flags for custom upstream endpoints:

Flag Default Description
--openai-api-target <host> api.openai.com Custom upstream for OpenAI API requests (e.g. Azure OpenAI or an internal LLM router). Can also be set via OPENAI_API_TARGET env var.
--anthropic-api-target <host> api.anthropic.com Custom upstream for Anthropic API requests (e.g. an internal Claude router). Can also be set via ANTHROPIC_API_TARGET env var.
--copilot-api-target <host> auto-derived Custom upstream for GitHub Copilot API requests (useful for GHES). Can also be set via COPILOT_API_TARGET env var.

Important: When using a custom --openai-api-target or --anthropic-api-target, you must add the target domain to --allow-domains so the firewall permits outbound traffic. AWF will emit a warning if a custom target is set but not in the allowlist.

Container configuration

The sidecar container:

  • Image: ghcr.io/github/gh-aw-firewall/api-proxy:latest
  • Base: node:22-alpine
  • Network: awf-net at 172.30.0.30
  • Ports: 10000 (OpenAI), 10001 (Anthropic), 10002 (GitHub Copilot)
  • Proxy: Routes via Squid at http://172.30.0.10:3128

Health check

Docker healthcheck on the /health endpoint (port 10000):

  • Interval: 5s
  • Timeout: 3s
  • Retries: 5
  • Start period: 5s

Troubleshooting

API keys not detected

⚠️  API proxy enabled but no API keys found in environment
   Set OPENAI_API_KEY or ANTHROPIC_API_KEY to use the proxy

Solution: Export API keys before running awf:

export OPENAI_API_KEY="sk-..."
# or
export ANTHROPIC_API_KEY="sk-ant-..."

Sidecar health check failing

Check if the API proxy container started:

docker ps | grep awf-api-proxy

View API proxy logs:

docker logs awf-api-proxy

API requests timing out

Ensure the API domains are whitelisted:

sudo awf --enable-api-proxy \
  --allow-domains api.openai.com,api.anthropic.com \
  -- your-command

Check Squid logs for denied requests:

docker exec awf-squid cat /var/log/squid/access.log | grep DENIED

Limitations

  • Only supports OpenAI and Anthropic APIs
  • Keys must be set as environment variables (not file-based)
  • No support for Azure OpenAI endpoints
  • No request/response logging (by design, for security)

Related documentation