proxy/AGENTS.md

AGENTS.md — sunbeam-proxy

Critical Rules

Read before you write. Read every file you intend to modify. Do not guess at code structure, function signatures, or types. This is a ~500-line Rust codebase — read the actual source.

Minimal changes only. Do exactly what is asked. Do not:

• Add features, abstractions, or "improvements" beyond the request
• Refactor surrounding code, rename variables, or restructure modules
• Add comments, docstrings, or type annotations to code you didn't change
• Add error handling or validation for scenarios that cannot happen
• Create helper functions or utilities for one-off operations
• Add backwards-compatibility shims, re-exports, or // removed comments
• Introduce new dependencies without being explicitly asked

Do not create new files unless the task absolutely requires it. Prefer editing existing files.

Do not over-engineer. Three similar lines of code is better than a premature abstraction. If a fix is one line, submit one line.

Ask before acting on anything destructive or irreversible: deleting files, force-pushing, modifying CI, running commands with side effects.

Project Overview

sunbeam-proxy is a TLS-terminating reverse proxy built on Pingora 0.8 (Cloudflare's proxy framework) with rustls. It runs in Kubernetes and handles:

• Host-prefix routing: routes foo.example.com by matching prefix foo against the config
• Path sub-routes: longest-prefix match within a host, with optional prefix stripping
• Static file serving: try_files chain with SPA fallback, replacing nginx/caddy for frontends
• URL rewrites: regex-based path rewrites compiled at startup
• Response body rewriting: find/replace in HTML/JS responses (like nginx sub_filter)
• Auth subrequests: gate path routes with HTTP auth checks (like nginx auth_request)
• HTTP response cache: per-route in-memory cache via pingora-cache with Cache-Control support
• Prometheus metrics: request totals, latency histograms, detection decisions, cache hit/miss
• Request IDs: UUID v4 per request, forwarded to upstreams and clients via X-Request-Id
• DDoS detection: KNN-based per-IP behavioral classification
• Scanner detection: logistic regression per-request classification with bot allowlist
• Rate limiting: leaky bucket per-identity throttling
• ACME HTTP-01 challenges: routes /.well-known/acme-challenge/* to cert-manager solver pods
• TLS cert hot-reload: watches K8s Secrets, writes cert files, triggers zero-downtime upgrade
• Config hot-reload: watches K8s ConfigMaps, triggers graceful upgrade on change
• SSH TCP passthrough: raw TCP proxy for SSH traffic (port 22 to Gitea)
• HTTP-to-HTTPS redirect: with per-route opt-out via disable_secure_redirection

See README.md for full feature documentation and configuration reference.

Source Files

src/main.rs          — binary entry point: server bootstrap, watcher spawn, SSH spawn
src/lib.rs           — library crate root: re-exports all modules
src/config.rs        — TOML config deserialization (Config, RouteConfig, PathRoute, CacheConfig, etc.)
src/proxy.rs         — ProxyHttp impl: request_filter, cache hooks, upstream_peer, body rewriting, logging
src/acme.rs          — Ingress watcher: maintains AcmeRoutes (path → solver backend)
src/watcher.rs       — Secret/ConfigMap watcher: cert write + graceful upgrade trigger
src/cert.rs          — fetch_and_write / write_from_secret: K8s Secret → cert files on disk
src/telemetry.rs     — JSON logging + optional OTEL tracing init
src/ssh.rs           — TCP proxy: tokio TcpListener + copy_bidirectional
src/metrics.rs       — Prometheus counters/histograms/gauge, metrics HTTP server, /health endpoint
src/static_files.rs  — Static file serving with try_files chain and SPA fallback
src/cache.rs         — pingora-cache MemCache backend and Cache-Control TTL parser
src/ddos/            — KNN-based DDoS detection (model, detector, training, replay)
src/scanner/         — Logistic regression scanner detection (model, detector, features, training, allowlist, watcher)
src/rate_limit/      — Leaky bucket rate limiter (limiter, key extraction)
src/dual_stack.rs    — Dual-stack (IPv4+IPv6) TCP listener
tests/e2e.rs         — end-to-end test: real SunbeamProxy over plain HTTP with echo backend
tests/proptest.rs    — property-based tests for static files, rewrites, config, metrics, etc.

Architecture Invariants — Do Not Break These

1. Separate OS threads for K8s watchers. The cert/config watcher and Ingress watcher run on their own std::thread with their own tokio::runtime. Pingora has its own internal runtime. Never share a tokio runtime between Pingora and the watchers.

2. Fresh K8s Client per runtime. Each runtime creates its own kube::Client. Tower workers are tied to the runtime that created them. Do not pass a Client across runtime boundaries.

3. std::sync::RwLock for AcmeRoutes, not tokio::sync. The RwLock guard is held across code paths in Pingora's async proxy calls. A tokio RwLock guard is Send but the waker cross-runtime issues make std::sync::RwLock the correct choice here.

4. insert_header() not headers.insert() on Pingora RequestHeader. Pingora maintains a CaseMap alongside base.headers. Using headers.insert() directly causes the header to be silently dropped during header_to_h1_wire serialization. Always use insert_header() or remove_header().

5. rustls crypto provider must be installed first. rustls::crypto::aws_lc_rs::default_provider().install_default() must run before any TLS initialization. This is in main() line 19.

6. Cert watcher writes certs from the Apply event payload. It does NOT re-fetch via the API. The watcher::Event::Apply(secret) carries the full Secret object; cert::write_from_secret writes directly from it, then triggers the upgrade.

7. Graceful upgrade = spawn new process with --upgrade + SIGQUIT self. Pingora transfers listening socket FDs via Unix socket. The new process inherits them. Do not change this flow.

Build & Test Commands

# Build (debug)
cargo build

# Build (release, with cross-compile for linux-musl if needed)
cargo build --release --target x86_64-unknown-linux-musl

# Run all tests (unit + e2e)
cargo test

# Run unit tests only (no e2e, which needs port 18889)
cargo test --lib

# Check without building
cargo check

# Lint
cargo clippy -- -D warnings

# Format check
cargo fmt -- --check

Always run cargo check after making changes. If it doesn't compile, fix it before proceeding. Do not submit code that doesn't compile.

Run cargo test after any behavioral change. The e2e test in tests/e2e.rs spins up a real proxy and echo backend — it catches real regressions.

Run cargo clippy -- -D warnings before finishing. Fix all warnings. Do not add #[allow(...)] attributes to suppress warnings unless there is a genuine false positive.

Rust & Pingora Conventions in This Codebase

• Error handling: anyhow::Result for fallible startup code, pingora_core::Result inside ProxyHttp trait methods. Map between them with pingora_core::Error::because().
• Logging: tracing::info!, tracing::warn!, tracing::error! with structured fields. The logging() method on the proxy uses target = "audit" for the request log line.
• No unwrap() in production paths. Use expect() only where failure is genuinely impossible and the message explains why. unwrap_or_else(|e| e.into_inner()) is acceptable for poisoned locks.
• Async: async_trait for the ProxyHttp impl. Pingora controls the runtime; do not spawn tasks on it.
• Config: All configuration is in TOML, deserialized with serde. Add new fields to the existing structs in config.rs with #[serde(default)] for backwards compatibility.
• Tests: Unit tests go in #[cfg(test)] mod tests inside the relevant source file. Integration tests go in tests/e2e.rs.

Common Mistakes to Avoid

• Do not add tokio::main to main.rs. Pingora manages its own runtime via server.run_forever(). The temporary runtimes for cert fetch and SSH are intentionally separate.
• Do not use headers.insert() on upstream requests. Use insert_header(). See invariant #4.
• Do not hold a RwLock guard across .await points in proxy.rs. Clone the data out, drop the guard, then proceed.
• Do not add new crate dependencies for things the existing deps already cover. Check Cargo.toml first.
• Do not modify the graceful upgrade flow (watcher.rs trigger_upgrade) unless explicitly asked. It is deliberately simple and correct.
• Do not add #[tokio::test] to e2e tests. They use std::thread + raw TcpStream intentionally to avoid runtime conflicts with Pingora.