feat(build): mTLS for buildkitd + public exposure via TLS passthrough

cert-manager self-signed CA issues server and client certs for BuildKit mTLS. Buildkitd serves TLS on its ClusterIP (hostNetwork removed) and is publicly reachable at build.DOMAIN_SUFFIX:443 through Pingora's new SNI-based TLS passthrough router. Clients authenticate with the client certificate from the buildkitd-client-tls secret.
2026-03-26 14:23:56 +00:00
parent 632099893a
commit 33f0e44545
5 changed files with 149 additions and 9 deletions
--- a/base/build/buildkitd-deployment.yaml
+++ b/base/build/buildkitd-deployment.yaml
@@ -15,21 +15,20 @@ spec:
      labels:
        app: buildkitd
    spec:
-      # Use host network so buildkitd can push to src.DOMAIN_SUFFIX (Gitea registry
-      # via Pingora) without DNS resolution issues. The registry runs on the same
-      # node, so host networking routes traffic back to localhost directly.
-      hostNetwork: true
-      dnsPolicy: None
-      dnsConfig:
-        nameservers:
-          - 8.8.8.8
-          - 1.1.1.1
+      # No hostNetwork — buildkitd is accessed via the ClusterIP service.
+      # Public access goes through Pingora's TLS passthrough (SNI router).
      containers:
        - name: buildkitd
          image: moby/buildkit:v0.28.0
          args:
            - --addr
            - tcp://0.0.0.0:1234
+            - --tlscacert
+            - /etc/buildkit/tls/ca.crt
+            - --tlscert
+            - /etc/buildkit/tls/tls.crt
+            - --tlskey
+            - /etc/buildkit/tls/tls.key
          ports:
            - containerPort: 1234
          securityContext:
@@ -41,3 +40,23 @@ spec:
            limits:
              cpu: "4"
              memory: "8Gi"
+          volumeMounts:
+            - name: server-tls
+              mountPath: /etc/buildkit/tls
+              readOnly: true
+      volumes:
+        - name: server-tls
+          projected:
+            sources:
+              - secret:
+                  name: buildkitd-server-tls
+                  items:
+                    - key: tls.crt
+                      path: tls.crt
+                    - key: tls.key
+                      path: tls.key
+              - secret:
+                  name: buildkit-ca-keypair
+                  items:
+                    - key: ca.crt
+                      path: ca.crt
--- a/base/build/buildkitd-mtls.yaml
+++ b/base/build/buildkitd-mtls.yaml
@@ -0,0 +1,93 @@
+# mTLS certificate infrastructure for BuildKit.
+#
+# Self-signed CA → server cert (for buildkitd) + client cert (for CLI).
+# This allows buildkitd to be publicly exposed through Pingora's TLS
+# passthrough while requiring client certificate authentication.
+#
+# cert-manager must be installed before applying this.
+---
+# ── CA Issuer ────────────────────────────────────────────────────────────────
+# Self-signed issuer bootstraps the CA keypair.
+apiVersion: cert-manager.io/v1
+kind: Issuer
+metadata:
+  name: buildkit-selfsign
+  namespace: build
+spec:
+  selfSigned: {}
+---
+# CA certificate — signs both server and client certs.
+apiVersion: cert-manager.io/v1
+kind: Certificate
+metadata:
+  name: buildkit-ca
+  namespace: build
+spec:
+  isCA: true
+  commonName: buildkit-ca
+  secretName: buildkit-ca-keypair
+  duration: 87600h    # 10 years
+  renewBefore: 8760h  # renew 1 year early
+  privateKey:
+    algorithm: ECDSA
+    size: 256
+  issuerRef:
+    name: buildkit-selfsign
+    kind: Issuer
+---
+# Issuer that signs certs using the CA above.
+apiVersion: cert-manager.io/v1
+kind: Issuer
+metadata:
+  name: buildkit-ca-issuer
+  namespace: build
+spec:
+  ca:
+    secretName: buildkit-ca-keypair
+---
+# ── Server certificate (for buildkitd) ──────────────────────────────────────
+apiVersion: cert-manager.io/v1
+kind: Certificate
+metadata:
+  name: buildkitd-server
+  namespace: build
+spec:
+  secretName: buildkitd-server-tls
+  duration: 8760h     # 1 year
+  renewBefore: 720h   # renew 30 days early
+  privateKey:
+    algorithm: ECDSA
+    size: 256
+  usages:
+    - digital signature
+    - key encipherment
+    - server auth
+  dnsNames:
+    - buildkitd
+    - buildkitd.build.svc.cluster.local
+    - build.DOMAIN_SUFFIX
+  issuerRef:
+    name: buildkit-ca-issuer
+    kind: Issuer
+---
+# ── Client certificate (for Sunbeam CLI) ────────────────────────────────────
+apiVersion: cert-manager.io/v1
+kind: Certificate
+metadata:
+  name: buildkitd-client
+  namespace: build
+spec:
+  secretName: buildkitd-client-tls
+  duration: 8760h
+  renewBefore: 720h
+  privateKey:
+    algorithm: ECDSA
+    size: 256
+  usages:
+    - digital signature
+    - key encipherment
+    - client auth
+  commonName: sunbeam-cli
+  issuerRef:
+    name: buildkit-ca-issuer
+    kind: Issuer
--- a/base/build/kustomization.yaml
+++ b/base/build/kustomization.yaml
@@ -3,5 +3,6 @@ kind: Kustomization

 resources:
  - namespace.yaml
+  - buildkitd-mtls.yaml
  - buildkitd-deployment.yaml
  - buildkitd-service.yaml
--- a/base/ingress/pingora-config.yaml
+++ b/base/ingress/pingora-config.yaml
@@ -362,6 +362,13 @@ data:
      prefix  = "/.well-known/matrix"
      backend = "http://tuwunel.matrix.svc.cluster.local:6167"

+    # TLS passthrough: SNI-routed connections relayed without TLS termination.
+    # BuildKit uses mTLS (client certs) so Pingora can't terminate — it peeks
+    # the ClientHello SNI and relays the raw TCP stream to the backend.
+    [[tls_passthrough]]
+    host_prefix = "build"
+    backend     = "buildkitd.build.svc.cluster.local:1234"
+
    # SSH TCP passthrough: port 22 → Gitea SSH pod (headless service → pod:2222).
    [ssh]
    listen  = "0.0.0.0:22"
--- a/docs/proxy.md
+++ b/docs/proxy.md
@@ -100,6 +100,7 @@ Every subdomain gets routed by prefix (the part before the first dot). The bounc
 | `hydra` | Hydra admin (auth-gated) | subrequest to `/userinfo` |
 | `search` | OpenSearch (auth-gated) | subrequest to `/userinfo` |
 | `vault` | OpenBao (auth-gated) | subrequest to `/userinfo` |
+| `build` | BuildKit daemon | TLS passthrough (mTLS) |

 Path sub-routes use longest-prefix-first matching within each host.

@@ -129,6 +130,25 @@ Pure Rust, no C dependencies in the TLS stack.
 - **Local:** mkcert wildcard cert
 - **Production:** Let's Encrypt via cert-manager (ACME HTTP-01 challenges routed by the proxy itself)

+### SNI-Based TLS Passthrough
+
+Some backends handle their own TLS (e.g. buildkitd with mTLS client certificate auth). The proxy supports **TLS passthrough** — peeking at the TLS ClientHello SNI without terminating the connection, then relaying the raw TCP stream to the backend.
+
+```toml
+[[tls_passthrough]]
+host_prefix = "build"
+backend     = "buildkitd.build.svc.cluster.local:1234"
+```
+
+When `tls_passthrough` routes are configured, the proxy's HTTPS listener splits into two layers:
+
+1. **SNI router** on `:443` — peeks the first ~1536 bytes of each connection, parses the ClientHello for the SNI hostname
+2. **Pingora TLS** on `127.0.0.1:10443` — normal TLS termination for HTTP traffic
+
+If the SNI matches a passthrough route, the connection is relayed directly to the backend (no TLS termination, no HTTP inspection). Otherwise it's forwarded to Pingora for normal processing. Non-TLS traffic and connections without SNI fall through to Pingora.
+
+When no `tls_passthrough` routes are configured, Pingora binds `:443` directly — zero overhead.
+
 ## ML Training Pipeline

 The models aren't downloaded — they're compiled into the binary. Weights baked in = zero model file overhead, L1-cache-resident inference.