The Hidden Egress Traps in Kubernetes

Data transfer quietly becomes the largest line item in multi-service architectures. Kubernetes makes it easy to create expensive paths without noticing. Here’s how to find and fix them.

Map the expensive paths

• Cross-AZ traffic: Services talking across availability zones incur double egress. Co-locate chatty microservices or use topology spread rules.
• Ingress/egress mix-ups: Public Ingress for internal traffic forces traffic to exit and re-enter. Use internal load balancers for service-to-service calls.
• Object storage habits: Apps pulling from S3/GCS in a different region pay egress on every request. Cache in-cluster or replicate buckets.
• Service mesh tax: Sidecars add bytes; tune telemetry sampling and avoid mTLS cross-zone when not needed.

Instrument cost visibility

• Emit egress.bytes and cross_az.bytes per namespace via CNI/flow logs.
• Tag traffic with service, team, and env labels to allocate back to owners.
• Set alerts on sudden jumps in cross-AZ bytes and on top talkers.

Quick wins

• Pin stateful services and their clients to the same AZ where possible.
• Switch to private endpoints for managed databases and queues.
• Enable response caching on high-QPS read endpoints; throttle chatty health checks.
• Right-size log shipping; avoid exporting full-body payloads across zones.

Governance that sticks

• Add a policy check: reject manifests missing topologySpreadConstraints for noisy namespaces.
• Run monthly traffic reviews with heatmaps that show which links cost the most.
• Tie quota requests to egress estimates; if a team adds a new region, they must include projected transfer cost.

You cannot optimize what you cannot see. Once egress paths are visible, fixing them is mostly about better defaults—keep traffic local, private, and cached.***