Engineering Manager, Inference Routing and Performance

Job description

About Anthropic Anthropic’s mission is to create reliable, interpretable, and steerable AI systems. We want AI to be safe and beneficial for our users and for society as a whole. Our team is a quickly growing group of committed researchers, engineers, policy experts, and business leaders working together to build beneficial AI systems. About the role: Every request that hits Claude — from claude.ai , the API, our cloud partners, or internal research — passes through a routing decision. Not a generic load balancer round-robin, but a decision that accounts for what's already cached where, which accelerator the request runs best on, and what else is in flight across the fleet. Get it right and you extract meaningfully more throughput from the same hardware. Get it wrong and you burn capacity, miss latency SLOs, or shed load that shouldn't have been shed. The Inference Routing team owns this layer. We build the cluster-level routing and coordination plane for Anthropic's inference fleet — the system that sits between the API surface and the inference engines themselves, making fleet-wide efficiency decisions in real time. As Anthropic moves from "many independent inference replicas" to...