Backend Engineering Manager, Cloud Inference

Job description

About the role: Backend Engineering Manager, Cloud Inference At Modular, we've rebuilt our inference stack from the ground up. Powering state-of-the-art models at state-of-the-art performance, with full portability across hardware.The Cloud Inference team builds the backend systems that power large-scale LLM inference on Kubernetes—reliable, secure, and performance-obsessed. This role manages a team of backend engineers responsible for the services and control plane around distributed inference (multi-node serving, fleet management, routing, observability, incident response, and operational excellence).You will lead a high-impact group working at the intersection of distributed systems and AI infrastructure. You will partner closely with product and other engineering teams to deliver customer outcomes while continuously raising the bar on reliability (availability, latency, and throughput) and operational maturity. LOCATION: Candidates based in the US or Canada are welcome to apply. You can work in our office in Los Altos, CA or remotely from home. Onboarding for new hires is conducted in-person in our Los Altos, CA office. What you will do: - Lead and grow a team of backend engine...