Engineering Manager, LLM Performance

At NVIDIA, wearen'tjust powering the AI revolution—we'reaccelerating it.We are accelerating LLM inference across the stackand across allopen sourceLLM frameworks like TensorRT LLM,vLLMandSGLang.With demand for AI exploding, particularly in the realm of large language models (LLMs) and vision language models (VLMs, VLAs), we are significantly expanding our team.

We'reseekinga highly skilled and driven Engineering Manager to take the lead inacceleratingthe next generation of LLM/VLM/VLA inference software technologies that will define the future of AI. This is a high-impact, hands-on leadership role at the intersection of deep technicalexpertiseand world-class management. Youwon'tjust manage;you'llarchitect and guide a brilliant team of engineers who arepushing the performance ofLLM inference. Your work will be highly collaborative, interfacing directly with NVIDIA Researchers, GPU Architects, and other teams across the company to ensure we ship production-grade, lightning-fast software that sets the global standard for AI performance.

WhatYou’llBe Doing:

• Lead and grow a team responsible forpushing the performance of LLM inference across multiple LLM frameworks, including TensorRT LLM,vLLM,SGLangand Dynamoon our datacenter products.

• Drive the design,implementationand optimization of features that are key to performance in LLM inference.

• Continuously improve the performance of LLM inference on current and upcoming NVIDIA datacenter architectures and GPUs.

• Continuously improvethe performance of LLM inference ofimportant foundation models.

• Work with inference benchmark teams to helptune performance for key workloads.

• Integratingcutting-edgetechnologies developed at NVIDIA and offering an intuitive developer experience for LLM deployment.

• Lead software development execution, with responsibility for project planning, milestone delivery, and cross-functional coordination.

What We Need to See:

• MS, PhD, or equivalent experience in Computer Science, Computer Engineering, AI, ora relatedtechnical field.

• 7+ overall years of overall software engineering experience, including 3+ years of technical leadership experience.

• Proven ability to lead and scale high-performing engineering teams, especially across distributed and cross-functional groups.

• Strong background in C++ or Python, withexpertisein software design and delivering production-quality software libraries.

• Demonstratedexpertisein large language models (LLM) and/or vision language models (VLM)and/or inference in general.

Ways to Stand Out from the Crowd:

• Deep understanding of GPU architecture, CUDA programming, and system-level performance tuning.

• Background in LLM inference or working with frameworks such as TensorRT-LLM,vLLM, orSGLang.

• Passion for building scalable, user-friendly APIs and enabling developers in the AI ecosystem.

• Have a proventrack recordof growing and managing a team that encourages idea sharing, empowers team members, and provides opportunities for professional growth.