Staff Software Engineer
Automate your job search with Sonara.
Submit 10x as many applications with less effort than one manual application.1
Reclaim your time by letting our AI handle the grunt work of job searching.
We continuously scan millions of openings to find your top matches.
Job Description
About Us
Hippocratic AI is developing the first safety-focused Large Language Model (LLM) for healthcare. Our mission is to dramatically improve healthcare accessibility and outcomes by bringing deep healthcare expertise to every person. No other technology has the potential for this level of global impact on health.
Why Join Our Team
Innovative mission: We are creating a safe, healthcare-focused LLM that can transform health outcomes on a global scale.
Visionary leadership: Hippocratic AI was co-founded by CEO Munjal Shah alongside physicians, hospital administrators, healthcare professionals, and AI researchers from top institutions including El Camino Health, Johns Hopkins, Washington University in St. Louis, Stanford, Google, Meta, Microsoft and NVIDIA.
Strategic investors: We have raised a total of $278 million in funding, backed by top investors such as Andreessen Horowitz, General Catalyst, Kleiner Perkins, NVIDIA’s NVentures, Premji Invest, SV Angel, and six health systems.
Team and expertise: We are working with top experts in healthcare and artificial intelligence to ensure the safety and efficacy of our technology.
For more information, visit www.HippocraticAI.com.
We value in-person teamwork and believe the best ideas happen together. Our team is expected to be in the office five days a week in Palo Alto, CA unless explicitly noted otherwise in the job description.
About the Role
As a Staff Software Engineer, you will work alongside a dynamic team of engineers, applied scientists, and healthcare professionals to drive advancements in our generative AI technologies. Your focus will be on building and optimizing the core systems and services that power model inference at scale.
Responsibilities
Build control plane components around scheduling and capacity planning of cross-cloud distributed inference platform.
Architect, develop, and maintain scalable backend infrastructure to support high-performance healthcare AI applications.
Contribute to platform capabilities such as latency-aware routing, model versioning, health monitoring, and observability.
Improve performance, autoscaling, GPU utilization, and resource efficiency in a cloud-native environment. Work across product, and ML teams to ensure the inference platform meets the scale, reliability, and latency demands of our use cases.
Gain hands-on experience with tools like llm-d, SGLang and container orchestration with Kubernetes
Mentor junior engineers, providing guidance on technical projects and fostering a culture of collaboration and innovation.
Qualifications
Must-Have:
Bachelor’s or Master’s degree in Computer Science, Computer Engineering, or a related field.
7+ years of experience in backend development with proficiency in languages like Python, Java, or Go.
Strong understanding of SaaS, and cloud orchestration software. (Control Plane)
In-depth knowledge of one or more of cloud platforms (AWS, GCP, or Azure) including Docker, Kubernetes, and cloud networking.
Familiar with concepts in ML model serving and inference runtimes
Excellent communication skills and experience working collaboratively within cross-functional teams.
Preferred:
Experience integrating infrastructure with production ML workloads.
Experience in model serving, inference, observability, and distributed infrastructure
Previous experience in healthcare-related technologies or regulated data environments.
Why You’ll Love Working Here: At Hippocratic AI, we’re reshaping healthcare with breakthrough technology. If you’re excited by innovation and eager to impact real-world healthcare challenges, you’ll thrive here.
Automate your job search with Sonara.
Submit 10x as many applications with less effort than one manual application.