AI Trust Data Scientist II

As a Data Scientist II on the AI Trust team, you will be a key driver in ensuring our artificial intelligence systems are safe, reliable, and effective. You will own and conduct critical analyses and experiments to measure the real-world impact of our AI, helping to define and validate that our systems are technically robust, trustworthy, and outcome-driven. In this deeply cross-functional position, you will collaborate with partners in engineering, product, and other functions to implement the standards and systems that make "safe and effective" AI a reality.

Please note that candidates for this position must be based in the Salt Lake City metro area and be willing to commute 2-3 days a week when this role transitions to a hybrid schedule in 2026. We're excited to be growing our presence in Salt Lake City!

What you'll do:

• Own and evolve the evaluation frameworks for our AI and ML models, translating high-level trust principles into specific, measurable tests.
• Define and conduct rigorous experiments to resolve ambiguous questions about the safety, reliability, and impact of our models.
• Collaborate with engineering partners to design and build production-quality code, creating automated, scalable, and pragmatic testing frameworks based on modern best practices.
• Partner with product, legal, and infrastructure teams to implement and monitor standards for trustworthy AI.
• Proactively identify gaps and develop novel evaluation approaches, which may include creating synthetic test data from user traces or building lightweight processes for non-technical partners to iterate on test sets.
• Synthesize complex evaluation results and industry trends into actionable insights and clearly communicate findings to diverse technical and non-technical stakeholders.

What success looks like:

• Measurable improvements in key performance and safety metrics on a quarterly basis.
• Successful design and delivery on POC experiments to improve our base LLM systems.
• Timely and successful delivery of AI evaluation readouts and safety reviews that directly inform key product improvements
• Demonstrated impact on our AI strategy through proactive identification of risks and opportunities for enhancement, confirmed by stakeholder feedback.
• Increased efficiency and coverage of our AI evaluation process, driven by your ownership of and improvements to automated testing frameworks and best practices.

What you'll bring:

• 2-3 + years of relevant industry experience in data science, machine learning, or a related field.
• Proficiency in Python and a solid understanding of core statistical concepts. You have a proven ability to write and review production-quality code.
• Proven experience in evaluating machine learning models, with exposure to large language models (LLMs) being a strong plus.
• Hands-on experience in one or more of the following areas:
• Analyzing A/B tests or other experiments with statistical rigor.
• Using evaluation tools (e.g., LangSmith, open-source libraries) to iteratively measure and improve model performance.
• Building data pipelines or tools to enable collaboration on test sets.
• Applied knowledge of concepts in AI ethics, such as fairness, bias, and interpretability.
• A strong interest in applying data science to complex, high-stakes domains like mental healthcare. You are motivated by our mission to remove every barrier to mental health. A pragmatic and proactive approach to problem-solving, with a history of developing creative solutions to complex problems.
• Exceptional communication and teamwork skills, with a proven ability to collaborate effectively with diverse, cross-functional teams.
• An avid learning mindset and a passion for staying at the forefront of trends in AI safety, evaluation, and reliability.

The target base salary range for this position is $120,000 - $150,250, and is part of a competitive total rewards package including stock options and benefits. Individual pay may vary from the target range and is determined by a number of factors including experience, location, internal pay equity, and other relevant business considerations. We review all employee pay and compensation programs annually using Radford Global Compensation Database at minimum to ensure competitive and fair pay.

Benefits provided by Spring Health:

Note: We have even more benefits than listed here and below, your recruiter will provide more in-depth information as you continue in the interview process. Benefits are subject to individual plan requirements and eligibility criteria.

• Health, Dental, Vision benefits start on your first day at Spring. You and your dependents also receive access to One Medical accounts HSA and FSA plans are also available, with Spring contributing up to $1K for HSAs, depending on your plan type.
• Employer sponsored 401(k) match of up to 2% for retirement planning
• A yearly allotment of no cost visits to the Spring Health network of therapists, coaches, and medication management providers for you and your dependents.
• We offer competitive paid time off policies including vacation, sick leave and company holidays.
• At 6 months tenure with Spring, we offer parental leave of 18 weeks for birthing parents and 16 weeks for non-birthing parents.
• Access to Noom, a weight management program-based in psychology, that's tailored to your unique needs and goals.
• Access to fertility care support through Carrot, in addition to $4,000 reimbursement for related fertility expenses.
• Access to Wellhub, which connects employees to the best options for fitness, mindfulness, nutrition, and sleep in one subscription
• Access to BrightHorizons, which provides sponsored child care, back-up care, and elder care
• Up to $1,000 Professional Development Reimbursement a year.
• $200 per year donation matching to support your favorite causes.