Senior Ai/Qa Engineer (519)
Johnson Service GroupBoston, MA
Automate your job search with Sonara.
Submit 10x as many applications with less effort than one manual application.1
Reclaim your time by letting our AI handle the grunt work of job searching.
We continuously scan millions of openings to find your top matches.
Job Description
Our client is seeking a Senior AI Engineer with strong AI quality experience to build, evaluate, and improve enterprise AI solutions, with a focus on GenAI, RAG, and agent-based workflows in an Azure environment. (remote US only)Must Have skills
- Experience building and supporting GenAI and LLM-based solutions
- Strong skills in Python
- Experience with Azure AI services, including Azure AI Foundry or related tools*
- Experience with RAG systems and retrieval-based AI workflows*
- Familiarity with LangChain, LangGraph, or agent-based orchestration
- Experience validating LLM outputs for quality, accuracy, and consistency*
- Exposure to evaluation frameworks, testing, or quality processes for AI systems
- Experience with Azure AI Search
- Exposure to AI test automation or monitoring workflows*
- Familiarity with Responsible AI, governance, or compliance practices*
- Experience with knowledge graphs, GraphRAG, Pinecone, FAISS, OpenSearch, or similar tools
- Experience in enterprise or regulated environments
- Experience supporting human-in-the-loop or audit-focused AI processes
- Build and improve enterprise AI solutions across GenAI, RAG, and agent-based systems
- Support development of LLM-powered applications in an Azure-based environment*
- Design and enhance retrieval pipelines and grounded response workflows*
- Contribute to evaluation and validation of AI outputs for quality, safety, and consistency*
- Help develop structured approaches for testing and monitoring AI solutions*
- Work with cross-functional teams to deliver reliable, governed, and production-ready AI systems*
- Support documentation, quality standards, and Responsible AI practices
Automate your job search with Sonara.
Submit 10x as many applications with less effort than one manual application.
FAQs About Senior Ai/Qa Engineer (519) Jobs at Johnson Service Group
What is the work location for this position at Johnson Service Group?
This job at Johnson Service Group is located in Boston, MA, according to the details provided by the employer. Some roles may also include multiple work locations depending on the requirement.
What pay range can candidates expect for this role at Johnson Service Group?
Employer has not shared pay details for this role.
What employment applies to this position at Johnson Service Group?
The employer has not provided this information. This may be discussed during the hiring process.
What is the process to apply for this position at Johnson Service Group?
You can apply for this role at Johnson Service Group either through Sonara's automated application system, which helps you submit applications 10X faster with minimal effort, or by applying manually using the direct link on the job page.