QA / Automation Engineer Agentic AI
Advanced Tech PlacementRoseland, NJ
Automate your job search with Sonara.
Submit 10x as many applications with less effort than one manual application.1
Reclaim your time by letting our AI handle the grunt work of job searching.
We continuously scan millions of openings to find your top matches.
Overview
Remote
On-site
Job Description
We are looking for a QA / Automation Engineer
This role involves building and maintaining automated tests for an agentic, LLM-driven application, focusing on creating a new Python-based framework for evaluating agentic systems.
Responsibilities:
- Build and maintain automated tests for an agentic, LLM-driven application, extending beyond conventional UI test automation.
- Develop a new, Python-based framework for evaluating agentic systems, including LLM-as-judge style evaluators and state checks for conversational/agentic flows.
- Utilize Playwright for front-end automation where applicable, while focusing on the Python-based evaluation layer for AI-driven behavior.
- Collaborate closely with developers in a pod structure, as QA is distributed and collaborative rather than siloed.
Requirements:
- Comfort operating with a light, UAT-style formal QA process, where quality ownership is shared with developers.
- Ability to work in an undefined, evolving testing landscape as the framework is being built and expanded.
Required Skills:
- Proficiency in Python, as it is the primary language for the role's evaluation/automation framework.
- Experience or strong aptitude for testing AI/LLM-based systems, including evaluators, LLM-as-judge techniques, and state/behavior validation for agentic or conversational systems.
- Working knowledge of Playwright for the front-end automation layer, though it is not the core skill being hired for.
Preferred Skills:
- Industry domain knowledge in HCM.
Automate your job search with Sonara.
Submit 10x as many applications with less effort than one manual application.
FAQs About QA / Automation Engineer Agentic AI Jobs at Advanced Tech Placement
What is the work location for this position at Advanced Tech Placement?
This job at Advanced Tech Placement is located in Roseland, NJ, according to the details provided by the employer. Some roles may also include multiple work locations depending on the requirement.
What pay range can candidates expect for this role at Advanced Tech Placement?
Employer has not shared pay details for this role.
What employment applies to this position at Advanced Tech Placement?
The employer has not provided this information. This may be discussed during the hiring process.
What is the process to apply for this position at Advanced Tech Placement?
You can apply for this role at Advanced Tech Placement either through Sonara's automated application system, which helps you submit applications 10X faster with minimal effort, or by applying manually using the direct link on the job page.