Director, Measurement Science

POSITION OVERVIEW NBME is seeking to hire a Director of Measurement Science to work in the Office of Research Strategy (ORS). In collaboration with the ORS Leadership team, the Director will help develop and manage NBME's research portfolio. This role will report to the Vice President of ORS and is looking for a strategic thought leader with a demonstrated track record in leading a program of work around defining complex skills and behaviors, developing innovative, evidence-centered assessments, and articulating validity, fairness, and efficacy arguments and claims. This role will manage and mentor senior PhD-level staff and guide NBME research strategy. This role has been designated as primarily remote, meaning it requires little to no onsite attendance. While this role can be designated as remote, you are offered the flexibility to select if you prefer to work primarily remotely, hybrid, or onsite. We're open to considering candidates from the following states: PA, NJ, DE, MD, VA, FL, TX, GA, IA, IN, KS, KY, MI, NC, TN, VT. Candidates residing outside of these locations are encouraged to apply; however, eligibility will be subject to additional review and approval prior to an offer. At NBME, we continue to innovate and improve how we fulfill the evolving needs of the healthcare community. This commitment starts and ends with the people at NBME. By recruiting and empowering talented individuals from various disciplines and backgrounds, which includes professionals with diverse life experiences, abilities, and perspectives, NBME can take a well-informed, robust approach to advancing medical education and assessment for years to come. RESPONSIBILITIES Accountable for strategic planning activities and associated prioritization initiatives focused on defining and developing assessments of difficult-to-assess skills and behaviors aligned with NBME strategic priorities and informed by emerging trends in measurement and in medical education. Act as point person for research portfolio development and monitoring within functional area in collaboration with NBME functional leaders. Develop and represent relationships with the medical education and assessment research communities to identify opportunities for collaborative research and development. Strategize, plan, direct, and supervise research initiatives as principal investigator. Identify, hire, manage, and mentor scientific staff with skills necessary to execute NBME's portfolio of research initiatives. Identify training needs and professional development opportunities. Assist with budget development. Consult as an expert with NBME functional leads and external personnel in the development of new products and services. Provide consultation as needed to improve research and innovation development activities for individual projects. Deliverables Recommendations for research priorities informed by customer feedback, marketing results, competitive analysis, advances in measurement and medical education, and developments in technology and data analytics. Lead development of deliverables for cross-function strategic initiatives. Effective working relationships with internal and external stakeholders. Management, development, and retention of scientific staff. Creates an environment that ensures NBME values are respected in day-to-day activities. Publications, presentations, and participation in external professional activities. QUALIFICATIONS Skills and Abilities Strong background in educational assessment and measurement, and/or in psychological assessment, including a well-developed understanding of issues of reliability, validity, fairness, efficacy, and other psychometric concepts related to test and assessment quality. Experience in developing assessments for a variety of purposes and uses, including formative purposes. Experience developing performance-based assessments and learning tools, such as simulation-based assessments, and incorporating the latest technologies into the assessment design. Experience developing assessments of difficult-to-assess skills and behaviors (e.g., teamwork, clinical reasoning, and communication) Excellent oral and written communication skills, including the ability to effectively disseminate results of research to technical and non-technical audiences. Excellent interpersonal skills, including the ability to work closely and collaboratively with researchers and individuals from a variety of professional backgrounds. Demonstrated ability to effectively supervise the work of others and mentor scientific staff. Experience Minimum 7 years of experience in the field. Track record of effectively managing a team of measurement scientists. Education Doctorate in educational/psychological measurement, industrial/organizational psychology, cognitive psychology or related field About NBME NBME is a not-for-profit organization that specializes in the creation of assessments and learning tools for physicians and health professionals. Our mission is to advance assessment of these professionals to achieve optimal care for all, supporting the development of a highly effective, diverse and compassionate health care workforce. Founded in 1915, NBME develops and manages the United States Medical Licensing Examination® with the Federation of State Medical Boards. We offer a comprehensive portfolio of assessment products for every stage of the medical school journey and provide assessment services to various health profession organizations. We are dedicated to advancing innovative assessment approaches through research, collaboration and contributions to the medical education and assessment communities. In 2024, we expanded our assessment capabilities to include simulation through the acquisition of MedVR Education. Learn more on NBME's website. NBME's Community Collaborations and Contributions NBME believes that a key path to meeting our mission is collaborating with and supporting our communities. Our Community Collaborations and Contributions programs provide resources and recognition to medical educators and researchers advancing the field of assessment throughout their careers. In addition, we invest in fee assistance, scholarships and pathway programs to aid learners and help support increased representation in health care. Headquartered in Philadelphia, NBME also gives to local organizations that advance health equity and access and positively impact social determinants of health. Compensation we are offering for this position is at $197,526 - $246,000/year. Please note that the offered rate for this position typically aligns with the minimum to midrange of this grade, but it can vary based on the successful candidate's qualifications and experience, department budget, and an internal equity review. The NBME offers competitive salaries, excellent benefits, and a rewarding work environment. Excellent Benefits include: Healthcare, Dental, Prescription, and Vision plans; 401(k) w/match, Tuition Reimbursement Plan, Commuter Benefit: Public Transit or Parking options. Remote Friendly Workplace. NBME is an Equal Opportunity Employer. We will consider all qualified applicants for employment without regard to race; color; sex; gender identity; sexual orientation; religious practices and observances; national origin; pregnancy, childbirth, or related medical conditions; status as a protected veteran or spouse/family member of a protected veteran; or disability. Job Posted by ApplicantPro