Staff Data Engineer - Data Engineering

The Role:

As a key technical leader and team architect working in a fast-paced environment, you will drive the design, development, and optimization of scalable data ingestion pipelines within the Arine platform. Leveraging expert-level proficiency in Python and AWS, you will architect solutions that handle diverse file types and large-scale healthcare datasets. You will have a direct impact on building reusable, configurable tools set for handling data needs for the entire company.

What You'll be Doing:

• Act as the team architect by leading system design reviews, offering recommendations, conducting comprehensive peer reviews, and demonstrating expert-level proficiency in Python and AWS services

• Architecting and implementing scalable data ingestion pipelines handling different file types into Arine platform

• Develop reusable components that can be integrated into data pipelines to enhance efficiency and minimize future implementation time

• Creating configuration-driven, containerized toolsets that can be easily used and maintained by diverse engineering profiles

• Work collaboratively with cross-functional teams to ensure their data requirements are met through ETL components

• Implementing incremental data ingestion strategies for large-scale healthcare datasets

• Building monitoring and alerting systems for data ingestion processes and pipeline health

• Applying software engineering best practices including test-driven development and modular design to data infrastructure

• Refactoring and rebuilding existing data ingestion processes to improve scalability and operational efficiency

• Working with containerization technologies (Docker, Kubernetes) to create portable and maintainable data solutions

• Identify and escalate inefficiencies within and across teams

• Provide technical guidance, mentorship to junior engineers, and promote best practices and coding standards

• Author and support high-quality technical documentation, assisting junior engineers in doing the same

Who You Are and What You Bring:

• 10+ years of professional experience in data engineering with focus on large-scale data ingestion and infrastructure

• Deep expertise in Python programming and modern data engineering tools

• Experience creating an automated production grade ETL process using Python and SQL

• Strong understanding of ETL/ELT frameworks and distributed data processing

• Experience with data processing, validation, cleaning and debugging data sets

• Experience with API integration for seamless data exchange between systems

• Proven experience handling and processing various file types and formats, including specialized healthcare standards such as HL7, 834, 837, and NCPDP

• Experience integrating and consolidating data from diverse source systems into a unified repository, including data from EHR and claim systems, as well as from file-based and API integrations

• Experience with processing large data sets (over 10GB)

• Experience with incremental data processing and change data capture (CDC) methodologies

• Strong experience designing scalable data architectures in AWS environment

• Deep understanding of software engineering principles including test-driven development, loose coupling, single responsibility, and modular design

• Experience with containerization technologies (Docker, Kubernetes) and building configuration-driven, maintainable systems

• Proven ability to build tools and systems that can be operated by diverse engineering profiles through configuration rather than code changes

• Passion for building new and improving existing data infrastructure with robust, maintainable, and operationally excellent data systems

• Familiarity with healthcare data and regulatory environments (HIPAA compliance) is a plus

• Strong collaboration and communication skills; comfortable working with diverse technical and non-technical stakeholders

• Excellent verbal and written communication skills with ability to explain technical infrastructure concepts to diverse audiences

Remote Work Requirements:

• An established private work area that ensures information privacy
• A stable high-speed internet connection for remote work
• This role is remote, but you will be required to come to on-site meetings multiple times per year. This may be in the interview process, onboarding, and team meetings

Perks:

Joining Arine offers you a dynamic role and the opportunity to contribute to the company's growth and shape its future. You'll have unparalleled learning and growth prospects, collaborating closely with experienced Clinicians, Engineers, Software Architects, Data Scientists, and Digital Health Entrepreneurs.

The posted range represents the expected base salary for this position and does not include any other potential components of the compensation package, benefits, and perks. Ultimately, the final pay decision will consider factors such as your experience, job level, location, and other relevant job-related criteria. The base salary range for this position is: $165,000-180,000/year.