Data Devops Engineer

Job Title: Data DevOps Engineer Location: [City, State] Job Type: Full-time Department: Data/Engineering Reports To: Data Engineering Manager Job Overview: We are seeking a highly skilled Data DevOps Engineer to join our growing data engineering team. The ideal candidate will play a critical role in maintaining and enhancing our data infrastructure, ensuring seamless data pipeline performance, automation, and deployment. You will work closely with data engineers, data scientists, and software engineers to create reliable and efficient workflows that support our data-driven decision-making processes. Key Responsibilities: Design, implement, and maintain data infrastructure: Build and manage data pipelines, databases, and other infrastructure to ensure the scalability and reliability of data systems. Automation & Orchestration: Automate manual processes using DevOps tools and best practices (e.g., CI/CD pipelines, infrastructure as code). Improve and maintain orchestration frameworks for data workflows. Monitoring & Troubleshooting: Monitor data infrastructure performance and quickly resolve issues related to data integrity, latency, or other operational challenges. Collaboration: Work closely with Data Engineers and Data Scientists to design, build, and deploy data solutions. Ensure smooth collaboration between development and operational teams. Cloud & On-Prem Infrastructure Management: Manage and optimize cloud platforms (AWS, GCP, Azure) and on-premise infrastructure for the storage, processing, and retrieval of large datasets. Security & Compliance: Ensure that data is secure and compliant with relevant regulatory requirements (GDPR, HIPAA, etc.). Implement data access controls, encryption, and backup strategies. Optimize Data Pipelines: Improve performance and reliability of data pipelines, ensuring data accuracy, timeliness, and efficient resource usage. Documentation: Create and maintain thorough documentation for data systems, processes, and workflows. Incident Management: Respond to incidents, perform root cause analysis, and develop strategies to prevent similar issues in the future. Required Qualifications: Education: Bachelor’s degree in Computer Science, Information Systems, Engineering, or related field. Experience: 3+ years of experience in DevOps, Data Engineering, or related fields. Hands-on experience with cloud services (AWS, Google Cloud, or Azure) for data infrastructure management. Strong experience with containerization (Docker, Kubernetes) and orchestration tools. Proficient in infrastructure-as-code tools (e.g., Terraform, CloudFormation, Ansible). Familiarity with CI/CD pipelines, version control (Git), and automation tools. Experience with data systems (Hadoop, Spark, Kafka, etc.) and data warehousing platforms (Redshift, BigQuery, Snowflake, etc.). Solid understanding of monitoring and logging tools (Prometheus, Grafana, ELK Stack). Proficient in scripting languages (Python, Bash, etc.). Preferred Qualifications: Experience with machine learning infrastructure and tools such as Kubeflow, MLflow, or TensorFlow. Familiarity with data governance and regulatory frameworks (GDPR, HIPAA). Knowledge of data integration tools (Apache NiFi, Talend, etc.). Experience with SQL/NoSQL databases and data modeling. Familiarity with Agile development methodologies. Skills and Competencies: Problem-Solving: Ability to identify and resolve complex data infrastructure issues efficiently. Collaboration: Strong communication and interpersonal skills, with a proven ability to work in cross-functional teams. Attention to Detail: Ability to monitor, debug, and optimize data flows and infrastructure for maximum efficiency and reliability. Adaptability: Willingness to learn new technologies and adapt to an evolving tech stack. Project Management: Experience managing and delivering on multiple priorities within deadlines. What We Offer: Competitive salary and benefits package. Opportunities for growth and development within a dynamic team. A collaborative and inclusive work environment. Flexible work arrangements (remote or hybrid options available). Exciting projects working with cutting-edge technology in data and cloud infrastructure.