Lead Data DevOps Engineer

Join our team as a Lead Data DevOps Engineer specializing in Google Cloud Platform, focusing on cloud-based data infrastructure and automation. You will take charge of building scalable data systems and streamlining deployment pipelines. Apply now to become a key contributor to our data platform excellence.

Responsibilities

• Create and maintain cloud data architectures using GCP services like DataFlow, GCS, BigQuery, Dataproc, and Cloud Composer
• Implement infrastructure provisioning and monitoring via IaC tools such as Terraform
• Partner with data engineers to develop automated, efficient data workflows using Python
• Configure CI/CD pipelines with Jenkins, GitLab CI, or GitHub Actions for seamless deployment
• Collaborate with multiple teams to enhance the reliability and performance of data platforms
• Support management of cloud data tools including Apache Spark, Apache Kafka, and Apache Airflow
• Diagnose and fix problems to maintain scalable and reliable cloud data environments

Requirements

• Minimum 5 years experience working in cloud environments, particularly with GCP services like BigQuery, Cloud Composer, and Dataproc
• Advanced Python skills combined with SQL expertise for data pipeline construction
• Knowledge of IaC solutions such as Terraform or AWS CloudFormation
• Experience in creating CI/CD workflows with Jenkins, GitHub Actions, or GitLab CI
• Background in Linux systems and shell scripting
• Understanding of networking principles such as TCP/IP, DNS, and NAT
• Proven skills in operating Apache Spark, Apache Airflow, or ELK Stack

Nice to have

• Hands-on familiarity with AWS or Azure services including ECS, S3, Data Lake, or Synapse
• Capability to use additional IaC tools like Ansible
• Experience with other data workflow automation technologies