Senior Data Platform Engineer - DataOps (Montreal, Ottawa)

About GHGSAT 
GHGSat offers greenhouse gas detection, measurement, and monitoring services to industrial and government customers around the world. We use our own satellites and aircraft sensors, combined with third-party data, to help industrial emitters better understand, control, and reduce their emissions. GHGSat’s capability is unique: The company provides high-resolution, local measurements of atmospheric methane and carbon dioxide concentrations from space, enabling us to detect greenhouse gas emitters and visualize and quantify their emissions. 

 

Job Description: 

As a Senior Data Platform Engineer, you will be responsible for designing, building, and maintaining our data infrastructure with a focus on automation, efficiency, and scalability. You will work closely with cross-functional teams to develop CI/CD pipelines, manage cloud data services, and ensure high performance of data products. Your expertise in DevOps, cloud platforms, and data technologies will be crucial to our success. 

 

Key Responsibilities: 

• Data Infrastructure Management: Provision and manage data infrastructure in cloud environments such as AWS, including services like Redshift, Glue, Sagemaker, AWS DMS, AWS RDS (PostgreSQL), AWS EKS, and AWS ECR. 

• Infrastructure as Code (IaC): Develop and maintain IaC using Terraform to automate the provisioning and management of cloud resources. 

• CI/CD Pipelines: Develop CI/CD pipelines and automation tools using GitLab to ensure quick and reliable data deployment across multiple environments. 

• Observability and Telemetry: Implement observability and telemetry solutions to monitor and manage the health and performance of data infrastructure. 

• ETL Processes: Design and implement ETL processes, leveraging tools like Airflow, DBT, and various data lake file formats such as Delta, Hudi, or Iceberg. 

• Container Orchestration: Utilize container-based orchestration services such as Kubernetes, ECS, and Fargate for managing and scaling containerized applications. 

• Documentation and Support: Maintain comprehensive documentation of system configurations and procedures, including run books for live support. 

• Performance Improvement: Proactively troubleshoot and optimize data products to enhance performance and integrate best practices throughout the data development lifecycle. 

• Collaboration: Work closely with data scientists, data analysts, and other engineering teams to understand data requirements and ensure the availability and reliability of data services. 

Required Qualifications: 

• Experience: 5+ years of hands-on experience with DevOps or DataOps in a similar role. 

• Proficiency in Terraform, GitLab, SQL, and Python. 

• Experience with Airflow, Kubernetes, Docker, DBT, and Linux. 

• Strong knowledge of AWS services such as DMS, RDS (PostgreSQL), EKS, and ECR. 

• Familiarity with data lake file formats like Delta, Hudi, or Iceberg. 

• Expertise in container-based orchestration services like Kubernetes, ECS, and Fargate. 

• Strong problem-solving skills with the ability to troubleshoot and resolve complex issues. 

• Experience designing and building data models using data warehousing concepts to enable access to data across multiple domains. 

• Communication: Excellent communication skills with the ability to collaborate effectively with cross-functional teams. 

• Education: Bachelor's degree in computer science, Engineering, or a related field. Advanced degree or relevant certifications are a plus. 

Preferred Qualifications: 

• Experience with additional cloud data platforms or services. 

• Knowledge of other programming languages or data tools. 

• Relevant certifications in cloud platforms (AWS, Azure, GCP) or terraform.