Data Engineering Lead, DevOps & Data Platforms
Overview
You own the core foundation of the Data Intelligence Platform Enablement Framework, building governance-as-code, infrastructure-as-code, CI/CD automation, and observability. You lead a team of engineers and collaborate with data scientists and analysts to ship a reliable, secure platform. Your work directly enables faster, compliant data product delivery across the organization. This role offers technical leadership and the chance to shape platform standards from day one.
What You'll Do8
- 1Design and implement governance-as-code policies using Terraform and Open Policy Agent to enforce data access and quality standards.
- 2Build infrastructure-as-code modules for AWS resources (e.g., S3, Glue, EMR) to standardize environment provisioning.
- 3Architect CI/CD pipelines with GitHub Actions or Jenkins to automate deployment of data pipelines and platform components.
- 4Establish workspace configuration standards for Databricks and Snowflake to ensure consistent security and performance.
- 5Implement monitoring and alerting with CloudWatch and Grafana to track platform health and data freshness.
- 6Drive adoption of Docker and Kubernetes for containerized data workloads, ensuring scalability and resilience.
- 7Collaborate with data engineers to refactor legacy pipelines into modular, observable services.
- 8Mentor junior engineers on DevOps best practices and infrastructure as code.
Requirements7
- 15+ years as a Data Engineer or DevOps Engineer with a focus on data platforms.
- 23+ years leading infrastructure projects using Terraform and AWS (EC2, S3, Lambda).
- 3Hands-on experience with Python and SQL for scripting and data manipulation.
- 4Proven track record implementing CI/CD pipelines with Jenkins or GitHub Actions.
- 5Experience with Kubernetes and Docker for container orchestration.
- 6Strong background in monitoring tools like CloudWatch, Prometheus, or Grafana.
- 7Familiarity with data governance frameworks (e.g., Collibra, Alation) is a plus.
Salary Insight
$146 - $156k per year
Location
Required Skills
Similar open positions
Explore active roles that match your skills and interests.
Marathon Petroleum Company LP
VerifiedSenior Data Operations Engineer, Cloud Platforms
Own the reliability, scalability, security, and performance of MPC's enterprise data platforms, serving teams across Data Engineering, Platform Engineering, and SRE. Lead modernization and automation for cloud-native systems including Databricks, Azure, Cognite, and streaming technologies. Mentor engineers, set operational standards, and drive compliance with governance controls. This role shapes the future of data platform operations at a major energy company, with direct impact on AI-driven innovation.
US031 Marsh & McLennan Shared Services LLC
VerifiedEnterprise Data Engineering & Analytics Director
Lead and scale enterprise data platform and analytics capabilities. Build trusted governed scalable data foundations that power analytics reporting and AI. Foster high performing inclusive culture. Drive modern cloud data platforms creating measurable business value.
EXTEU-AC-CareerSite
VerifiedPrincipal Data Architect, Data Engineering & AI
You will own the enterprise data architecture and strategy, enabling scalable AI/ML, telemetry analytics, and business intelligence across a global organization. You will lead the design of cloud-native data platforms on AWS and Azure, from ingestion to insights, and set the blueprint for data storage, integration, and governance. Collaborating with Data Science and AI teams, you will industrialize ML pipelines and drive data as a strategic asset. This role combines deep technical leadership with executive influence, shaping the future of data-driven decision making.
Finoit Inc.
VerifiedSenior Software Engineer Data Infrastructure
Design and lead development of scalable data pipelines for AI/ML platforms. Own design and implementation of distributed systems using Python and cloud services. Drive improvements in data quality and visualization. Lead cross-functional teams to deliver high-performance solutions.
SEI Investments
VerifiedTechnical Lead, Data Governance & Cloud Platforms
Own the design and delivery of data governance solutions for the SEI Wealth Platform, including archive, purge, masking, and data sub-setting. You will lead a team of developers, partnering with architects, operations, and business stakeholders to build scalable, secure data pipelines using PLSQL, Java, Spring Boot, and Kubernetes. This role requires a blend of engineering depth and technical leadership, with a focus on cloud-native patterns and Snowflake, DBT, and Airflow. You will drive architectural decisions, ensure code quality, and implement monitoring and validation controls to guarantee data accuracy and traceability. The role offers the chance to influence data governance strategy in a leading financial technology firm.
TECHNEPTUNE CONSULTING INC
VerifiedData Solution Architect - Expertise in AWS and Data Platforms
Lead ownership of scalable data solutions at Technep Tuning Inc. in San Francisco. Design and deliver enterprise-grade data platforms leveraging advanced cloud technologies. This role focuses on architecting robust data ecosystems that drive business insights and operational efficiency.