Lead Data Engineer GCP & OCI
Overview
Lead a modern data platform team owning enterprise-scale data pipelines, warehouses, and analytics environments built on GCP and OCI. The stack includes BigQuery, Dataflow, Databricks, and Oracle Autonomous Database, serving 100+ business-critical reports. You will set the technical direction, mentor engineers, and deliver a migration of legacy pipelines to GCP within the first quarter. This role blends hands-on architecture with team leadership, and you will directly influence the platform's scalability and reliability.
What You'll Do8
- 1Design and implement end-to-end data pipelines on GCP using Dataflow, Pub/Sub, and Cloud Storage, migrating legacy workflows to BigQuery.
- 2Architect and manage data warehousing solutions on BigQuery and Oracle Autonomous Database, optimizing for cost and query performance.
- 3Lead the integration of OCI services with GCP, ensuring cross-cloud data flow and unified governance.
- 4Drive the adoption of Databricks for advanced analytics and machine learning pipelines, collaborating with data scientists.
- 5Mentor a team of 5 data engineers, conducting code reviews and setting best practices for CI/CD and infrastructure as code using Terraform.
- 6Monitor and tune Airflow DAGs to ensure reliable scheduling and alerting for data quality and SLA compliance.
- 7Build and maintain data models using dbt, ensuring consistency across reporting layers for finance and operations.
- 8Own the incident response for data platform issues, performing root cause analysis and implementing preventive measures.
Requirements8
- 17+ years in data engineering with 4+ years leading technical teams in cloud environments.
- 2Expert-level Python skills for building data processing frameworks and automation scripts.
- 3Deep hands-on experience with GCP services: BigQuery, Dataflow, Dataproc, and Cloud Composer.
- 4Proven experience with OCI core services: OCI Data Integration, Autonomous Data Warehouse, and Object Storage.
- 5Strong knowledge of SQL and data modeling, including star schemas and slowly changing dimensions.
- 63+ years using Terraform for infrastructure deployment and management across multi-cloud environments.
- 7Familiarity with Databricks platform and Spark for large-scale data processing.
- 8Excellent communication skills to present technical plans to stakeholders and executives.
Salary Insight
$70 - $80k per year
Location
Required Skills
Similar open positions
Explore active roles that match your skills and interests.
Apex Systems
VerifiedData Engineering Lead, DevOps & Data Platforms
You own the core foundation of the Data Intelligence Platform Enablement Framework, building governance-as-code, infrastructure-as-code, CI/CD automation, and observability. You lead a team of engineers and collaborate with data scientists and analysts to ship a reliable, secure platform. Your work directly enables faster, compliant data product delivery across the organization. This role offers technical leadership and the chance to shape platform standards from day one.
Next Gen Software Solutions
VerifiedGCP DevOps Engineer | Berkeley Height NJ SFO CA
The ideal candidate will own the design and implementation of scalable cloud infrastructure while leading cross-functional teams. This role focuses on high impact outcomes within a dynamic environment. You will shape the technical direction of our cloud initiatives.
Customer Engineer Data Analytics Role at Google Cloud
Design and deliver high impact data analytics solutions for startups leveraging Google Cloud platforms. Lead technical engagements to drive adoption and digital transformation. Shape product direction through customer insights and collaborate across cross functional teams. Differentiate Google Cloud offerings for innovative SaaS providers.
StatusNeo Inc.
VerifiedLooker + Google BigQuery Architect - Enterprise Data Engineer
Lead analytics solutioning architecture design and performance optimization across enterprise-scale data ecosystem. Own BI modernization journey. Standalone statement of ownership and scale. Differentiate by anchoring BI transformation.
Rivago infotech inc
VerifiedLead Data Engineer - ETL & Python
Lead the design and delivery of ETL pipelines that move petabytes of data across Oracle and Python stacks. You own the integration architecture and mentor a team of developers to ship modernized data flows. This role sits in the data platform group, partnering with DevOps to automate deployments. You will replace legacy batch jobs with CI/CD-driven processes and set the technical direction for data quality. 8+ years in ETL and 3+ years leading developers makes you the linchpin for this modernization effort.
Finoit Inc.
VerifiedSenior Software Engineer Data Infrastructure
Design and lead development of scalable data pipelines for AI/ML platforms. Own design and implementation of distributed systems using Python and cloud services. Drive improvements in data quality and visualization. Lead cross-functional teams to deliver high-performance solutions.