Lead Databricks Engineer, Lakehouse & ETL Pipelines
Overview
You will own the Lakehouse platform architecture for Anblicks in Dallas, designing and implementing end-to-end data solutions on Databricks. You will define best practices and build scalable ETL/ELT pipelines that power Spark workloads. Working closely with stakeholders, you will shape technical strategy and deliver production-grade data systems. This role centers on driving performance, cost, and scalability across a growing data estate.
What You'll Do8
- 1Design end-to-end data architecture on the Databricks Lakehouse platform
- 2Build scalable ETL/ELT pipelines using Apache Spark (PySpark/Scala) and Databricks workflows
- 3Implement Delta Lake for data reliability, performance, and governance
- 4Define and enforce data engineering best practices across the team
- 5Collaborate with stakeholders to translate business needs into technical solutions
- 6Optimize performance, cost, and scalability of data workloads in Databricks
- 7Drive the adoption of Unity Catalog for data governance and access control
- 8Mentor engineers on Spark and Databricks design patterns
Requirements8
- 15+ years in data engineering with a focus on Apache Spark
- 23+ years hands-on with Databricks Lakehouse platform
- 3Expertise in PySpark and Scala for data processing
- 4Deep understanding of Delta Lake and its performance tuning
- 5Experience designing and operating ETL/ELT pipelines at scale
- 6Proven ability to optimize data workloads for cost and performance
- 7Strong knowledge of data modeling, partitioning, and query optimization
- 8Bachelor's degree in Computer Science, Engineering, or related field
Salary Insight
Salary not disclosed in listing
Location
Required Skills
Similar open positions
Explore active roles that match your skills and interests.
Healthcare Triangle Inc
VerifiedDatabricks Lead - Data Engineering
Own the design and delivery of scalable data engineering solutions on the Databricks platform, driving value across the full data lifecycle. Lead a team of engineers building PySpark, Spark SQL, and Delta Lake pipelines. Collaborate with data architects and stakeholders to shape the lakehouse architecture. This role offers direct ownership of technical strategy and a chance to set standards for data quality and performance.
Headway Tek Inc
VerifiedDatabricks Architect, Enterprise Lakehouse Platform
You will own the Databricks platform architecture end-to-end, designing and implementing next-generation data platforms on the Lakehouse paradigm. You will drive scalability, governance, performance, and cost optimization across enterprise data ecosystems, partnering with data engineering, data science, and business teams. This role goes beyond pipeline development, offering leadership in platform strategy and adoption at scale. You will define the roadmap for Unity Catalog, Delta Lake, and Spark workloads in a hybrid environment.
Healthcare Triangle Inc
VerifiedDatabricks Engineer Lead, Data Engineering & Lakehouse
Lead the design and development of scalable data engineering solutions using Databricks in a long-term contract role with Healthcare Triangle Inc. You will own the full lifecycle of data pipelines and lakehouse architecture, working with Azure and AWS cloud platforms. Your team relies on you to set technical direction and mentor engineers. This role stands out for its focus on Unity Catalog and cost optimization in a mission-critical healthcare environment.
Protos IT
VerifiedDatabricks Data Engineer, ETL & Lakehouse Architect
Design and own scalable data pipelines on Databricks and Apache Spark for a federal contractor in the DC/MD/VA area. You will modernize legacy ETL into a Delta Lake lakehouse, working cross-functionally with analytics and platform teams. This contract role stands out for its hybrid setup and direct impact on mission-critical data infrastructure.
Rivago infotech inc
VerifiedData Architect (Databricks), Lakehouse Platform
You will lead enterprise-scale data platform modernization, migrating from AWS EMR, Apache NiFi, and legacy ETL frameworks to the Databricks Lakehouse Platform. You will architect and implement scalable Lakehouse solutions using Delta Lake, Unity Catalog, and Databricks Workflows. You will design end-to-end data pipelines for batch, streaming, and CDC, and govern data across the organization. This role centers on technical leadership and hands-on architecture, shaping the data strategy for a long-term project.
Donato Technologies Inc
VerifiedSenior Data Engineer, Databricks & ETL Pipelines
Own the design and delivery of scalable data pipelines on Databricks for a Houston-based enterprise. You join a data engineering team of 6 inside a larger analytics organization, collaborating with architects, analysts, and application teams. Build reliable ETL/ELT solutions that move millions of records daily. This contract role demands hands-on coding in Python and Spark from day one, with direct influence on platform architecture.