Databricks Data Engineer, ETL & Lakehouse Architect
Overview
Design and own scalable data pipelines on Databricks and Apache Spark for a federal contractor in the DC/MD/VA area. You will modernize legacy ETL into a Delta Lake lakehouse, working cross-functionally with analytics and platform teams. This contract role stands out for its hybrid setup and direct impact on mission-critical data infrastructure.
What You'll Do8
- 1Design and deploy ETL/ELT pipelines on Databricks handling terabyte-scale datasets
- 2Build Delta Lake tables with ACID transactions and time travel for data reliability
- 3Optimize Spark jobs for performance and cost, tuning shuffle partitions and caching
- 4Automate pipeline orchestration with Airflow and Azure Data Factory
- 5Implement data quality checks and monitoring using Great Expectations and Databricks jobs
- 6Collaborate with data scientists to expose curated features for MLflow experiments
- 7Migrate on-premises Hadoop workloads to a Databricks lakehouse architecture
- 8Produce technical documentation and runbooks for pipeline operations
Requirements9
- 15+ years in data engineering with Python and SQL
- 23+ years building pipelines on Apache Spark
- 3Hands-on Databricks experience, including cluster configuration and job scheduling
- 4Strong Delta Lake and lakehouse design skills
- 5Experience with ETL/ELT tools like Airflow, dbt, or Informatica
- 6Proficiency with cloud platforms AWS or Azure, plus S3 or Azure Data Lake
- 7US citizenship or permanent residency required for federal work
- 8Bachelor's degree in computer science or equivalent practical experience
- 9Excellent communication and client-facing skills
Salary Insight
Salary not disclosed in listing
Location
Required Skills
Similar open positions
Explore active roles that match your skills and interests.
ePace Technologies, Inc
VerifiedDatabricks Data Engineer, ETL & Delta Lake
You will own enterprise-scale data pipelines on Databricks, driving the architecture and delivery of mission-critical data flows in cloud environments. You will join a platform team of engineers and data specialists, working with Delta Lake, Python, and SQL to process complex Sales Incentive Compensation (SIC) data. This role stands out for its focus on Delta Live Tables and high-volume, low-latency processing in a dynamic consulting environment.
Anblicks
VerifiedLead Databricks Engineer, Lakehouse & ETL Pipelines
You will own the Lakehouse platform architecture for Anblicks in Dallas, designing and implementing end-to-end data solutions on Databricks. You will define best practices and build scalable ETL/ELT pipelines that power Spark workloads. Working closely with stakeholders, you will shape technical strategy and deliver production-grade data systems. This role centers on driving performance, cost, and scalability across a growing data estate.
Healthcare Triangle Inc
VerifiedDatabricks Lead - Data Engineering
Own the design and delivery of scalable data engineering solutions on the Databricks platform, driving value across the full data lifecycle. Lead a team of engineers building PySpark, Spark SQL, and Delta Lake pipelines. Collaborate with data architects and stakeholders to shape the lakehouse architecture. This role offers direct ownership of technical strategy and a chance to set standards for data quality and performance.
Healthcare Triangle Inc
VerifiedDatabricks Engineer Lead, Data Engineering & Lakehouse
Lead the design and development of scalable data engineering solutions using Databricks in a long-term contract role with Healthcare Triangle Inc. You will own the full lifecycle of data pipelines and lakehouse architecture, working with Azure and AWS cloud platforms. Your team relies on you to set technical direction and mentor engineers. This role stands out for its focus on Unity Catalog and cost optimization in a mission-critical healthcare environment.
Donato Technologies Inc
VerifiedSenior Data Engineer, Databricks & ETL Pipelines
Own the design and delivery of scalable data pipelines on Databricks for a Houston-based enterprise. You join a data engineering team of 6 inside a larger analytics organization, collaborating with architects, analysts, and application teams. Build reliable ETL/ELT solutions that move millions of records daily. This contract role demands hands-on coding in Python and Spark from day one, with direct influence on platform architecture.
Rivago infotech inc
VerifiedData Architect (Databricks), Lakehouse Platform
You will lead enterprise-scale data platform modernization, migrating from AWS EMR, Apache NiFi, and legacy ETL frameworks to the Databricks Lakehouse Platform. You will architect and implement scalable Lakehouse solutions using Delta Lake, Unity Catalog, and Databricks Workflows. You will design end-to-end data pipelines for batch, streaming, and CDC, and govern data across the organization. This role centers on technical leadership and hands-on architecture, shaping the data strategy for a long-term project.