Data Tech Lead, Java & Cloud Data Platform
Overview
Databricks and streaming data expertise to design, develop, and support scalable cloud data platforms at scale. Lead data pipelines using PySpark and Java streaming technologies and own cloud migration initiatives. Collaborate with cross functional teams, build robust CI/CD with GitLab CI/CD and drive architecture decisions. This role offers a chance to influence data platform strategy and accelerate data driven outcomes.
What You'll Do10
- 1Build scalable Databricks ETL/ELT pipelines using Lakeflow and LakeBase to enable reliable data processing
- 2Design and implement streaming data solutions in Java and PySpark to support real time analytics
- 3Lead data platform architecture discussions and drive migration of workloads to cloud based infrastructure
- 4Ship robust data models and governance practices ensuring data quality, lineage, and security
- 5Own CI/CD pipelines with GitLab CI/CD to automate testing, deployment, and monitoring
- 6Scale data pipelines to handle increasing data volumes and onboarding new data sources
- 7Collaborate with data scientists, engineers, and product teams to translate requirements into technical solutions
- 8Mentor junior engineers and establish best practices for monitoring, alerting, and incident response
- 9Debug performance bottlenecks and optimize data processing workflows for throughput and latency
- 10Drive documentation and maintainable code standards across the data platform team
Requirements10
- 15+ years building data platforms with Databricks and PySpark
- 2Strong proficiency in Java streaming technologies and Python
- 3Experience with GitLab CI/CD pipelines and cloud migrations (AWS or Azure)
- 4Proven track record delivering scalable ETL/ELT pipelines in cloud environments
- 5Familiarity with data governance, security, and data quality practices
- 6Excellent collaboration and communication skills in cross functional teams
- 7Bachelor's degree in Computer Science or related field or equivalent experience
- 8Onsite role in Atlanta, GA or Berkley Hills, NJ and ability to work contract based
- 9Prior experience leading data platform initiatives and mentoring junior engineers
- 10Knowledge of Lakeflow and LakeBase is a plus
Salary Insight
$60k per year
Location
Required Skills
Similar open positions
Explore active roles that match your skills and interests.
Parkar Consulting Group, LLC
VerifiedSenior Data Engineer, Databricks & Cloud Migration
Own data engineering initiatives at scale in a client environment, building robust data pipelines and migrating on-prem systems to cloud platforms. You will work with a cross functional team of data engineers, cloud engineers, and analytics partners in New York City onsite. This role emphasizes hands on Databricks development, Python, and SQL while collaborating to design scalable data architectures. What makes this role unique is leading migration projects from legacy systems to modern cloud data platforms while delivering measurable data capabilities for business users.
Healthcare Triangle Inc
VerifiedDatabricks Lead - Data Engineering
Own the design and delivery of scalable data engineering solutions on the Databricks platform, driving value across the full data lifecycle. Lead a team of engineers building PySpark, Spark SQL, and Delta Lake pipelines. Collaborate with data architects and stakeholders to shape the lakehouse architecture. This role offers direct ownership of technical strategy and a chance to set standards for data quality and performance.
Anblicks
VerifiedLead Databricks Engineer, Lakehouse & ETL Pipelines
You will own the Lakehouse platform architecture for Anblicks in Dallas, designing and implementing end-to-end data solutions on Databricks. You will define best practices and build scalable ETL/ELT pipelines that power Spark workloads. Working closely with stakeholders, you will shape technical strategy and deliver production-grade data systems. This role centers on driving performance, cost, and scalability across a growing data estate.
Rivago infotech inc
VerifiedData Architect (Databricks), Lakehouse Platform
You will lead enterprise-scale data platform modernization, migrating from AWS EMR, Apache NiFi, and legacy ETL frameworks to the Databricks Lakehouse Platform. You will architect and implement scalable Lakehouse solutions using Delta Lake, Unity Catalog, and Databricks Workflows. You will design end-to-end data pipelines for batch, streaming, and CDC, and govern data across the organization. This role centers on technical leadership and hands-on architecture, shaping the data strategy for a long-term project.
Techouts, Inc
VerifiedDatabricks Senior Developer, Data Engineering & Migration
Own the end-to-end data engineering stack at scale, building and migrating data systems to the cloud. You will drive production-ready Databricks pipelines and collaborate with cloud and data teams to modernize on-prem data assets. This role emphasizes hands-on development, migration know-how, and translating legacy SQL to Spark SQL. You will contribute to a strong data platform that enables analytics and insights across the organization.
Healthcare Triangle Inc
VerifiedDatabricks Engineer Lead, Data Engineering & Lakehouse
Lead the design and development of scalable data engineering solutions using Databricks in a long-term contract role with Healthcare Triangle Inc. You will own the full lifecycle of data pipelines and lakehouse architecture, working with Azure and AWS cloud platforms. Your team relies on you to set technical direction and mentor engineers. This role stands out for its focus on Unity Catalog and cost optimization in a mission-critical healthcare environment.