Senior Data Engineer AVP - Cloud Data Pipelines & Warehouses
Overview
Lead the design and execution of enterprise-grade Java data pipelines and warehouses to ensure accurate accessible secure data. Build and maintain robust systems for ingestion transformation and storage across large volumes. Drive optimization of pipeline performance and collaborate with cross-functional teams. Shape data governance and compliance while mentoring technical staff. Seeking a leader who can influence decision making and foster a culture of excellence.
What You'll Do11
- 1Design and develop Java based data pipelines for high volume workloads using Spark Kafka and cloud native technologies
- 2Lead a Java development team providing mentorship code reviews and establishing development standards
- 3Implement error handling data validation and monitoring systems for pipeline reliability and quality
- 4Develop scalable APIs using Kubernetes for container orchestration and replica management
- 5Create data warehouses and lakes adhering to security measures and managing appropriate data volumes
- 6Collaborate with regulatory teams to define rules for FINRA SEC and other compliance systems
- 7Optimize pipeline performance through efficient partitioning caching and parallel processing techniques
- 8Establish logging alerting and observability solutions for proactive bottleneck identification
- 9Guide team members through complex technical challenges and drive project objectives
- 10Define and communicate data governance standards security protocols and compliance measures
- 11Mentor junior engineers and champion best practices in software development
Requirements5
- 15+ years building ETL pipelines with Spark and Airflow
- 23-5 seasons leading technical teams and delivering complex solutions
- 35+ years experience with Java and Kubernetes
- 42-5 years proficiency in data engineering and cloud infrastructure
- 51 year overseeing people and coaching for performance improvement
Salary Insight
$190 - $195k per year
Location
Required Skills
Similar open positions
Explore active roles that match your skills and interests.
Finoit Inc.
VerifiedSenior Software Engineer Data Infrastructure
Design and lead development of scalable data pipelines for AI/ML platforms. Own design and implementation of distributed systems using Python and cloud services. Drive improvements in data quality and visualization. Lead cross-functional teams to deliver high-performance solutions.
Wise Skulls Corp.
VerifiedSoftware Engineer (C++/Java) Data Platform
Own scalable high throughput data processing platforms using C++ and Java. Lead ship of core infrastructure components. Drive performance improvements across large scale systems. Shape architecture decisions for future growth. This role differs by focusing on low latency streaming solutions.
RightTalents
VerifiedSenior Data Engineer
Lead design and development of scalable data pipelines. Own end-to-end ETL processes. Drive performance improvements across large datasets. Shape architecture for cloud-native data platforms. Differentiate by delivering production-grade solutions.
SEI Investments
VerifiedTechnical Lead, Data Governance & Cloud Platforms
Own the design and delivery of data governance solutions for the SEI Wealth Platform, including archive, purge, masking, and data sub-setting. You will lead a team of developers, partnering with architects, operations, and business stakeholders to build scalable, secure data pipelines using PLSQL, Java, Spring Boot, and Kubernetes. This role requires a blend of engineering depth and technical leadership, with a focus on cloud-native patterns and Snowflake, DBT, and Airflow. You will drive architectural decisions, ensure code quality, and implement monitoring and validation controls to guarantee data accuracy and traceability. The role offers the chance to influence data governance strategy in a leading financial technology firm.
MMD Services, Inc
VerifiedSenior Data Engineer MMD Services Cloud Pipeline Specialist
Lead ownership of scalable data infrastructure supporting over one thousand locations. Build robust ETL systems using Spark and Airflow. Drive performance improvements across massive datasets. Shape architecture for future growth. This role impacts enterprise-wide data strategy.
4 BMCAP
VerifiedSenior Data Engineer Seattle AWS Spark Airflow Python
Design and lead scalable data pipelines for enterprise analytics. Own development of ETL processes using Python and Spark. Scale infrastructure on AWS. Drive improvements in data quality and performance. This role differs by focusing on cloud-native architecture and cross-functional leadership.