Senior AI/ML Engineer
Overview
Senior AI/ML Engineer builds and operates the serving, deployment, and LLMOps infrastructure that carries models, prompts, and retrieval pipelines from prototype to governed production at Bain's Private Equity Group Innovation team. You own the model and prompt lifecycle end-to-end, from packaging and registry to promotion and rollback, and build evaluation harnesses, inference services, and observability that keep production ML measurable and reliable. Partnering with Data Scientists, Data Engineers, and the Agent/AI squad, you set the standard for production ML systems and mentor mid-level engineers. This hands-on role demands deploying, measuring, and operating models and pipelines at scale in a hybrid work environment.
What You'll Do10
- 1Build and operate production inference and serving systems for models, embeddings, and re-rankers, tuning request batching, concurrency, and throughput against latency and cost SLAs.
- 2Own the model and prompt lifecycle in MLflow, implementing packaging, registry governance, promotion workflows, staged rollout behind feature flags, and clean rollback.
- 3Develop and maintain LLMOps tooling including prompt and instruction versioning, model-gateway configuration with Portkey, inference orchestration, and response caching with cost controls.
- 4Construct and run production RAG and retrieval pipelines end-to-end: structure-aware chunking, contextual embedding, hybrid vector plus keyword retrieval, cross-encoder re-ranking, and context assembly.
- 5Design and maintain model and retrieval evaluation frameworks using golden datasets, metric definitions, LLM-as-judge with calibration, regression gates in CI, and production drift monitoring.
- 6Instrument production ML systems with structured logs, OpenTelemetry spans, and Prometheus metrics to track token usage, latency percentiles, retrieval hit rates, drift, and hallucinations, with dashboards and alerting.
- 7Collaborate with the Agent/AI squad to serve model and retrieval outputs as structured tool responses for the Agent Gateway, and partner with Data Engineers on feature and embedding pipelines.
- 8Drive production ML incident response to resolution, treating deployment, monitoring, and maintenance as core delivery responsibilities.
- 9Set and enforce engineering standards for ML and serving code, contribute to repository conventions, and mentor mid-level engineers through thorough code reviews.
- 10Use AI coding assistants to accelerate pipeline scaffolding and evaluation-harness development, while reviewing all generated code against production standards before committing.
Requirements10
- 16+ years building and operating production ML systems, including model deployment, serving, and post-deployment monitoring.
- 2Bachelor's degree in Computer Science, Engineering, Machine Learning, Data Science, Statistics, or related field (or equivalent practical experience).
- 3Demonstrated experience owning the model and prompt lifecycle end-to-end (packaging, registry, promotion, rollout, monitoring, rollback).
- 4Demonstrated experience building and operating production RAG or retrieval systems end-to-end, from embedding and retrieval through re-ranking and evaluation.
- 5Strong Python for production ML, with code written to production standards (testing, linting, typing).
- 6Experience in a modern cloud ML platform environment (Databricks or AWS), including managed training/serving and governed model access.
- 7Demonstrated ability to mentor other engineers and raise engineering standards through code review and repository conventions.
- 8Hands-on experience with MLflow, Portkey, LangChain or LlamaIndex, and Docker/Kubernetes for containerizing training/inference workloads.
- 9Experience with pgvector or dedicated vector databases, embedding pipeline design, and index tuning at scale.
- 10Skill in designing model evaluation and monitoring with golden datasets, metric definition, LLM-as-judge, regression gates, and drift detection.
Salary Insight
$141 - $169k per year
Location
Required Skills
Similar open positions
Explore active roles that match your skills and interests.
Bain & Co.
VerifiedSenior AI/ML Engineer, LLMOps & Production RAG
You will build and operate the deployment, serving, and LLMOps infrastructure for Bain & Co.'s private equity data products, taking models from prototype to governed production. You own the model and prompt lifecycle end-to-end, including packaging, registry, promotion, staged rollout, and rollback. You partner with Data Scientists on productionizing models and with Data Engineers on feature and embedding pipelines to serve ML and retrieval outputs into agent workflows. This hands-on role sets the standard for production ML systems, with a focus on measurable reliability and operational excellence. You will mentor mid-level engineers and drive incident response, making MLflow, Kubernetes, and AWS your daily toolkit.
Bain & Co.
VerifiedSenior AI/ML Engineer, LLMOps & RAG Systems
You build production inference, serving, and LLMOps infrastructure for Python-based ML systems, taking models from prototype to governed deployment. Bain's Private Equity Group Innovation team creates proprietary data and software products, and your work supports over 1,000 professionals across the investment lifecycle. You collaborate with Data Scientists, Data Engineers, and the Agent/AI squad to deliver reliable, observable ML systems. This hands-on role sets engineering standards and mentors mid-level engineers, with daily use of MLflow, Kubernetes, and Databricks.
Bain & Co.
VerifiedSenior AI/ML Engineering Manager, LLMOps & RAG
Lead the Data and ML Engineering function for Bain's Diligence Platform, owning teams that build Python-based feature and embedding pipelines, MLflow model serving, and LLMOps infrastructure. Manage a team of data and ML engineers while setting architectural direction for production RAG and retrieval systems. Partner with Data Science and Product leadership to translate strategy into measurable engineering work. This player-coach role offers the chance to shape the platform's technical standards and mentor senior engineers.
Bain & Co.
VerifiedSenior AI/ML Engineering Manager Bain & Co Austin Texas
Lead Data and ML Engineering for Diligence Platform. Own people delivery and technical direction. Drive architecture and operational excellence. Shape platform strategy with cross-functional partners.
Bain & Co.
VerifiedAI/ML Engineer, RAG & Production Pipelines | Bain
You will own the data, feature, and retrieval pipelines that power production RAG and ML systems for Bain's Private Equity Group. Working with Senior ML Engineers and the Engineering Manager, you will implement and operate ingestion, embedding, and retrieval components, shipping reliable, observable code from day one. You will partner with Data Engineers and Data Scientists on larger pipeline and RAG workstreams, building habits and judgment for Senior ML Engineer growth. This role offers hands-on ownership growth and exposure to cutting-edge AI solutions in private equity.
Bain & Co.
VerifiedAI/ML Engineer at Bain & Co.
AI/ML Engineer leading data pipeline development for private equity platform. Own production RAG and ML systems while collaborating with cross-functional teams. Drive technical excellence and innovation in data-driven decision making.