MLOps interview questions

DevOps manages the lifecycle of code: build, test, release, run. MLOps inherits all of that and adds two more moving parts, data and models. A model's behavior is determined by its training data, parameters and code together, so you have to version and test all of them. It also degrades without any code change, because the data distribution shifts, so monitoring has to cover prediction quality and input data, not only uptime and latency. A good answer mentions reproducibility, drift, and training-serving consistency.

It is a mismatch between how features are produced in training and how they are produced at prediction time. Causes include separate code paths for the two, different data sources, time-travel leakage in the training set, and library version differences. Prevention: share one feature implementation or generate both from one definition, package preprocessing with the model as one artifact, build training sets with point-in-time correctness, and log the served feature vectors so you can compare their distribution to training data.

Data drift (covariate shift) means the distribution of the inputs changed, for example a new customer segment. Concept drift means the relationship between inputs and the target changed, so the same inputs now deserve a different prediction, for example fraudsters changing tactics. Data drift can be detected without labels by comparing distributions. Concept drift usually requires labels or good proxies, because inputs may look the same. Neither always means the model must be retrained; the answer depends on whether quality metrics actually moved.

Code in Git. Data and large files with a tool that keeps pointers in Git and content in remote storage, such as DVC, or with immutable, timestamped partitions plus recorded hashes. Models in a registry with a version, lineage back to the run that produced them, and a stage or alias. Configuration and hyperparameters alongside the run. Environment as a lockfile and a container image. The test is whether you can reproduce any production model from the records.

It gives you a searchable history of what was tried and a way to link a deployed model to the run that made it. Each run should record parameters, metrics (including per-segment metrics where relevant), the Git commit, the data version or hash, the environment, and the output artifacts. MLflow Tracking is a common open-source choice. Without the data version and commit, a tracker is a diary and not a reproducibility tool.

A registry is the source of truth for which model versions exist, where each came from, and which one serves production. It enables promotion with an audit trail, rollback by pointing back to a previous version, and a clean interface between training and serving: the service asks for "the production version of model X" and does not care which run produced it. In MLflow, for example, versions are numbered and aliases are movable pointers to specific versions.

A feature store is worth it when several models share features, when online and offline computation must stay consistent, or when point-in-time joins for training are painful to build by hand. It is not worth it for one or two models with simple features; there it adds a system to run and a layer to debug. Strong candidates say what problem they would be solving before naming a product.

Wrap the steps (data validation, preprocessing, training, evaluation, registration) in a pipeline that runs without manual steps. Triggers can be a schedule, arrival of enough new data, or a monitoring alert for degradation or drift. Retraining alone is not enough: the pipeline must validate the new model against the current production model on the same held-out data and across segments, and only then register it as a candidate. Promotion should be gated by those checks, with a human approval step where the stakes justify it.

Ask whether the prediction is needed at request time and how fresh it must be. If scores can be precomputed on a schedule and looked up, batch is simpler, cheaper and easier to validate and roll back. Online serving is justified when inputs only exist at request time or freshness matters. The cost of online is on-call, latency budgets and autoscaling. Hybrid designs precompute heavy features in batch and run a light model online. See model deployment strategies for a longer comparison.

Shadow: the new model sees copies of live requests but its outputs are only logged, so there is no user impact; it cannot measure behavioral effects. Canary: a small slice of real traffic gets the new model, with automatic rollback if metrics degrade. Blue/green: two environments, with an atomic switch and instant rollback. A/B: randomized, persistent groups compared on a business metric to decide which model is better. The key distinction is that the first three manage release risk and A/B answers a causal question.

Measure before guessing. Break latency down by stage: network, deserialization, feature lookup, preprocessing, model inference, post-processing. Compare payload sizes and batch sizes with staging. Check cold starts and model load time, CPU throttling and memory limits, whether the production hardware matches the test hardware, and whether dependencies such as the feature store add tail latency. Look at percentiles, not averages. Typical fixes are request batching, a smaller or quantized model, caching, and moving feature computation offline.

The answer should be quick and boring. Deploy by immutable version identifier, keep the previous version available, and make rollback a configuration or traffic change rather than a rebuild. For batch, rerun with the previous version. Define rollback thresholds before release. Also consider downstream effects: predictions already written to a table or acted upon may need to be corrected, which is why logging which version produced each prediction matters.

Four layers. Service health: latency, error rate, throughput, resource use. Data quality: schema violations, null rates, ranges, freshness. Data and prediction drift: distributions of inputs and outputs compared with a reference window. Model quality: accuracy-type metrics computed when labels arrive, plus business proxies that arrive sooner. Everything should be sliceable by model version and key segments, and alerts should have owners and a runbook.

Use leading indicators. Monitor input and prediction distributions against the training reference. Track proxy outcomes that arrive faster, for example early-life behavior that correlates with the eventual label. Sample some cases for manual or expedited labeling. Keep a backtesting process: when labels do arrive, compute quality for each past period and model version so you learn how well the leading indicators predicted the actual outcome.

Several layers. Unit tests for data transformations and feature logic. Data tests on schema and distributions of the training set. Training smoke tests that run a small sample end to end and check the loss decreases and outputs have the right shape and no NaNs. Model tests on a fixed evaluation set with minimum thresholds, per segment, and compared with the production model. Behavioral tests such as invariances and known edge cases. Serving tests: the container starts, the prediction API honors its contract, and load behavior is acceptable. A framework like pytest handles most of this.

Resist building a platform. Start from the pain: what is slow or risky today? A reasonable first design: Git-based projects with a standard template, a shared tracking server and registry, containerized training jobs launched by CI or a scheduler, a standard serving template (a container exposing a validated API), and common monitoring dashboards. Add an orchestrator when pipelines become dependent on each other, and a feature store when feature duplication causes real inconsistencies. Name the build-versus-buy tradeoffs and the staffing needed to run what you propose, since an unmaintained platform is worse than none.

Clarify requirements first: latency budget, availability target, cost of false positives and negatives, label delay (chargebacks arrive weeks later). Serving: a low-latency online service with precomputed account-level features in a fast store and request-time features computed in the service, with a rule-based fallback if the model is unavailable. Release: shadow first, then a small canary with automatic rollback on error and latency, plus decision-rate monitoring. Monitoring: input drift, score distribution, approval and decline rates by segment, and delayed performance when chargebacks arrive. Governance: version every decision with the model version and feature vector for audit. Mention adversarial adaptation as a reason concept drift is expected.

The artifacts change: prompts, retrieval indexes, tool definitions and evaluation sets are what you version, and you often do not train anything. Failures are semantic, so you need evaluation (programmatic checks, model-graded scoring, user feedback) on top of service metrics. Cost per request is an operational metric because it scales with tokens. Providers change model versions, so regression tests matter. Traces are the main debugging tool. See LLM monitoring and observability.

Separate retrieval from generation. Build a question set from real documents with the source passages recorded. Measure whether retrieval puts the right passage in the top results, then measure answer correctness and faithfulness to the retrieved context, with human review on a sample to calibrate any model-based judge. Include questions that should be refused because the answer is not in the corpus. Run the set in CI for every change to prompts, models and retrieval settings.