MLflow and DVC Together: A Reproducible Machine Learning Workflow

Reproducibility Needs More Than a Saved Model

A model artifact alone cannot explain which rows trained it, which code created its features, or which parameters produced its metrics. Git, DVC, and MLflow solve different parts of that chain:

Do not use an MLflow run name as a substitute for a data version, or commit a large dataset directly to Git because the experiment tracker has a copy.

---

Record One Join Key Across the Systems

At training time capture the Git SHA and DVC data revision in MLflow:

import subprocess
import mlflow

git_sha = subprocess.check_output(["git", "rev-parse", "HEAD"], text=True).strip()
dvc_status = subprocess.check_output(["dvc", "status", "--json"], text=True)

with mlflow.start_run():
    mlflow.set_tag("git.sha", git_sha)
    mlflow.log_text(dvc_status, "provenance/dvc-status.json")
    mlflow.log_params({"max_depth": 8, "class_weight": "balanced"})
    # train, evaluate, and log the resulting metrics and model

The pipeline should refuse a release from a dirty code tree or uncommitted data change unless an explicit experimental policy allows it. A convenient run is not automatically a reproducible run.

Keep Evaluation Honest

Version the split definition or stable row identifiers, not only the source CSV. Otherwise a changed random seed or row order can quietly move examples between training and holdout sets. Log class balance, feature schema, and confusion-matrix artifacts alongside headline metrics.

Accuracy can improve while recall for the important minority class collapses. Promotion rules should name the operational metric and threshold, compare against the current champion, and retain evidence for why a candidate passed.

Recovery Workflow

When a reported model cannot be reproduced:

1. Resolve the MLflow run and read its Git SHA, data revision, parameters, and environment artifact.
2. Check out the code revision and pull the corresponding DVC objects.
3. Verify checksums and feature schema before training.
4. Re-run evaluation against the recorded split.
5. Compare metric artifacts and dependency versions before blaming nondeterminism.

Practice missing-run recovery in Restore MLflow Experiment Records and data-lineage repair in Recover DVC Dataset Drift.