Reproducible Experiment Records

Connect code, data, and results

A run's parameters and metrics are far more useful if they're anchored to the exact code and data that produced them. Tag each run with its Git commit hash (identifying the precise snapshot of source code that was running) and its Data Version Control (DVC) data revision (covered in the next module) - together these two IDs let anyone, months later, check out the identical code and identical data and reproduce the run from scratch. It's also worth saving the exact prompt template and the evaluation output as artifacts on the run, so the inputs that produced a result are preserved right alongside the result itself. One firm rule, though: never log passwords, API keys, or raw prompts that contain sensitive personal or customer data as MLflow parameters or artifacts - those get stored, potentially shared, and rarely get cleaned up.

with mlflow.start_run():
    mlflow.set_tag("git_commit", "a1b2c3d")
    mlflow.set_tag("dvc_rev", "v3-cleaned")
    try:
        ...  # do the actual work
    except Exception:
        mlflow.end_run(status="FAILED")
        raise

Using MLflow's run context manager - the with mlflow.start_run(): block, as shown above - matters for a subtle but important reason: if an exception is raised partway through, the with block still correctly closes the run and, when handled as shown, marks it as failed rather than leaving it hanging in an ambiguous "still running" state forever. Finally, give experiments genuinely descriptive names and runs meaningful, human-readable descriptions rather than relying purely on MLflow's auto-generated IDs - a list of forty runs named chunk-size-sweep with clear descriptions is navigable months later; a list of forty runs identified only by random hex strings is not.

Goal: The MLflow lab starts with a script pointed at the wrong tracking store - the exact failure mode from the previous section. Correct the URI and prove the required run actually exists in the shared local backend.