Track experiments and models with MLflow

Dit deel is nog niet vertaald. Je ziet de Engelse tekst.

$ als gewone gebruiker# als root (of met sudo)

Run an MLflow server

A tracking server with a database for metadata and object storage for artifacts.

$pip install mlflow
$mlflow server --host 0.0.0.0 --port 5000 --backend-store-uri sqlite:///mlflow.db --default-artifact-root ./mlruns
$export MLFLOW_TRACKING_URI=http://localhost:5000

In production: PostgreSQL as backend store, Azure Blob / S3 as artifact store, behind a reverse proxy with login.

3.2

Log a training run

Wrap training in a run; log what you need to compare runs later.

train_mlflow.py

import mlflow
import mlflow.sklearn
from sklearn.datasets import load_iris
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import accuracy_score
from sklearn.model_selection import train_test_split
 
mlflow.set_experiment("iris")
X, y = load_iris(return_X_y=True)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.2, random_state=42)
 
with mlflow.start_run():
    n = 200
    model = RandomForestClassifier(n_estimators=n, random_state=42).fit(X_tr, y_tr)
    acc = accuracy_score(y_te, model.predict(X_te))
    mlflow.log_param("n_estimators", n)
    mlflow.log_metric("accuracy", acc)
    mlflow.sklearn.log_model(model, name="model", registered_model_name="iris-classifier")

3.3

Compare runs

The MLflow UI (port 5000) lists runs with their parameters and metrics; choose the best one.

$mlflow runs list --experiment-id 1
$mlflow ui

3.4

Promote models in the registry

Registered models have versions; aliases (like champion, challenger) mark which version serves.

promote.py

from mlflow import MlflowClient
 
c = MlflowClient()
c.set_registered_model_alias("iris-classifier", "champion", version=3)
mv = c.get_model_version_by_alias("iris-classifier", "champion")
print(mv.version, mv.run_id)

$mlflow models serve -m "models:/iris-classifier@champion" -p 8081 --env-manager local
$curl -s -X POST http://localhost:8081/invocations -H 'Content-Type: application/json' -d '{"inputs": [[5.1, 3.5, 1.4, 0.2]]}'

Onthoud

The serving system asks the registry for "the champion" — promotion is changing an alias, not copying files.

Alle commando's in dit hoofdstuk