Convert Figma logo to code with AI

mlflow logomlflow

Open source platform for the machine learning lifecycle

19,276
4,318
19,276
1,685

Top Related Projects

186,879

An Open Source Machine Learning Framework for Everyone

85,015

Tensors and Dynamic neural networks in Python with strong GPU acceleration

scikit-learn: machine learning in Python

9,390

The AI developer platform. Use Weights & Biases to train and fine-tune models, and manage models from experimentation to production.

34,860

Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.

11,258

A hyperparameter optimization framework

Quick Overview

MLflow is an open-source platform for managing the end-to-end machine learning lifecycle. It provides tools for tracking experiments, packaging code into reproducible runs, and sharing and deploying models. MLflow works with any machine learning library and can be used with any programming language.

Pros

  • Language-agnostic: Works with any ML library and programming language
  • Comprehensive lifecycle management: Covers experiment tracking, model packaging, and deployment
  • Easy integration: Can be easily integrated into existing ML workflows
  • Active community: Regularly updated with new features and improvements

Cons

  • Learning curve: Can be complex for beginners to set up and use effectively
  • Limited built-in visualizations: May require additional tools for advanced data visualization
  • Potential overhead: For simple projects, MLflow might introduce unnecessary complexity

Code Examples

Tracking an experiment:

import mlflow

with mlflow.start_run():
    mlflow.log_param("param1", 5)
    mlflow.log_metric("accuracy", 0.85)
    mlflow.log_artifact("model.pkl")

Saving and loading a model:

import mlflow.sklearn
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestRegressor

# Train a model
X_train, X_test, y_train, y_test = train_test_split(X, y)
rf = RandomForestRegressor()
rf.fit(X_train, y_train)

# Save the model
mlflow.sklearn.log_model(rf, "random_forest_model")

# Load the model
loaded_model = mlflow.sklearn.load_model("random_forest_model")

Comparing runs:

from mlflow.tracking import MlflowClient

client = MlflowClient()
runs = client.search_runs(["experiment_id"], "metrics.accuracy > 0.9")
for run in runs:
    print(f"Run ID: {run.info.run_id}, Accuracy: {run.data.metrics['accuracy']}")

Getting Started

  1. Install MLflow:

    pip install mlflow
    
  2. Start the MLflow UI:

    mlflow ui
    
  3. In your Python script, import and use MLflow:

    import mlflow
    
    with mlflow.start_run():
        # Your ML code here
        mlflow.log_param("param_name", param_value)
        mlflow.log_metric("metric_name", metric_value)
    
  4. View your results in the MLflow UI at http://localhost:5000

Competitor Comparisons

186,879

An Open Source Machine Learning Framework for Everyone

Pros of TensorFlow

  • More comprehensive deep learning framework with extensive neural network capabilities
  • Larger ecosystem and community support for AI/ML development
  • Better performance for large-scale machine learning tasks

Cons of TensorFlow

  • Steeper learning curve for beginners
  • More complex setup and configuration process
  • Heavier resource requirements for development and deployment

Code Comparison

MLflow example:

import mlflow

mlflow.start_run()
mlflow.log_param("param1", 5)
mlflow.log_metric("accuracy", 0.85)
mlflow.end_run()

TensorFlow example:

import tensorflow as tf

model = tf.keras.Sequential([
    tf.keras.layers.Dense(64, activation='relu'),
    tf.keras.layers.Dense(10, activation='softmax')
])
model.compile(optimizer='adam', loss='categorical_crossentropy')

Summary

MLflow is focused on experiment tracking and model management, making it easier to organize and compare machine learning experiments. TensorFlow, on the other hand, is a comprehensive deep learning framework that provides powerful tools for building and training neural networks. While MLflow is more accessible for beginners and offers simpler experiment tracking, TensorFlow excels in performance and advanced AI/ML capabilities. The choice between the two depends on the specific requirements of your machine learning project and your level of expertise in the field.

85,015

Tensors and Dynamic neural networks in Python with strong GPU acceleration

Pros of PyTorch

  • More comprehensive deep learning framework with extensive neural network capabilities
  • Larger community and ecosystem, with more third-party libraries and resources
  • Dynamic computational graph allows for more flexible and intuitive model development

Cons of PyTorch

  • Steeper learning curve for beginners compared to MLflow's simpler interface
  • Less focus on experiment tracking and model management features
  • Requires more code to set up and manage experiments

Code Comparison

MLflow example:

import mlflow

mlflow.start_run()
mlflow.log_param("learning_rate", 0.01)
mlflow.log_metric("accuracy", 0.85)
mlflow.end_run()

PyTorch example:

import torch
import torch.nn as nn

model = nn.Linear(10, 1)
criterion = nn.MSELoss()
optimizer = torch.optim.SGD(model.parameters(), lr=0.01)

for epoch in range(100):
    # Training loop

MLflow focuses on experiment tracking and model management, while PyTorch provides a comprehensive deep learning framework. MLflow's code is simpler for logging experiments, whereas PyTorch requires more setup for model definition and training. PyTorch offers greater flexibility and power for complex neural network architectures, but MLflow excels in organizing and tracking machine learning workflows across different frameworks.

scikit-learn: machine learning in Python

Pros of scikit-learn

  • Comprehensive library for machine learning algorithms and data preprocessing
  • Well-established, mature project with extensive documentation and community support
  • Seamless integration with other scientific Python libraries (NumPy, Pandas, etc.)

Cons of scikit-learn

  • Focused primarily on model development, lacking built-in experiment tracking and model management
  • Limited support for deep learning and GPU acceleration
  • Less emphasis on model deployment and production workflows

Code Comparison

scikit-learn:

from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)
model = RandomForestClassifier()
model.fit(X_train, y_train)

MLflow:

import mlflow
from sklearn.ensemble import RandomForestClassifier

with mlflow.start_run():
    model = RandomForestClassifier()
    model.fit(X_train, y_train)
    mlflow.sklearn.log_model(model, "random_forest_model")

The code comparison illustrates that scikit-learn focuses on model implementation and training, while MLflow adds experiment tracking and model logging capabilities on top of the machine learning workflow.

9,390

The AI developer platform. Use Weights & Biases to train and fine-tune models, and manage models from experimentation to production.

Pros of Wandb

  • More comprehensive visualization and reporting tools
  • Better support for distributed and collaborative experiments
  • Easier integration with cloud platforms and services

Cons of Wandb

  • Steeper learning curve for beginners
  • Requires internet connection for full functionality
  • Some advanced features are only available in paid plans

Code Comparison

MLflow:

import mlflow

mlflow.start_run()
mlflow.log_param("param1", value1)
mlflow.log_metric("metric1", value2)
mlflow.end_run()

Wandb:

import wandb

wandb.init(project="my_project")
wandb.config.param1 = value1
wandb.log({"metric1": value2})
wandb.finish()

Both MLflow and Wandb are popular experiment tracking and model management tools. MLflow offers a more straightforward, open-source approach with local storage options, while Wandb provides advanced collaboration features and cloud-based solutions. MLflow is generally easier for beginners to set up and use, but Wandb offers more powerful visualization and reporting capabilities. The choice between the two often depends on specific project requirements, team size, and infrastructure preferences.

34,860

Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.

Pros of Ray

  • Distributed computing framework for scaling AI and ML applications
  • Supports both task-parallel and data-parallel computations
  • Offers a rich ecosystem of libraries for reinforcement learning, hyperparameter tuning, and more

Cons of Ray

  • Steeper learning curve compared to MLflow
  • More complex setup and configuration for distributed environments
  • Less focus on experiment tracking and model management

Code Comparison

MLflow example:

import mlflow

with mlflow.start_run():
    mlflow.log_param("param1", 5)
    mlflow.log_metric("accuracy", 0.85)
    mlflow.sklearn.log_model(model, "model")

Ray example:

import ray

@ray.remote
def task(x):
    return x * x

futures = [task.remote(i) for i in range(4)]
print(ray.get(futures))

MLflow focuses on experiment tracking and model management, while Ray emphasizes distributed computing and scalability. MLflow's API is simpler for logging experiments, while Ray's API is designed for parallel and distributed task execution. Ray offers more flexibility for large-scale computations but requires more setup. MLflow provides better out-of-the-box support for model versioning and deployment.

11,258

A hyperparameter optimization framework

Pros of Optuna

  • Focuses specifically on hyperparameter optimization with advanced algorithms
  • Provides visualization tools for hyperparameter importance and optimization history
  • Supports distributed optimization out of the box

Cons of Optuna

  • Limited experiment tracking capabilities compared to MLflow
  • Lacks built-in model packaging and deployment features
  • Smaller ecosystem and community support

Code Comparison

MLflow experiment logging:

import mlflow

with mlflow.start_run():
    mlflow.log_param("learning_rate", 0.01)
    mlflow.log_metric("accuracy", 0.85)
    mlflow.sklearn.log_model(model, "model")

Optuna hyperparameter optimization:

import optuna

def objective(trial):
    lr = trial.suggest_loguniform('lr', 1e-5, 1e-1)
    model = train_model(lr)
    return evaluate_model(model)

study = optuna.create_study(direction='maximize')
study.optimize(objective, n_trials=100)

Both MLflow and Optuna are powerful tools for machine learning workflows, but they serve different primary purposes. MLflow is a comprehensive platform for managing the entire ML lifecycle, including experiment tracking, model packaging, and deployment. Optuna, on the other hand, specializes in hyperparameter optimization with advanced algorithms and visualization capabilities. The choice between the two depends on the specific needs of your ML project.

Convert Figma logo designs to code with AI

Visual Copilot

Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.

Try Visual Copilot

README

MLflow: A Machine Learning Lifecycle Platform

Latest Docs Apache 2 License Total Downloads Slack Twitter

MLflow is an open-source platform, purpose-built to assist machine learning practitioners and teams in handling the complexities of the machine learning process. MLflow focuses on the full lifecycle for machine learning projects, ensuring that each phase is manageable, traceable, and reproducible


The core components of MLflow are:

  • Experiment Tracking 📝: A set of APIs to log models, params, and results in ML experiments and compare them using an interactive UI.
  • Model Packaging 📦: A standard format for packaging a model and its metadata, such as dependency versions, ensuring reliable deployment and strong reproducibility.
  • Model Registry 💾: A centralized model store, set of APIs, and UI, to collaboratively manage the full lifecycle of MLflow Models.
  • Serving 🚀: Tools for seamless model deployment to batch and real-time scoring on platforms like Docker, Kubernetes, Azure ML, and AWS SageMaker.
  • Evaluation 📊: A suite of automated model evaluation tools, seamlessly integrated with experiment tracking to record model performance and visually compare results across multiple models.
  • Observability 🔍: Tracing integrations with various GenAI libraries and a Python SDK for manual instrumentation, offering smoother debugging experience and supporting online monitoring.
MLflow Hero

Installation

To install the MLflow Python package, run the following command:

pip install mlflow

Alternatively, you can install MLflow from on differnet package hosting platforms:

PyPIPyPI - mlflow PyPI - mlflow-skinny
conda-forgeConda - mlflow Conda - mlflow-skinny
CRANCRAN - mlflow
Maven CentralMaven Central - mlflow-client Maven Central - mlflow-parent Maven Central - mlflow-scoring Maven Central - mlflow-spark

Documentation 📘

Official documentation for MLflow can be found at here.

Usage

Experiment Tracking (Doc)

The following examples trains a simple regression model with scikit-learn, while enabling MLflow's autologging feature for experiment tracking.

import mlflow

from sklearn.model_selection import train_test_split
from sklearn.datasets import load_diabetes
from sklearn.ensemble import RandomForestRegressor

# Enable MLflow's automatic experiment tracking for scikit-learn
mlflow.sklearn.autolog()

# Load the training dataset
db = load_diabetes()
X_train, X_test, y_train, y_test = train_test_split(db.data, db.target)

rf = RandomForestRegressor(n_estimators=100, max_depth=6, max_features=3)
# MLflow triggers logging automatically upon model fitting
rf.fit(X_train, y_train)

Once the above code finishes, run the following command in a separate terminal and access the MLflow UI via the printed URL. An MLflow Run should be automatically created, which tracks the training dataset, hyper parameters, performance metrics, the trained model, dependencies, and even more.

mlflow ui

Serving Models (Doc)

You can deploy the logged model to a local inference server by a one-line command using the MLflow CLI. Visit the documentation for how to deploy models to other hosting platforms.

mlflow models serve --model-uri runs:/<run-id>/model

Evaluating Models (Doc)

The following example runs automatic evaluation for question-answering tasks with several built-in metrics.

import mlflow
import pandas as pd

# Evaluation set contains (1) input question (2) model outputs (3) ground truth
df = pd.DataFrame(
    {
        "inputs": ["What is MLflow?", "What is Spark?"],
        "outputs": [
            "MLflow is an innovative fully self-driving airship powered by AI.",
            "Sparks is an American pop and rock duo formed in Los Angeles.",
        ],
        "ground_truth": [
            "MLflow is an open-source platform for managing the end-to-end machine learning (ML) "
            "lifecycle.",
            "Apache Spark is an open-source, distributed computing system designed for big data "
            "processing and analytics.",
        ],
    }
)
eval_dataset = mlflow.data.from_pandas(
    df, predictions="outputs", targets="ground_truth"
)

# Start an MLflow Run to record the evaluation results to
with mlflow.start_run(run_name="evaluate_qa"):
    # Run automatic evaluation with a set of built-in metrics for question-answering models
    results = mlflow.evaluate(
        data=eval_dataset,
        model_type="question-answering",
    )

print(results.tables["eval_results_table"])

Observability (Doc)

MLflow Tracing provides LLM observability for various GenAI libraries such as OpenAI, LangChain, LlamaIndex, DSPy, AutoGen, and more. To enable auto-tracing, call mlflow.xyz.autolog() before running your models. Refer to the documentation for customization and manual instrumentation.

import mlflow
from openai import OpenAI

# Enable tracing for OpenAI
mlflow.openai.autolog()

# Query OpenAI LLM normally
response = OpenAI().chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hi!"}],
    temperature=0.1,
)

Then navigate to the "Traces" tab in the MLflow UI to find the trace records OpenAI query.

Community

  • For help or questions about MLflow usage (e.g. "how do I do X?") visit the docs or Stack Overflow.
  • Alternatively, you can ask the question to our AI-powered chat bot. Visit the doc website and click on the "Ask AI" button at the right bottom to start chatting with the bot.
  • To report a bug, file a documentation issue, or submit a feature request, please open a GitHub issue.
  • For release announcements and other discussions, please subscribe to our mailing list (mlflow-users@googlegroups.com) or join us on Slack.

Contributing

We happily welcome contributions to MLflow! We are also seeking contributions to items on the MLflow Roadmap. Please see our contribution guide to learn more about contributing to MLflow.

Core Members

MLflow is currently maintained by the following core members with significant contributions from hundreds of exceptionally talented community members.