mitsuba3

Mitsuba 3: A Retargetable Forward and Inverse Renderer

2,007

229

2,007

View on GitHub

Top Related Projects

OpenShadingLanguage

2,072

Advanced shading language for production GI renderers

pbrt-v4

2,810

Source code to pbrt, the ray tracer described in the forthcoming 4th edition of the "Physically Based Rendering: From Theory to Implementation" book.

appleseed

2,188

A modern open source rendering engine for animation and visual effects

Quick Overview

Mitsuba 3 is an open-source rendering system for physically-based graphics. It is a research-oriented rendering framework that supports various rendering techniques, including path tracing, photon mapping, and bidirectional methods. Mitsuba 3 is designed to be extensible and efficient, making it suitable for both academic research and production environments.

Pros

Highly extensible architecture allowing easy integration of new rendering algorithms and techniques
Supports differentiable rendering, enabling gradient-based optimization of scene parameters
Efficient implementation with support for GPU acceleration using CUDA and OptiX
Comprehensive documentation and active community support

Cons

Steep learning curve for beginners due to its research-oriented nature
Limited built-in support for real-time rendering techniques
Requires significant computational resources for complex scenes and advanced rendering methods
Some features may be less stable or well-documented compared to commercial rendering solutions

Code Examples

Basic scene setup and rendering:

import mitsuba as mi

mi.set_variant('cuda_ad_rgb')

scene = mi.load_file('scene.xml')
sensor = scene.sensors()[0]

image = mi.render(scene, spp=64)
mi.util.write_bitmap('output.exr', image)

Differentiable rendering and optimization:

import mitsuba as mi
import drjit as dr

mi.set_variant('cuda_ad_rgb')

scene = mi.load_file('scene.xml')
params = mi.traverse(scene)

optimizer = mi.ad.Adam(params, lr=0.02)

for i in range(100):
    loss = mi.render(scene, spp=16)
    dr.backward(loss)
    optimizer.step()

Custom integrator implementation:

import mitsuba as mi

class CustomIntegrator(mi.SamplingIntegrator):
    def sample(self, scene, sampler, ray, medium, active):
        # Custom integrator logic here
        return mi.Spectrum(0.0)

mi.register_integrator("custom", lambda props: CustomIntegrator(props))

Getting Started

To get started with Mitsuba 3:

Install Mitsuba 3 using pip:
```
pip install mitsuba
```

Import Mitsuba and set the desired variant:

import mitsuba as mi
mi.set_variant('cuda_ad_rgb')

Load a scene and render it:

scene = mi.load_file('path/to/scene.xml')
image = mi.render(scene, spp=64)
mi.util.write_bitmap('output.exr', image)

For more detailed information and advanced usage, refer to the official Mitsuba 3 documentation.

Competitor Comparisons

Falcor

2,633

Real-Time Rendering Framework

Pros of Falcor

More focused on real-time rendering and game development
Extensive support for DirectX 12 and Vulkan
Includes built-in profiling and debugging tools

Cons of Falcor

Less flexible for non-real-time rendering applications
Steeper learning curve for researchers not familiar with game engine architectures
Limited support for offline rendering techniques

Code Comparison

Falcor (C++):

void SimpleRenderPass::execute(RenderContext* pRenderContext, const RenderData& renderData)
{
    // Bind resources and dispatch
    mpVars->setTexture("gOutput", renderData["output"]->asTexture());
    mpState->setProgram(mpProgram);
    pRenderContext->draw(mpState.get(), mpVars.get(), mpVertexBuffer.get(), mpIndexBuffer.get(), 3, 0);
}

Mitsuba 3 (Python):

def render(scene, spp=16):
    sensor = scene.sensors()[0]
    film = sensor.film()
    sampler = sensor.sampler()
    integrator = scene.integrator()
    
    with dr.suspend_grad():
        result = integrator.sample(scene, sampler, sensor, spp=spp)
    
    return film.develop()

This comparison highlights Falcor's focus on low-level graphics programming and real-time rendering, while Mitsuba 3 provides a higher-level, more research-oriented interface for physically-based rendering.

OpenShadingLanguage

2,072

Advanced shading language for production GI renderers

Pros of OpenShadingLanguage

Widely adopted in the film and VFX industry, with support from major studios
Extensive documentation and community resources
Flexible and extensible shading language designed for production use

Cons of OpenShadingLanguage

Steeper learning curve compared to Mitsuba3's Python-based workflow
Less focus on research-oriented features and differentiable rendering
May require more setup and integration effort for custom rendering pipelines

Code Comparison

OpenShadingLanguage:

shader example(
    color Diffuse = color(0.8),
    output closure color BSDF = diffuse(N)
)
{
    BSDF = Diffuse * diffuse(N);
}

Mitsuba3:

class MyBSDF(mi.BSDF):
    def __init__(self, props):
        mi.BSDF.__init__(self, props)
        self.diffuse = props['diffuse']

    def sample(self, si, sample1, sample2, active):
        # Implement BSDF sampling here
        pass

OpenShadingLanguage uses a C-like syntax for shader definitions, while Mitsuba3 employs Python classes for material definitions. OSL is more specialized for shading, whereas Mitsuba3's approach is more integrated with its overall Python-based architecture.

pbrt-v4

2,810

Source code to pbrt, the ray tracer described in the forthcoming 4th edition of the "Physically Based Rendering: From Theory to Implementation" book.

Pros of pbrt-v4

More comprehensive documentation and educational resources
Wider industry adoption and recognition
Extensive support for various rendering techniques and algorithms

Cons of pbrt-v4

Less flexible for real-time rendering applications
Steeper learning curve for beginners
Limited support for GPU acceleration compared to Mitsuba3

Code Comparison

Mitsuba3:

import mitsuba as mi
mi.set_variant('cuda_ad_rgb')

scene = mi.load_file('scene.xml')
image = mi.render(scene, spp=64)
mi.util.write_bitmap('output.exr', image)

pbrt-v4:

#include <pbrt/pbrt.h>

int main(int argc, char *argv[]) {
    pbrt::Init(options);
    pbrt::ParseFile(filename);
    pbrt::Render();
    pbrt::CleanUp();
    return 0;
}

Both repositories offer powerful rendering capabilities, but Mitsuba3 provides a more user-friendly Python interface and better GPU support, while pbrt-v4 excels in educational value and industry recognition. The choice between them depends on specific project requirements and user expertise.

LuxCore

1,149

LuxCore source repository

Pros of LuxCore

More mature and production-ready renderer with a longer development history
Extensive documentation and user guides available
Wider range of supported platforms, including Windows, macOS, and Linux

Cons of LuxCore

Less focus on research-oriented features and experimental rendering techniques
Potentially slower development cycle for cutting-edge rendering algorithms
More complex codebase due to its longer history and broader feature set

Code Comparison

LuxCore (C++):

Scene *Scene::FromProperties(const Properties &scnProps, const Properties &cfgProps) {
    Scene *scene = new Scene(cfgProps);
    scene->Parse(scnProps);
    return scene;
}

Mitsuba3 (Python):

def load_file(filename: str, integrator: Optional[dict] = None,
              resx: Optional[int] = None, resy: Optional[int] = None,
              **kwargs) -> Tuple[mi.Scene, mi.Sensor]:
    from mitsuba import xml
    return xml.load_file(filename, integrator, resx, resy, **kwargs)

Both renderers use different approaches for scene creation and loading. LuxCore employs a C++ based approach with Properties objects, while Mitsuba3 utilizes Python with optional parameters and dictionary-based configuration.

OpenUSD

5,989

Universal Scene Description

Pros of OpenUSD

Widely adopted industry standard for 3D content creation and exchange
Extensive documentation and community support
Robust toolset for complex scene composition and animation

Cons of OpenUSD

Steeper learning curve compared to Mitsuba3
Larger codebase and potentially more complex setup
Primarily focused on scene description rather than rendering

Code Comparison

OpenUSD (C++):

#include "pxr/usd/usd/stage.h"
#include "pxr/usd/usdGeom/sphere.h"

auto stage = pxr::UsdStage::CreateNew("sphere.usda");
auto spherePrim = pxr::UsdGeomSphere::Define(stage, "/mySphere");

Mitsuba3 (Python):

import mitsuba as mi

scene = mi.load_file("sphere.xml")
params = mi.traverse(scene)
image = mi.render(scene, spp=16)

The code snippets demonstrate the different focus of each project. OpenUSD is centered around scene description and composition, while Mitsuba3 is geared towards physically-based rendering. OpenUSD requires more setup for scene creation, whereas Mitsuba3 provides a more streamlined approach for rendering tasks.

appleseed

2,188

A modern open source rendering engine for animation and visual effects

Pros of appleseed

More extensive documentation and user guides
Wider range of supported platforms, including Windows and macOS
Active community with regular releases and updates

Cons of appleseed

Less focus on research-oriented features compared to Mitsuba3
Potentially slower rendering performance for certain scenes
Steeper learning curve for beginners

Code Comparison

appleseed:

import appleseed as asr

# Create a project
project = asr.Project('my_project')

# Add a camera to the scene
camera = asr.Camera('pinhole_camera', 'camera', {'film_width': 0.024})
project.get_scene().cameras().insert(camera)

Mitsuba3:

import mitsuba as mi

# Create a scene
scene = mi.load_dict({
    'type': 'scene',
    'camera': {
        'type': 'perspective',
        'fov': 45,
        'to_world': mi.ScalarTransform4f.look_at(
            origin=[0, 0, 4],
            target=[0, 0, 0],
            up=[0, 1, 0]
        )
    }
})

Both renderers offer Python APIs for scene setup, but Mitsuba3's approach is more concise and uses a dictionary-based configuration.

Convert designs to code with AI

Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.

Try Visual Copilot

README

Mitsuba banner

Mitsuba Renderer 3

Documentation	Tutorial videos	Linux	MacOS	Windows	PyPI

Warning

There currently is a large amount of undocumented and unstable work going on in the master branch. We'd highly recommend you use our latest release until further notice.

Introduction

Mitsuba 3 is a research-oriented rendering system for forward and inverse light transport simulation developed at EPFL in Switzerland. It consists of a core library and a set of plugins that implement functionality ranging from materials and light sources to complete rendering algorithms.

Mitsuba 3 is retargetable: this means that the underlying implementations and data structures can transform to accomplish various different tasks. For example, the same code can simulate both scalar (classic one-ray-at-a-time) RGB transport or differential spectral transport on the GPU. This all builds on Dr.Jit, a specialized just-in-time (JIT) compiler developed specifically for this project.

Main Features

Cross-platform: Mitsuba 3 has been tested on Linux (x86_64), macOS (aarch64, x86_64), and Windows (x86_64).
High performance: The underlying Dr.Jit compiler fuses rendering code into kernels that achieve state-of-the-art performance using an LLVM backend targeting the CPU and a CUDA/OptiX backend targeting NVIDIA GPUs with ray tracing hardware acceleration.
Python first: Mitsuba 3 is deeply integrated with Python. Materials, textures, and even full rendering algorithms can be developed in Python, which the system JIT-compiles (and optionally differentiates) on the fly. This enables the experimentation needed for research in computer graphics and other disciplines.
Differentiation: Mitsuba 3 is a differentiable renderer, meaning that it can compute derivatives of the entire simulation with respect to input parameters such as camera pose, geometry, BSDFs, textures, and volumes. It implements recent differentiable rendering algorithms developed at EPFL.
Spectral & Polarization: Mitsuba 3 can be used as a monochromatic renderer, RGB-based renderer, or spectral renderer. Each variant can optionally account for the effects of polarization if desired.

Tutorial videos, documentation

We've recorded several YouTube videos that provide a gentle introduction Mitsuba 3 and Dr.Jit. Beyond this you can find complete Juypter notebooks covering a variety of applications, how-to guides, and reference documentation on readthedocs.

Installation

We provide pre-compiled binary wheels via PyPI. Installing Mitsuba this way is as simple as running

pip install mitsuba

on the command line. The Python package includes four variants by default:

scalar_spectral
scalar_rgb
llvm_ad_rgb
cuda_ad_rgb

The first two perform classic one-ray-at-a-time simulation using either a RGB or spectral color representation, while the latter two can be used for inverse rendering on the CPU or GPU. To access additional variants, you will need to compile a custom version of Dr.Jit using CMake. Please see the documentation for details on this.

Requirements

Python >= 3.8
(optional) For computation on the GPU: Nvidia driver >= 495.89
(optional) For vectorized / parallel computation on the CPU: LLVM >= 11.1

Usage

Here is a simple "Hello World" example that shows how simple it is to render a scene using Mitsuba 3 from Python:

# Import the library using the alias "mi"
import mitsuba as mi
# Set the variant of the renderer
mi.set_variant('scalar_rgb')
# Load a scene
scene = mi.load_dict(mi.cornell_box())
# Render the scene
img = mi.render(scene)
# Write the rendered image to an EXR file
mi.Bitmap(img).write('cbox.exr')

Tutorials and example notebooks covering a variety of applications can be found in the documentation.

About

This project was created by Wenzel Jakob. Significant features and/or improvements to the code were contributed by SÃ©bastien Speierer, Nicolas Roussel, Merlin Nimier-David, Delio Vicini, Tizian Zeltner, Baptiste Nicolet, Miguel Crespo, Vincent Leroy, and Ziyi Zhang.

When using Mitsuba 3 in academic projects, please cite:

@software{Mitsuba3,
    title = {Mitsuba 3 renderer},
    author = {Wenzel Jakob and SÃ©bastien Speierer and Nicolas Roussel and Merlin Nimier-David and Delio Vicini and Tizian Zeltner and Baptiste Nicolet and Miguel Crespo and Vincent Leroy and Ziyi Zhang},
    note = {https://mitsuba-renderer.org},
    version = {3.1.1},
    year = 2022
}

Top Related Projects

Convert designs to code with AI

Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.

Try Visual Copilot