ORB_SLAM3

ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAM

6,394

2,514

6,394

525

View on GitHub

Top Related Projects

ORB_SLAM2

9,286

Real-Time SLAM for Monocular, Stereo and RGB-D Cameras, with Loop Detection and Relocalization Capabilities

Kimera-VIO

1,527

Visual Inertial Odometry with SLAM capabilities and 3D Mesh generation.

VINS-Mono

4,926

A Robust and Versatile Monocular Visual-Inertial State Estimator

PythonRobotics

23,098

Python sample codes for robotics algorithms.

kalibr

4,351

The Kalibr visual-inertial calibration toolbox

Quick Overview

ORB_SLAM3 is an open-source, real-time SLAM (Simultaneous Localization and Mapping) library for monocular, stereo, and RGB-D cameras. It's the first system capable of performing visual, visual-inertial, and multi-map SLAM with monocular, stereo, and RGB-D cameras, using pin-hole and fisheye lens models. ORB_SLAM3 is accurate, robust, and quick to relocalize in previously mapped areas.

Pros

Versatile: Supports various camera configurations and lens models
Accurate and robust: Provides precise localization and mapping
Real-time performance: Suitable for robotics and augmented reality applications
Multi-map SLAM: Capable of handling multiple maps and environments

Cons

Complex setup: Requires careful configuration and calibration
Computational demands: May require powerful hardware for optimal performance
Learning curve: Understanding and using the system effectively can be challenging
Limited to visual and inertial data: Does not incorporate other sensor types like LiDAR

Code Examples

Initializing the SLAM system:

#include "System.h"

ORB_SLAM3::System SLAM(vocabulary_path, settings_path, ORB_SLAM3::System::MONOCULAR);

Processing a frame:

cv::Mat image = cv::imread(image_path, cv::IMREAD_UNCHANGED);
double timestamp = frame_number / camera_fps;
Sophus::SE3f pose = SLAM.TrackMonocular(image, timestamp);

Saving and loading a map:

// Save map
SLAM.SaveMap("map.osa");

// Load map
SLAM.LoadMap("map.osa");

Getting Started

Clone the repository:

git clone https://github.com/UZ-SLAMLab/ORB_SLAM3.git ORB_SLAM3

Build the project:

cd ORB_SLAM3
chmod +x build.sh
./build.sh

Download a vocabulary file and prepare your camera settings YAML file.

Run an example:

./Examples/Monocular/mono_euroc Vocabulary/ORBvoc.txt Examples/Monocular/EuRoC.yaml PATH_TO_SEQUENCE_FOLDER/mav0/cam0/data Examples/Monocular/EuRoC_TimeStamps/SEQUENCE.txt

Replace PATH_TO_SEQUENCE_FOLDER and SEQUENCE with appropriate values for your dataset.

Competitor Comparisons

ORB_SLAM2

9,286

Real-Time SLAM for Monocular, Stereo and RGB-D Cameras, with Loop Detection and Relocalization Capabilities

Pros of ORB_SLAM2

Simpler implementation, easier to understand and modify for beginners
More stable and well-tested in various environments
Extensive documentation and community support

Cons of ORB_SLAM2

Limited to monocular, stereo, and RGB-D cameras
Lacks multi-map and multi-session capabilities
No IMU integration for improved tracking

Code Comparison

ORB_SLAM2:

void Tracking::Track()
{
    if(mState==NO_IMAGES_YET)
    {
        mState = NOT_INITIALIZED;
    }

ORB_SLAM3:

void Tracking::Track()
{
    if(mState==NO_IMAGES_YET)
    {
        mState = NOT_INITIALIZED;
    }

    if(mpAtlas->isInertial() && !mpAtlas->GetCurrentMap()->isImuInitialized())
    {
        if(!mbFirstImuFrame)
        {
            mCurrentFrame.SetNewBias(mLastFrame.GetImuBias());
            mbFirstImuFrame=true;
        }
    }

The code comparison shows that ORB_SLAM3 includes additional functionality for IMU integration, which is not present in ORB_SLAM2. This demonstrates the enhanced capabilities of ORB_SLAM3 in handling inertial data for improved tracking and mapping.

Kimera-VIO

1,527

Visual Inertial Odometry with SLAM capabilities and 3D Mesh generation.

Pros of Kimera-VIO

Integrates visual-inertial odometry (VIO) with mesh reconstruction for 3D mapping
Supports both stereo and RGB-D cameras, offering more flexibility in sensor choices
Provides a modular architecture, allowing easier integration of new components

Cons of Kimera-VIO

May have higher computational requirements due to additional mesh reconstruction
Less extensive testing and validation compared to ORB_SLAM3's long-standing development
Potentially more complex setup and configuration for new users

Code Comparison

ORB_SLAM3 (C++):

Frame::Frame(const cv::Mat &imLeft, const cv::Mat &imRight, const double &timeStamp, ORBextractor* extractorLeft, ORBextractor* extractorRight, ORBVocabulary* voc, cv::Mat &K, cv::Mat &distCoef, const float &bf, const float &thDepth)
    :mpORBvocabulary(voc),mpORBextractorLeft(extractorLeft),mpORBextractorRight(extractorRight), mTimeStamp(timeStamp), mK(K.clone()),mDistCoef(distCoef.clone()), mbf(bf), mThDepth(thDepth),
     mpReferenceKF(static_cast<KeyFrame*>(NULL))
{
    // Frame initialization
    mvKeys.reserve(N);
    mvKeysRight.reserve(N);
    mvpMapPoints = vector<MapPoint*>(N,static_cast<MapPoint*>(NULL));
}

Kimera-VIO (C++):

Frame::Frame(const cv::Mat& img,
             const cv::Mat& depth_img,
             const Timestamp& timestamp,
             const CameraParams& cam_params)
    : timestamp_(timestamp),
      cam_params_(cam_params),
      img_(img),
      depth_img_(depth_img) {
  CHECK(!img_.empty());
  extractFeatures();
}

Both repositories use C++ and OpenCV, but Kimera-VIO's code appears more concise and modern, utilizing C++11 features and CHECK macros for assertions.

VINS-Mono

4,926

A Robust and Versatile Monocular Visual-Inertial State Estimator

Pros of VINS-Mono

Designed specifically for aerial robotics applications
Integrates visual-inertial odometry for improved accuracy in dynamic environments
Supports loop closure and relocalization capabilities

Cons of VINS-Mono

Limited to monocular camera setups
May have higher computational requirements due to IMU integration
Less versatile for general-purpose SLAM applications

Code Comparison

VINS-Mono (C++):

void FeatureTracker::readImage(const cv::Mat &_img, double _cur_time)
{
    cv::Mat img;
    TicToc t_r;
    frame_cnt++;
    cv::remap(_img, img, undist_map1_, undist_map2_, CV_INTER_LINEAR);
    // ... (additional processing)
}

ORB_SLAM3 (C++):

void Frame::ExtractORB(int flag, const cv::Mat &im, const int x0, const int y0, const int x1, const int y1, const float fscale)
{
    vector<int> vLapping = {x0,y0,x1,y1};
    if(flag==0)
        monoLeft = (*mpORBextractorLeft)(im,cv::Mat(),mvKeys,mDescriptors,vLapping);
    else
        monoRight = (*mpORBextractorRight)(im,cv::Mat(),mvKeysRight,mDescriptorsRight,vLapping);
}

PythonRobotics

23,098

Python sample codes for robotics algorithms.

Pros of PythonRobotics

Written in Python, making it more accessible for beginners and easier to prototype
Covers a wide range of robotics algorithms, not just SLAM
Includes visualizations and examples for better understanding

Cons of PythonRobotics

Generally slower performance compared to C++ implementations
Less suitable for real-time applications in production environments
May lack some advanced features and optimizations present in specialized SLAM systems

Code Comparison

PythonRobotics (Python):

def update(self, z, R):
    y = z - self.H @ self.x
    S = self.H @ self.P @ self.H.T + R
    K = self.P @ self.H.T @ np.linalg.inv(S)
    self.x = self.x + K @ y
    self.P = (np.eye(len(self.x)) - K @ self.H) @ self.P

ORB_SLAM3 (C++):

void Frame::ComputeBoW()
{
    if(mBowVec.empty())
    {
        vector<cv::Mat> vCurrentDesc = Converter::toDescriptorVector(mDescriptors);
        mpORBvocabulary->transform(vCurrentDesc,mBowVec,mFeatVec,4);
    }
}

kalibr

4,351

The Kalibr visual-inertial calibration toolbox

Pros of kalibr

Specialized for sensor calibration, particularly for multi-sensor systems
Supports a wide range of sensor types, including cameras, IMUs, and LiDARs
Provides detailed calibration reports and visualization tools

Cons of kalibr

Focused solely on calibration, not a complete SLAM solution
May require more setup and configuration for specific sensor combinations
Less suitable for real-time applications compared to ORB_SLAM3

Code Comparison

kalibr (Python-based calibration script):

import kalibr_common as kc
import kalibr_camera_calibration as kcc

calibrator = kcc.CameraCalibrator()
calibrator.loadData(bag_file)
calibrator.calibrate()
calibrator.saveResults(output_dir)

ORB_SLAM3 (C++ main loop for monocular SLAM):

ORB_SLAM3::System SLAM(argv[1], argv[2], ORB_SLAM3::System::MONOCULAR, true);

while(1)
{
    if(imageGrabber.getNewFrame(im))
    {
        Sophus::SE3f Tcw = SLAM.TrackMonocular(im, imageGrabber.tframe);
    }
}

This comparison highlights the different focus areas of the two projects, with kalibr specializing in sensor calibration and ORB_SLAM3 providing a complete SLAM solution.

Convert designs to code with AI

Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.

Try Visual Copilot

README

ORB-SLAM3

V1.0, December 22th, 2021

Authors: Carlos Campos, Richard Elvira, Juan J. GÃ³mez RodrÃguez, JosÃ© M. M. Montiel, Juan D. Tardos.

The Changelog describes the features of each version.

ORB-SLAM3 is the first real-time SLAM library able to perform Visual, Visual-Inertial and Multi-Map SLAM with monocular, stereo and RGB-D cameras, using pin-hole and fisheye lens models. In all sensor configurations, ORB-SLAM3 is as robust as the best systems available in the literature, and significantly more accurate.

We provide examples to run ORB-SLAM3 in the EuRoC dataset using stereo or monocular, with or without IMU, and in the TUM-VI dataset using fisheye stereo or monocular, with or without IMU. Videos of some example executions can be found at ORB-SLAM3 channel.

This software is based on ORB-SLAM2 developed by Raul Mur-Artal, Juan D. Tardos, J. M. M. Montiel and Dorian Galvez-Lopez (DBoW2).

Related Publications:

[ORB-SLAM3] Carlos Campos, Richard Elvira, Juan J. GÃ³mez RodrÃguez, JosÃ© M. M. Montiel and Juan D. TardÃ³s, ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAM, IEEE Transactions on Robotics 37(6):1874-1890, Dec. 2021. PDF.

[IMU-Initialization] Carlos Campos, J. M. M. Montiel and Juan D. TardÃ³s, Inertial-Only Optimization for Visual-Inertial Initialization, ICRA 2020. PDF

[ORBSLAM-Atlas] Richard Elvira, J. M. M. Montiel and Juan D. TardÃ³s, ORBSLAM-Atlas: a robust and accurate multi-map system, IROS 2019. PDF.

[ORBSLAM-VI] RaÃºl Mur-Artal, and Juan D. TardÃ³s, Visual-inertial monocular SLAM with map reuse, IEEE Robotics and Automation Letters, vol. 2 no. 2, pp. 796-803, 2017. PDF.

[Stereo and RGB-D] RaÃºl Mur-Artal and Juan D. TardÃ³s. ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Cameras. IEEE Transactions on Robotics, vol. 33, no. 5, pp. 1255-1262, 2017. PDF.

[Monocular] RaÃºl Mur-Artal, JosÃ© M. M. Montiel and Juan D. TardÃ³s. ORB-SLAM: A Versatile and Accurate Monocular SLAM System. IEEE Transactions on Robotics, vol. 31, no. 5, pp. 1147-1163, 2015. (2015 IEEE Transactions on Robotics Best Paper Award). PDF.

[DBoW2 Place Recognition] Dorian GÃ¡lvez-LÃ³pez and Juan D. TardÃ³s. Bags of Binary Words for Fast Place Recognition in Image Sequences. IEEE Transactions on Robotics, vol. 28, no. 5, pp. 1188-1197, 2012. PDF

1. License

ORB-SLAM3 is released under GPLv3 license. For a list of all code/library dependencies (and associated licenses), please see Dependencies.md.

For a closed-source version of ORB-SLAM3 for commercial purposes, please contact the authors: orbslam (at) unizar (dot) es.

If you use ORB-SLAM3 in an academic work, please cite:

@article{ORBSLAM3_TRO,
  title={{ORB-SLAM3}: An Accurate Open-Source Library for Visual, Visual-Inertial 
           and Multi-Map {SLAM}},
  author={Campos, Carlos AND Elvira, Richard AND G\Â´omez, Juan J. AND Montiel, 
          Jos\'e M. M. AND Tard\'os, Juan D.},
  journal={IEEE Transactions on Robotics}, 
  volume={37},
  number={6},
  pages={1874-1890},
  year={2021}
 }

2. Prerequisites

We have tested the library in Ubuntu 16.04 and 18.04, but it should be easy to compile in other platforms. A powerful computer (e.g. i7) will ensure real-time performance and provide more stable and accurate results.

C++11 or C++0x Compiler

We use the new thread and chrono functionalities of C++11.

Pangolin

We use Pangolin for visualization and user interface. Dowload and install instructions can be found at: https://github.com/stevenlovegrove/Pangolin.

OpenCV

We use OpenCV to manipulate images and features. Dowload and install instructions can be found at: http://opencv.org. Required at leat 3.0. Tested with OpenCV 3.2.0 and 4.4.0.

Eigen3

Required by g2o (see below). Download and install instructions can be found at: http://eigen.tuxfamily.org. Required at least 3.1.0.

DBoW2 and g2o (Included in Thirdparty folder)

We use modified versions of the DBoW2 library to perform place recognition and g2o library to perform non-linear optimizations. Both modified libraries (which are BSD) are included in the Thirdparty folder.

Python

Required to calculate the alignment of the trajectory with the ground truth. Required Numpy module.

(win) http://www.python.org/downloads/windows
(deb) sudo apt install libpython2.7-dev
(mac) preinstalled with osx

ROS (optional)

We provide some examples to process input of a monocular, monocular-inertial, stereo, stereo-inertial or RGB-D camera using ROS. Building these examples is optional. These have been tested with ROS Melodic under Ubuntu 18.04.

3. Building ORB-SLAM3 library and examples

Clone the repository:

git clone https://github.com/UZ-SLAMLab/ORB_SLAM3.git ORB_SLAM3

We provide a script build.sh to build the Thirdparty libraries and ORB-SLAM3. Please make sure you have installed all required dependencies (see section 2). Execute:

cd ORB_SLAM3
chmod +x build.sh
./build.sh

This will create libORB_SLAM3.so at lib folder and the executables in Examples folder.

4. Running ORB-SLAM3 with your camera

Directory Examples contains several demo programs and calibration files to run ORB-SLAM3 in all sensor configurations with Intel Realsense cameras T265 and D435i. The steps needed to use your own camera are:

Calibrate your camera following Calibration_Tutorial.pdf and write your calibration file your_camera.yaml
Modify one of the provided demos to suit your specific camera model, and build it
Connect the camera to your computer using USB3 or the appropriate interface
Run ORB-SLAM3. For example, for our D435i camera, we would execute:

./Examples/Stereo-Inertial/stereo_inertial_realsense_D435i Vocabulary/ORBvoc.txt ./Examples/Stereo-Inertial/RealSense_D435i.yaml

5. EuRoC Examples

EuRoC dataset was recorded with two pinhole cameras and an inertial sensor. We provide an example script to launch EuRoC sequences in all the sensor configurations.

Download a sequence (ASL format) from http://projects.asl.ethz.ch/datasets/doku.php?id=kmavvisualinertialdatasets
Open the script "euroc_examples.sh" in the root of the project. Change pathDatasetEuroc variable to point to the directory where the dataset has been uncompressed.
Execute the following script to process all the sequences with all sensor configurations:

./euroc_examples

Evaluation

EuRoC provides ground truth for each sequence in the IMU body reference. As pure visual executions report trajectories centered in the left camera, we provide in the "evaluation" folder the transformation of the ground truth to the left camera reference. Visual-inertial trajectories use the ground truth from the dataset.

Execute the following script to process sequences and compute the RMS ATE:

./euroc_eval_examples

6. TUM-VI Examples

TUM-VI dataset was recorded with two fisheye cameras and an inertial sensor.

Download a sequence from https://vision.in.tum.de/data/datasets/visual-inertial-dataset and uncompress it.
Open the script "tum_vi_examples.sh" in the root of the project. Change pathDatasetTUM_VI variable to point to the directory where the dataset has been uncompressed.
Execute the following script to process all the sequences with all sensor configurations:

./tum_vi_examples

Evaluation

In TUM-VI ground truth is only available in the room where all sequences start and end. As a result the error measures the drift at the end of the sequence.

Execute the following script to process sequences and compute the RMS ATE:

./tum_vi_eval_examples

7. ROS Examples

Building the nodes for mono, mono-inertial, stereo, stereo-inertial and RGB-D

Tested with ROS Melodic and ubuntu 18.04.

Add the path including Examples/ROS/ORB_SLAM3 to the ROS_PACKAGE_PATH environment variable. Open .bashrc file:

gedit ~/.bashrc

and add at the end the following line. Replace PATH by the folder where you cloned ORB_SLAM3:

export ROS_PACKAGE_PATH=${ROS_PACKAGE_PATH}:PATH/ORB_SLAM3/Examples/ROS

Execute build_ros.sh script:

chmod +x build_ros.sh
./build_ros.sh

Running Monocular Node

For a monocular input from topic /camera/image_raw run node ORB_SLAM3/Mono. You will need to provide the vocabulary file and a settings file. See the monocular examples above.

rosrun ORB_SLAM3 Mono PATH_TO_VOCABULARY PATH_TO_SETTINGS_FILE

Running Monocular-Inertial Node

For a monocular input from topic /camera/image_raw and an inertial input from topic /imu, run node ORB_SLAM3/Mono_Inertial. Setting the optional third argument to true will apply CLAHE equalization to images (Mainly for TUM-VI dataset).

rosrun ORB_SLAM3 Mono PATH_TO_VOCABULARY PATH_TO_SETTINGS_FILE [EQUALIZATION]

Running Stereo Node

For a stereo input from topic /camera/left/image_raw and /camera/right/image_raw run node ORB_SLAM3/Stereo. You will need to provide the vocabulary file and a settings file. For Pinhole camera model, if you provide rectification matrices (see Examples/Stereo/EuRoC.yaml example), the node will recitify the images online, otherwise images must be pre-rectified. For FishEye camera model, rectification is not required since system works with original images:

rosrun ORB_SLAM3 Stereo PATH_TO_VOCABULARY PATH_TO_SETTINGS_FILE ONLINE_RECTIFICATION

Running Stereo-Inertial Node

For a stereo input from topics /camera/left/image_raw and /camera/right/image_raw, and an inertial input from topic /imu, run node ORB_SLAM3/Stereo_Inertial. You will need to provide the vocabulary file and a settings file, including rectification matrices if required in a similar way to Stereo case:

rosrun ORB_SLAM3 Stereo_Inertial PATH_TO_VOCABULARY PATH_TO_SETTINGS_FILE ONLINE_RECTIFICATION [EQUALIZATION]

Running RGB_D Node

For an RGB-D input from topics /camera/rgb/image_raw and /camera/depth_registered/image_raw, run node ORB_SLAM3/RGBD. You will need to provide the vocabulary file and a settings file. See the RGB-D example above.

rosrun ORB_SLAM3 RGBD PATH_TO_VOCABULARY PATH_TO_SETTINGS_FILE

Running ROS example: Download a rosbag (e.g. V1_02_medium.bag) from the EuRoC dataset (http://projects.asl.ethz.ch/datasets/doku.php?id=kmavvisualinertialdatasets). Open 3 tabs on the terminal and run the following command at each tab for a Stereo-Inertial configuration:

roscore

rosrun ORB_SLAM3 Stereo_Inertial Vocabulary/ORBvoc.txt Examples/Stereo-Inertial/EuRoC.yaml true

rosbag play --pause V1_02_medium.bag /cam0/image_raw:=/camera/left/image_raw /cam1/image_raw:=/camera/right/image_raw /imu0:=/imu

Once ORB-SLAM3 has loaded the vocabulary, press space in the rosbag tab.

Remark: For rosbags from TUM-VI dataset, some play issue may appear due to chunk size. One possible solution is to rebag them with the default chunk size, for example:

rosrun rosbag fastrebag.py dataset-room1_512_16.bag dataset-room1_512_16_small_chunks.bag

8. Running time analysis

A flag in include\Config.h activates time measurements. It is necessary to uncomment the line #define REGISTER_TIMES to obtain the time stats of one execution which is shown at the terminal and stored in a text file(ExecTimeMean.txt).

9. Calibration

You can find a tutorial for visual-inertial calibration and a detailed description of the contents of valid configuration files at Calibration_Tutorial.pdf

Top Related Projects

Convert designs to code with AI

Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.

Try Visual Copilot