Top Related Projects
Autoware - the world's leading open-source software project for autonomous driving
An open autonomous driving platform
openpilot is an operating system for robotics. Currently, it upgrades the driver assistance system on 300+ supported cars.
Detectron2 is a platform for object detection, segmentation and other visual recognition tasks.
Quick Overview
The Waymo Open Dataset repository contains tools, tutorials, and documentation for working with Waymo's autonomous driving dataset. It provides resources for researchers and developers to access and analyze large-scale, high-quality data collected from Waymo's self-driving vehicles, including LiDAR and camera data, object labels, and more.
Pros
- Offers access to a large, diverse, and high-quality autonomous driving dataset
- Provides comprehensive tools and tutorials for data processing and analysis
- Supports multiple programming languages (Python, C++)
- Regularly updated with new features and improvements
Cons
- Requires significant computational resources to process large datasets
- Learning curve for understanding the data format and tools
- Limited to Waymo's specific data format and scenarios
- May require additional steps for integration with other autonomous driving frameworks
Code Examples
- Loading and visualizing LiDAR data:
import tensorflow as tf
import math
import numpy as np
import itertools
import matplotlib.pyplot as plt
from waymo_open_dataset import dataset_pb2
from waymo_open_dataset import label_pb2
from waymo_open_dataset.utils import frame_utils
def visualize_lidar(frame):
range_images, camera_projections, range_image_top_pose = frame_utils.parse_range_image_and_camera_projection(frame)
points, cp_points = frame_utils.convert_range_image_to_point_cloud(
frame,
range_images,
camera_projections,
range_image_top_pose)
# Visualize the point cloud
fig = plt.figure(figsize=(20, 12))
ax = fig.add_subplot(111, projection='3d')
ax.scatter(points[0][:, 0], points[0][:, 1], points[0][:, 2], s=0.1)
ax.set_xlabel('x')
ax.set_ylabel('y')
ax.set_zlabel('z')
plt.show()
# Load a frame and visualize its LiDAR data
dataset = tf.data.TFRecordDataset('path/to/segment.tfrecord')
for data in dataset:
frame = dataset_pb2.Frame()
frame.ParseFromString(data.numpy())
visualize_lidar(frame)
break
- Accessing object labels:
from waymo_open_dataset.utils import transform_utils
def print_object_labels(frame):
for label in frame.laser_labels:
print(f"Type: {label.type}")
print(f"ID: {label.id}")
print(f"Box: {label.box}")
print(f"Metadata: {label.metadata}")
print("---")
# Load a frame and print its object labels
dataset = tf.data.TFRecordDataset('path/to/segment.tfrecord')
for data in dataset:
frame = dataset_pb2.Frame()
frame.ParseFromString(data.numpy())
print_object_labels(frame)
break
- Computing 3D IoU (Intersection over Union):
from waymo_open_dataset import label_pb2
from waymo_open_dataset.protos import metrics_pb2
def compute_3d_iou(box1, box2):
iou = metrics_pb2.Objects()
iou.objects.append(box1)
iou.objects.append(box2)
iou_result = metrics_pb2.ComputeDetectionMetrics.iou(iou)
return iou_result[0]
# Example usage
box1 = label_pb2.Label.Box()
box2 = label_pb2.Label.Box()
# Set box parameters...
iou = compute_3d_iou(box1, box2)
print(f"3D IoU: {iou}")
Getting Started
- Install the Waymo Open Dataset package:
pip install waymo-open-dataset-tf-2-6-0==1.4.3
Competitor Comparisons
Autoware - the world's leading open-source software project for autonomous driving
Pros of Autoware
- Comprehensive open-source autonomous driving software stack
- Active community and regular updates
- Supports various sensors and platforms
Cons of Autoware
- Steeper learning curve due to complexity
- May require more computational resources
- Limited pre-trained models compared to Waymo Open Dataset
Code Comparison
Waymo Open Dataset (Python):
import tensorflow as tf
import math
import numpy as np
import itertools
from waymo_open_dataset import dataset_pb2 as open_dataset
Autoware (C++):
#include <ros/ros.h>
#include <sensor_msgs/PointCloud2.h>
#include <pcl_ros/point_cloud.h>
#include <pcl/point_types.h>
#include <autoware_msgs/DetectedObjectArray.h>
The Waymo Open Dataset focuses on providing a large-scale dataset for autonomous driving research, while Autoware offers a complete software stack for autonomous vehicle development. Waymo's repository primarily contains tools for data handling and evaluation, whereas Autoware includes modules for perception, planning, and control. The code snippets reflect this difference, with Waymo's code centered around dataset manipulation and Autoware's code demonstrating ROS integration and sensor data processing.
An open autonomous driving platform
Pros of Apollo
- Comprehensive autonomous driving platform with full-stack capabilities
- Active development and regular updates from a large community
- Extensive documentation and tutorials for implementation
Cons of Apollo
- Steeper learning curve due to complex architecture
- Requires more computational resources for deployment
- Limited to specific hardware configurations
Code Comparison
Apollo (C++):
void Planning::RunOnce(const LocalView& local_view,
ADCTrajectory* const trajectory_pb) {
// Planning logic implementation
}
Waymo Open Dataset (Python):
def process_example(example):
# Data processing and feature extraction
return processed_features
Key Differences
- Apollo focuses on end-to-end autonomous driving solutions, while Waymo Open Dataset primarily provides data and tools for research
- Apollo offers a complete software stack, whereas Waymo Open Dataset is centered around dataset access and evaluation metrics
- Apollo uses C++ as its primary language, while Waymo Open Dataset utilizes Python for data processing and analysis
Use Cases
- Apollo: Developing and testing full autonomous driving systems
- Waymo Open Dataset: Research, benchmarking, and algorithm development for perception and prediction tasks
openpilot is an operating system for robotics. Currently, it upgrades the driver assistance system on 300+ supported cars.
Pros of openpilot
- Open-source and actively maintained by a community of developers
- Designed for real-world deployment on consumer vehicles
- Supports a wide range of car models and can be installed by end-users
Cons of openpilot
- Limited to specific hardware configurations (e.g., comma devices)
- Smaller dataset compared to Waymo Open Dataset
- May have less rigorous safety testing and validation processes
Code Comparison
openpilot:
def update(self):
if self.sm.updated['carState']:
self.v_cruise_kph = self.sm['carState'].cruiseState.speed * CV.MS_TO_KPH
self.v_cruise_cluster_kph = self.sm['carState'].cruiseState.speedCluster * CV.MS_TO_KPH
Waymo Open Dataset:
def parse_range_image_and_camera_projection(range_image, camera_projection):
return range_image_utils.parse_range_image_and_camera_projection(
range_image, camera_projection)
The code snippets show different focuses: openpilot deals with real-time vehicle control, while Waymo Open Dataset processes sensor data for research and development purposes.
Detectron2 is a platform for object detection, segmentation and other visual recognition tasks.
Pros of Detectron2
- Comprehensive library for object detection and segmentation tasks
- Modular design allows easy customization and extension
- Extensive documentation and community support
Cons of Detectron2
- Focused primarily on computer vision tasks, less versatile for other domains
- Steeper learning curve for beginners compared to simpler datasets
Code Comparison
Detectron2 (model configuration):
cfg = get_cfg()
cfg.merge_from_file(model_zoo.get_config_file("COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml"))
cfg.MODEL.ROI_HEADS.SCORE_THRESH_TEST = 0.5
cfg.MODEL.WEIGHTS = model_zoo.get_checkpoint_url("COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml")
Waymo Open Dataset (data loading):
dataset = tf.data.TFRecordDataset(FILENAME, compression_type='')
for data in dataset:
frame = open_dataset.Frame()
frame.ParseFromString(bytearray(data.numpy()))
The Waymo Open Dataset provides a large-scale dataset for autonomous driving research, while Detectron2 offers a powerful framework for implementing and experimenting with various computer vision models. Detectron2 is more suitable for researchers and developers working on general object detection and segmentation tasks, while the Waymo Open Dataset is specifically tailored for autonomous driving applications.
Convert
designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual CopilotREADME
Waymo Open Dataset
The Waymo Open Dataset is a collection of datasets and evaluation code that we have released publicly to aid the research community in making advancements in machine perception and autonomous driving technology.
The Waymo Open Dataset includes three datasets:
- The Perception dataset, with high resolution sensor data and labels for various tasks.
- The Motion dataset, with object trajectories and corresponding 3D maps for 103,354 scenes.
- The End-To-End Driving dataset, with camera data and high-level commands.
In this codebase, we provide evaluation code to support several tasks based on these three dataset.
See release history for a detailed list of changes.
More information can be found on the Waymo Open Dataset website.
License
This code repository (excluding
src/waymo_open_dataset/wdl_limited
folder) is licensed under the Apache License, Version 2.0. The code appearing in
src/waymo_open_dataset/wdl_limited
is
licensed under terms appearing therein. The Waymo Open Dataset itself is
licensed under separate terms. Please
visit https://waymo.com/open/terms/ for
details. Code located in each of the subfolders located at
src/waymo_open_dataset/wdl_limited
is
licensed under (a) a BSD 3-clause copyright license and (b) an additional
limited patent license. Each limited
patent license is applicable only to code under the respective wdl_limited
subfolder, and is licensed for use only with the use case laid out in such
license in connection with the Waymo Open Dataset, as authorized by and in
compliance with the Waymo Dataset License Agreement for Non-Commercial Use. See
wdl_limited/camera/,
wdl_limited/camera_segmentation/,
wdl_limited/sim_agents_metrics/,
respectively, for details.
Website
To read more about the dataset and access it, please visit https://www.waymo.com/open.
Contents
This code repository contains:
- Definition of the dataset format
- Evaluation metrics
- Helper functions in TensorFlow to help with building models
Citation
for Perception dataset
@InProceedings{Sun_2020_CVPR, author = {Sun, Pei and Kretzschmar, Henrik and Dotiwalla, Xerxes and Chouard, Aurelien and Patnaik, Vijaysai and Tsui, Paul and Guo, James and Zhou, Yin and Chai, Yuning and Caine, Benjamin and Vasudevan, Vijay and Han, Wei and Ngiam, Jiquan and Zhao, Hang and Timofeev, Aleksei and Ettinger, Scott and Krivokon, Maxim and Gao, Amy and Joshi, Aditya and Zhang, Yu and Shlens, Jonathon and Chen, Zhifeng and Anguelov, Dragomir}, title = {Scalability in Perception for Autonomous Driving: Waymo Open Dataset}, booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)}, month = {June}, year = {2020} }
for Motion dataset
@InProceedings{Ettinger_2021_ICCV, author={Ettinger, Scott and Cheng, Shuyang and Caine, Benjamin and Liu, Chenxi and Zhao, Hang and Pradhan, Sabeek and Chai, Yuning and Sapp, Ben and Qi, Charles R. and Zhou, Yin and Yang, Zoey and Chouard, Aur'elien and Sun, Pei and Ngiam, Jiquan and Vasudevan, Vijay and McCauley, Alexander and Shlens, Jonathon and Anguelov, Dragomir}, title={Large Scale Interactive Motion Forecasting for Autonomous Driving: The Waymo Open Motion Dataset}, booktitle= Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)}, month={October}, year={2021}, pages={9710-9719} }
@InProceedings{Kan_2024_icra, author={Chen, Kan and Ge, Runzhou and Qiu, Hang and Ai-Rfou, Rami and Qi, Charles R. and Zhou, Xuanyu and Yang, Zoey and Ettinger, Scott and Sun, Pei and Leng, Zhaoqi and Mustafa, Mustafa and Bogun, Ivan and Wang, Weiyue and Tan, Mingxing and Anguelov, Dragomir}, title={WOMD-LiDAR: Raw Sensor Dataset Benchmark for Motion Forecasting}, month={May}, booktitle= Proceedings of the IEEE International Conference on Robotics and Automation (ICRA)}, year={2024} }
Dataset Metadata
The following table is necessary for this dataset to be indexed by search engines such as Google Dataset Search.
property | value | ||||||
---|---|---|---|---|---|---|---|
name | Waymo Open Dataset: An autonomous driving dataset |
||||||
alternateName | Waymo Open Dataset |
||||||
url | https://github.com/waymo-research/waymo-open-dataset |
||||||
sameAs | https://github.com/waymo-research/waymo-open-dataset |
||||||
sameAs | https://www.waymo.com/open |
||||||
description | The Waymo Open Dataset is comprised of high-resolution sensor data collected by autonomous vehicles operated by the Waymo Driver in a wide variety of conditions. Weâre releasing this dataset publicly to aid the research community in making advancements in machine perception and self-driving technology. |
||||||
provider |
|
||||||
license |
|
Top Related Projects
Autoware - the world's leading open-source software project for autonomous driving
An open autonomous driving platform
openpilot is an operating system for robotics. Currently, it upgrades the driver assistance system on 300+ supported cars.
Detectron2 is a platform for object detection, segmentation and other visual recognition tasks.
Convert
designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual Copilot