Top Related Projects
Hello AI World guide to deploying deep-learning inference networks and deep vision primitives with TensorRT and NVIDIA Jetson.
NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT.
Build and run Docker containers leveraging NVIDIA GPUs
A GPU-accelerated library containing highly optimized building blocks and an execution engine for data processing to accelerate deep learning training and inference applications.
Quick Overview
NVIDIA-AI-IOT/redtail is an open-source project focused on autonomous drone navigation using deep learning. It provides a complete system for training and deploying neural networks for visual navigation, including datasets, training scripts, and inference code for NVIDIA Jetson platforms.
Pros
- Comprehensive end-to-end solution for autonomous drone navigation
- Optimized for NVIDIA Jetson platforms, leveraging GPU acceleration
- Includes pre-trained models and datasets for quick start and benchmarking
- Supports both simulated and real-world environments for testing and deployment
Cons
- Primarily focused on NVIDIA hardware, limiting compatibility with other platforms
- Requires significant computational resources for training and inference
- May have a steep learning curve for users new to deep learning or robotics
- Documentation could be more extensive for some components
Code Examples
- Loading a pre-trained model:
from redtail import TrailNet
model = TrailNet.load_model('path/to/pretrained_model.pth')
- Performing inference on an image:
import cv2
from redtail import preprocess_image
image = cv2.imread('path/to/image.jpg')
preprocessed = preprocess_image(image)
prediction = model.predict(preprocessed)
- Training a new model:
from redtail import TrailNetTrainer
trainer = TrailNetTrainer(
data_path='path/to/dataset',
model_type='resnet18',
batch_size=32,
num_epochs=100
)
trainer.train()
Getting Started
-
Clone the repository:
git clone https://github.com/NVIDIA-AI-IOT/redtail.git cd redtail
-
Install dependencies:
pip install -r requirements.txt
-
Download pre-trained models:
./scripts/download_models.sh
-
Run inference on a sample image:
from redtail import TrailNet, preprocess_image import cv2 model = TrailNet.load_model('models/trailnet_resnet18.pth') image = cv2.imread('samples/trail_image.jpg') preprocessed = preprocess_image(image) prediction = model.predict(preprocessed) print(f"Predicted direction: {prediction}")
Competitor Comparisons
Hello AI World guide to deploying deep-learning inference networks and deep vision primitives with TensorRT and NVIDIA Jetson.
Pros of jetson-inference
- More comprehensive and up-to-date documentation
- Wider range of pre-trained models and examples
- Better integration with NVIDIA Jetson platforms
Cons of jetson-inference
- Less focused on specific drone/UAV applications
- May require more setup and configuration for custom use cases
Code Comparison
jetson-inference:
import jetson.inference
import jetson.utils
net = jetson.inference.detectNet("ssd-mobilenet-v2")
camera = jetson.utils.videoSource("csi://0")
display = jetson.utils.videoOutput("display://0")
redtail:
from redtail.models import TrailNet
from redtail.utils import Camera
model = TrailNet.load_model("trail_following_model.pth")
camera = Camera()
The jetson-inference code demonstrates easier integration with Jetson-specific libraries, while redtail focuses on drone-specific functionality. jetson-inference provides a more general-purpose approach to AI inference, whereas redtail is tailored for autonomous drone navigation tasks.
NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT.
Pros of TensorRT
- Optimized for NVIDIA GPUs, providing faster inference
- Supports a wide range of deep learning frameworks
- Extensive documentation and community support
Cons of TensorRT
- Limited to NVIDIA hardware
- Steeper learning curve for beginners
- May require model conversion for some frameworks
Code Comparison
TensorRT:
import tensorrt as trt
logger = trt.Logger(trt.Logger.WARNING)
builder = trt.Builder(logger)
network = builder.create_network()
parser = trt.OnnxParser(network, logger)
Redtail:
from caffe.proto import caffe_pb2
import caffe
net = caffe.Net('deploy.prototxt', 'model.caffemodel', caffe.TEST)
Summary
TensorRT is a powerful inference optimization library for NVIDIA GPUs, supporting various deep learning frameworks. It offers excellent performance but is limited to NVIDIA hardware and may have a steeper learning curve.
Redtail, on the other hand, is focused on autonomous drone navigation using deep learning. It provides a more specialized solution for drone AI but may not have the same broad applicability as TensorRT.
The code comparison shows TensorRT's focus on optimizing models for inference, while Redtail's example demonstrates its use of Caffe for neural network deployment in drone applications.
Build and run Docker containers leveraging NVIDIA GPUs
Pros of nvidia-docker
- Broader application: Enables GPU acceleration for any Docker container
- Easier integration: Seamlessly works with existing Docker workflows
- Active development: Regularly updated and maintained by NVIDIA
Cons of nvidia-docker
- Limited scope: Focused solely on Docker containerization
- Less specialized: Not tailored for specific AI or IoT applications
- Steeper learning curve: Requires understanding of both Docker and NVIDIA GPU technologies
Code Comparison
redtail (Python):
def detect_objects(image):
# Object detection code for drones
pass
nvidia-docker (Shell):
docker run --gpus all nvidia/cuda:11.0-base nvidia-smi
Summary
While redtail is specifically designed for AI-powered autonomous drone navigation, nvidia-docker provides a more general solution for GPU acceleration in containerized environments. redtail offers specialized functionality for drone applications, but nvidia-docker's versatility makes it applicable to a wider range of GPU-accelerated tasks. The choice between the two depends on the specific requirements of your project.
A GPU-accelerated library containing highly optimized building blocks and an execution engine for data processing to accelerate deep learning training and inference applications.
Pros of DALI
- Focuses on data loading and preprocessing for deep learning
- Optimized for GPU acceleration, improving performance
- Supports a wide range of data formats and augmentation techniques
Cons of DALI
- Steeper learning curve compared to Redtail
- More complex setup and integration process
- Limited to data preprocessing, not a complete autonomous drone solution
Code Comparison
DALI Example:
import nvidia.dali as dali
@dali.pipeline_def
def image_pipeline():
images, labels = dali.fn.readers.file(file_root=data_dir, random_shuffle=True)
images = dali.fn.decoders.image(images, device="mixed")
images = dali.fn.resize(images, resize_x=224, resize_y=224)
return images, labels
Redtail Example:
from redtail.sensors import Camera
from redtail.control import PIDController
camera = Camera()
controller = PIDController()
while True:
image = camera.capture()
control_output = controller.compute(image)
Summary
DALI is a specialized library for data loading and preprocessing, optimized for GPU acceleration and supporting various data formats. Redtail, on the other hand, is a complete solution for autonomous drone navigation, including sensor integration and control algorithms. While DALI offers better performance for data preprocessing, Redtail provides a more comprehensive toolkit for drone autonomy.
Convert designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual CopilotREADME
NVIDIA Redtail project
Autonomous visual navigation components for drones and ground vehicles using deep learning. Refer to wiki for more information on how to get started.
This project contains deep neural networks, computer vision and control code, hardware instructions and other artifacts that allow users to build a drone or a ground vehicle which can autonomously navigate through highly unstructured environments like forest trails, sidewalks, etc. Our TrailNet DNN for visual navigation is running on NVIDIA's Jetson embedded platform. Our arXiv paper describes TrailNet and other runtime modules in detail.
The project's deep neural networks (DNNs) can be trained from scratch using publicly available data. A few pre-trained DNNs are also available as a part of this project. In case you want to train TrailNet DNN from scratch, follow the steps on this page.
The project also contains Stereo DNN models and runtime which allow to estimate depth from stereo camera on NVIDIA platforms.
IROS 2018: we presented our work at IROS 2018 conference as a part of Vision-based Drones: What's Next? workshop.
CVPR 2018: we presented our work at CVPR 2018 conference as a part of Workshop on Autonomous Driving.
References and Demos
- Stereo DNN, CVPR18 paper, Stereo DNN video demo
- TrailNet Forest Drone Navigation, IROS17 paper, TrailNet DNN video demo
- GTC 2017 talk: slides, video
- Demo video showing 250m autonomous flight with TrailNet DNN flying the drone
- Demo video showing our 1 kilometer autonomous drone flight with TrailNet DNN
- Demo video showing TrailNet DNN driving a robotic rover around the office
- Demo video showing TrailNet generalization to ground vehicles and other environments
News
-
2020-02-03: Alternative implementations. redtail is no longer being developed, but fortunately our community stepped in and continued developing the project. We thank our users for the interest in redtail, questions and feedback!
Some alternative implementations are listed below.
- @mtbsteve: https://github.com/mtbsteve/redtail
-
2018-10-10: Stereo DNN ROS node and fixes.
- Added Stereo DNN ROS node and visualizer node.
- Fixed issue with nvidia-docker v2.
-
2018-09-19: Updates to Stereo DNN.
- Moved to TensorRT 4.0
- Enabled FP16 support in
ResNet18 2D
model, resulting in 2x performance increase (20fps on Jetson TX2). - Enabled TensorRT serialization in
ResNet18 2D
model to reduce model loading time from minutes to less than a second. - Better logging and profiler support.
-
2018-06-04: CVPR 2018 workshop. Fast version of Stereo DNN.
- Presenting our work at CVPR 2018 conference as a part of Workshop on Autonomous Driving.
- Added fast version of Stereo DNN model based on ResNet18 2D model. The model runs at 10fps on Jetson TX2. See README for details and check out updated sample_app.
-
GTC 2018: Here is our Stereo DNN session page at GTC18 and the recorded video presentation
-
2018-03-22: redtail 2.0.
- Added Stereo DNN models and inference library (TensorFlow/TensorRT). For more details, see the README.
- Migrated to JetPack 3.2. This change brings latest components such as CUDA 9.0, cuDNN 7.0, TensorRT 3.0, OpenCV 3.3 and others to Jetson platform. Note that this is a breaking change.
- Added support for INT8 inference. This enables fast inference on devices that have hardware implementation of INT8 instructions. More details are on our wiki.
-
2018-02-15: added support for the TBS Discovery platform.
- Step by step instructions on how to assemble the TBS Discovery drone.
- Instructions on how to attach and use a ZED stereo camera.
- Detailed instructions on how to calibrate, test and fly the drone.
-
2017-10-12: added full simulation Docker image, experimental support for APM Rover and support for MAVROS v0.21+.
- Redtail simulation Docker image contains all the components required to run full Redtail simulation in Docker. Refer to wiki for more information.
- Experimental support for APM Rover. Refer to wiki for more information.
- Several other changes including support for MAVROS v0.21+, updated Jetson install script and few bug fixes.
-
2017-09-07: NVIDIA Redtail project is released as an open source project.
Redtail's AI modules allow building autonomous drones and mobile robots based on Deep Learning and NVIDIA Jetson TX1 and TX2 embedded systems. Source code, pre-trained models as well as detailed build and test instructions are released on GitHub.
-
2017-07-26: migrated code and scripts to JetPack 3.1 with TensorRT 2.1.
TensorRT 2.1 provides significant improvements in DNN inference performance as well as new features and bug fixes. This is a breaking change which requires re-flashing Jetson with JetPack 3.1.
Top Related Projects
Hello AI World guide to deploying deep-learning inference networks and deep vision primitives with TensorRT and NVIDIA Jetson.
NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT.
Build and run Docker containers leveraging NVIDIA GPUs
A GPU-accelerated library containing highly optimized building blocks and an execution engine for data processing to accelerate deep learning training and inference applications.
Convert designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual Copilot