Point Density-Aware Voxels for LiDAR 3D Object Detection (CVPR 2022)

PDV

PDV is LiDAR 3D object detection method. This repository is based off [OpenPCDet].

Point Density-Aware Voxels for LiDAR 3D Object Detection
Jordan S. K. Hu, Tianshu Kuai, Steven L. Waslander
[Paper]

PDV diagram

Overview

Changelog

[2022-03-07] PDV v0.1.0 is released.

Model Zoo

KITTI 3D Object Detection Baselines

Selected supported methods are shown in the below table. The results are the 3D detection performance of moderate difficulty on the val set of KITTI dataset.

  • All models are trained with 2 NVIDIA Tesla P100 GPUs and are available for download.
  • The training time is measured with 2 NVIDIA Tesla P100 GPUs and PyTorch 1.7.
training time [email protected] [email protected] [email protected] download
PDV ~23 hours 85.05 57.41 75.95 model-147M

Waymo Open Dataset Baselines

We could not provide the above pretrained models due to Waymo Dataset License Agreement, but you could easily achieve similar performance by training with the default configs. PDV is trained with 10% data (~16k frames) on 4 NVIDIA Tesla V100s GPUs.

Installation

Please refer to INSTALL.md for the installation of OpenPCDet.

Quick Demo

Please refer to DEMO.md for a quick demo to test with a pretrained model and visualize the predicted results on your custom data or the original KITTI data.

Getting Started

Please refer to GETTING_STARTED.md to learn more usage about this project.

License

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

Acknowledgement

We would like to thank the authors of OpenPCDet for their open source release of their codebase.

Citation

If you find this project useful in your research, please consider citing:

@article{PDV,
    title={Point Density-Aware Voxels for LiDAR 3D Object Detection},
    author={Jordan S. K. Hu and
            Tianshu Kuai and
            Steven L. Waslander},
    journal={CVPR},
    year={2022}
}
Owner
Toronto Robotics and AI Laboratory
Toronto Robotics and AI Laboratory
Comments
  • codes running issues

    codes running issues

    Hi, I tried to tun your source code,but the following error occurred:“AttributeError: 'Tensor' object has no attribute 'isnan” The wrong sentence is “points_out_of_range = ((xyz_local_grid < 0) | (xyz_local_grid >= grid_size) | (xyz_local_grid.isnan())) .any(-1).flatten()”in the folder “pcdet/utils/density_utils.py".

    What is the cause and how to improve it?

  • The referenced results of PVRCNN(++)

    The referenced results of PVRCNN(++)

    Hi, Nice work! May I ask where the results of PVRCNN and PVRCNN++ come from? The numbers look a little bit strange to me since I can not find them in the original papers of PVRCNN(++).

  • What should I do if I want to train this model on my own dataset?

    What should I do if I want to train this model on my own dataset?

    I notice that for each dataset, info files (.pkl) are generated. Do you have any rules about generating these info files? Because I want to train this model on my own dataset. Thanks a lot.

  • what is the role of MAX_NUM_BOXES?

    what is the role of MAX_NUM_BOXES?

    Thanks for your work, I would like to know what is the role of MAX_NUM_BOXES, is it because the RoIs may have overlap, causing a point to fall into different RoI?

  • GLEW could not be initialized: Missing GL version

    GLEW could not be initialized: Missing GL version

    Hello, i've met a problem in running demo.py 2022-04-04 14:55:26.887 ( 15.100s) [ F0A55E80]vtkOpenGLRenderWindow.c:493 ERR| vtkEGLRenderWindow (0x561980223bb0): GLEW could not be initialized: Missing GL version run-demo.sh: line 19: 7579 Segmentation fault (core dumped) I've tried several approaches but all of them failed. Do you have any idea about this bug?

Related tags
Small Robot, with LIDAR and DepthCamera. Using ROS for Maping and Navigation
Small Robot, with LIDAR and DepthCamera. Using ROS for Maping and Navigation

?? RoboCop ?? Small Robot, with LIDAR and DepthCamera. Using ROS for Maping and Navigation Made by Clemente Donoso, ?? Chile ???? RoboCop Lateral Fron

Jan 4, 2022
Programming of Robotics Systems course at the University of Aveiro, Portugal, 2021-2022.
Programming of Robotics Systems course at the University of Aveiro, Portugal, 2021-2022.

Programação de Sistemas Robóticos Miguel Riem Oliveira Universidade de Aveiro 2021-2022 Projeto AtlasCar Projecto RACE IROS 2014 AtlasCar2 ATOM IROS 2

Apr 8, 2022
Get the AltAz coordinates for a given object using astropy and output on a OLED screen.
Get the AltAz coordinates for a given object using astropy and output on a OLED screen.

Star Coordinates Get the AltAz coordinates for a given object using astropy and output on a OLED screen. As a very very newcomer to the astronomy scen

Jan 31, 2022
Official code of the paper "Expanding Low-Density Latent Regions for Open-Set Object Detection" (CVPR 2022)
Official code of the paper

OpenDet Expanding Low-Density Latent Regions for Open-Set Object Detection (CVPR2022) Jiaming Han, Yuqiang Ren, Jian Ding, Xingjia Pan, Ke Yan, Gui-So

May 20, 2022
ImVoxelNet: Image to Voxels Projection for Monocular and Multi-View General-Purpose 3D Object Detection
ImVoxelNet: Image to Voxels Projection for Monocular and Multi-View General-Purpose 3D Object Detection

ImVoxelNet: Image to Voxels Projection for Monocular and Multi-View General-Purpose 3D Object Detection This repository contains implementation of the

May 16, 2022
BADet: Boundary-Aware 3D Object Detection from Point Clouds (Pattern Recognition 2022)
BADet: Boundary-Aware 3D Object Detection from Point Clouds (Pattern Recognition 2022)

BADet: Boundary-Aware 3D Object Detection from Point Clouds (Pattern Recognition

Apr 21, 2022
Fog Simulation on Real LiDAR Point Clouds for 3D Object Detection in Adverse Weather
 Fog Simulation on Real LiDAR Point Clouds for 3D Object Detection in Adverse Weather

LiDAR fog simulation Created by Martin Hahner at the Computer Vision Lab of ETH Zurich. This is the official code release of the paper Fog Simulation

May 5, 2022
Point Cloud Denoising input segmentation output raw point-cloud valid/clear fog rain de-noised Abstract Lidar sensors are frequently used in environme
Point Cloud Denoising input segmentation output raw point-cloud valid/clear fog rain de-noised Abstract Lidar sensors are frequently used in environme

Point Cloud Denoising input segmentation output raw point-cloud valid/clear fog rain de-noised Abstract Lidar sensors are frequently used in environme

May 17, 2022
Synthetic LiDAR sequential point cloud dataset with point-wise annotations
Synthetic LiDAR sequential point cloud dataset with point-wise annotations

SynLiDAR dataset: Learning From Synthetic LiDAR Sequential Point Cloud This is official repository of the SynLiDAR dataset. For technical details, ple

May 12, 2022
The all new way to turn your boring vector meshes into the new fad in town; Voxels!
The all new way to turn your boring vector meshes into the new fad in town; Voxels!

Voxelator The all new way to turn your boring vector meshes into the new fad in town; Voxels! Notes: I have not tested this on a rotated mesh. With fu

Feb 3, 2022
Uncertainty-aware Semantic Segmentation of LiDAR Point Clouds for Autonomous Driving
Uncertainty-aware Semantic Segmentation of LiDAR Point Clouds for Autonomous Driving

SalsaNext: Fast, Uncertainty-aware Semantic Segmentation of LiDAR Point Clouds for Autonomous Driving Abstract In this paper, we introduce SalsaNext f

May 15, 2022
Motion detector, Full body detection, Upper body detection, Cat face detection, Smile detection, Face detection (haar cascade), Silverware detection, Face detection (lbp), and Sending email notifications
Motion detector, Full body detection, Upper body detection, Cat face detection, Smile detection, Face detection (haar cascade), Silverware detection, Face detection (lbp), and Sending email notifications

Security camera running OpenCV for object and motion detection. The camera will send email with image of any objects it detects. It also runs a server that provides web interface with live stream video.

Jun 30, 2021
CLOCs: Camera-LiDAR Object Candidates Fusion for 3D Object Detection

CLOCs is a novel Camera-LiDAR Object Candidates fusion network. It provides a low-complexity multi-modal fusion framework that improves the performance of single-modality detectors. CLOCs operates on the combined output candidates of any 3D and any 2D detector, and is trained to produce more accurate 3D and 2D detection results.

May 13, 2022
A general python framework for single object tracking in LiDAR point clouds, based on PyTorch Lightning.
A general python framework for single object tracking in LiDAR point clouds, based on PyTorch Lightning.

Open3DSOT A general python framework for single object tracking in LiDAR point clouds, based on PyTorch Lightning. The official code release of BAT an

May 18, 2022
HSC4D: Human-centered 4D Scene Capture in Large-scale Indoor-outdoor Space Using Wearable IMUs and LiDAR. CVPR 2022
HSC4D: Human-centered 4D Scene Capture in Large-scale Indoor-outdoor Space Using Wearable IMUs and LiDAR. CVPR 2022

HSC4D: Human-centered 4D Scene Capture in Large-scale Indoor-outdoor Space Using Wearable IMUs and LiDAR. CVPR 2022 [Project page | Video] Getting sta

May 17, 2022
[CVPR 2022] CoTTA Code for our CVPR 2022 paper Continual Test-Time Domain Adaptation

CoTTA Code for our CVPR 2022 paper Continual Test-Time Domain Adaptation Prerequisite Please create and activate the following conda envrionment. To r

May 20, 2022
Part-Aware Data Augmentation for 3D Object Detection in Point Cloud
Part-Aware Data Augmentation for 3D Object Detection in Point Cloud

Part-Aware Data Augmentation for 3D Object Detection in Point Cloud This repository contains a reference implementation of our Part-Aware Data Augment

May 10, 2022
Unofficial implementation of Point-Unet: A Context-Aware Point-Based Neural Network for Volumetric Segmentation

Point-Unet This is an unofficial implementation of the MICCAI 2021 paper Point-Unet: A Context-Aware Point-Based Neural Network for Volumetric Segment

May 2, 2022
(CVPR 2021) Back-tracing Representative Points for Voting-based 3D Object Detection in Point Clouds
(CVPR 2021) Back-tracing Representative Points for Voting-based 3D Object Detection in Point Clouds

BRNet Introduction This is a release of the code of our paper Back-tracing Representative Points for Voting-based 3D Object Detection in Point Clouds,

May 7, 2022
(CVPR 2021) Back-tracing Representative Points for Voting-based 3D Object Detection in Point Clouds
(CVPR 2021) Back-tracing Representative Points for Voting-based 3D Object Detection in Point Clouds

BRNet Introduction This is a release of the code of our paper Back-tracing Representative Points for Voting-based 3D Object Detection in Point Clouds,

May 7, 2022