Detectron2 is FAIR's next-generation platform for object detection and segmentation.

Detectron2 is Facebook AI Research's next generation software system that implements state-of-the-art object detection algorithms. It is a ground-up rewrite of the previous version, Detectron, and it originates from maskrcnn-benchmark.

What's New

  • It is powered by the PyTorch deep learning framework.
  • Includes more features such as panoptic segmentation, Densepose, Cascade R-CNN, rotated bounding boxes, PointRend, DeepLab, etc.
  • Can be used as a library to support different projects on top of it. We'll open source more research projects in this way.
  • It trains much faster.
  • Models can be exported to TorchScript format or Caffe2 format for deployment.

See our blog post to see more demos and learn about detectron2.

Installation

See INSTALL.md.

Getting Started

Follow the installation instructions to install detectron2.

See Getting Started with Detectron2, and the Colab Notebook to learn about basic usage.

Learn more at our documentation. And see projects/ for some projects that are built on top of detectron2.

Model Zoo and Baselines

We provide a large set of baseline results and trained models available for download in the Detectron2 Model Zoo.

License

Detectron2 is released under the Apache 2.0 license.

Citing Detectron2

If you use Detectron2 in your research or wish to refer to the baseline results published in the Model Zoo, please use the following BibTeX entry.

@misc{wu2019detectron2,
  author =       {Yuxin Wu and Alexander Kirillov and Francisco Massa and
                  Wan-Yen Lo and Ross Girshick},
  title =        {Detectron2},
  howpublished = {\url{https://github.com/facebookresearch/detectron2}},
  year =         {2019}
}
Comments
  • Add support for ONNX-only and Caffe2 ONNX export

    Add support for ONNX-only and Caffe2 ONNX export

    Summary of changes

    This PR fixes both ONNX-only and Caffe2 ONNX exporters for the latest versions of this repo and PyTorch.

    For ONNX-only, the main issue is that add_export_config(cfg) is not exposed when Caffe2 is not compiled along with PyTorch, but for ONNX-only scenarios, such dependency is not needed. Therefore, add_export_config is moved from detectron2/export/api.py to detectron2/export/__init__.py

    A second contribution is a new test_export_onnx.py test file that export almost the same models as the test_export_tracing.py tests.

    For the Caffe2-ONNX, the main issue was a dependency on ONNX optimizer pass which is deprecated in newer ONNX versions. This PR removes such dependency because fuse_bn_into_conv optimization pass is already performed by torch.onnx.export anyway.

    Fixes https://github.com/facebookresearch/detectron2/issues/3488 Fixes https://github.com/pytorch/pytorch/issues/69674 (PyTorch repo)

  • How do I compute validation loss during training?

    How do I compute validation loss during training?

    How do I compute validation loss during training?

    I'm trying to compute the loss on a validation dataset for each iteration during training. To do so, I've created my own hook:

    class ValidationLoss(detectron2.engine.HookBase):
        def __init__(self, config, dataset_name):
            super(ValidationLoss, self).__init__()
            self._loader = detectron2.data.build_detection_test_loader(config, dataset_name)
            
        def after_step(self):
            for batch in self._loader:
                loss = self.trainer.model(batch)
                log.debug(f"validation loss: {loss}")
    

    ... which I register with a DefaultTrainer. The hook code is called during training, but fails with the following:

    INFO:detectron2.engine.train_loop:Starting training from iteration 0
    ERROR:detectron2.engine.train_loop:Exception during training:
    Traceback (most recent call last):
      File "/ascldap/users/tshead/miniconda3/lib/python3.7/site-packages/detectron2/engine/train_loop.py", line 133, in train
        self.after_step()
      File "/ascldap/users/tshead/miniconda3/lib/python3.7/site-packages/detectron2/engine/train_loop.py", line 153, in after_step
        h.after_step()
      File "<ipython-input-6-63b308743b7d>", line 8, in after_step
        loss = self.trainer.model(batch)
      File "/ascldap/users/tshead/miniconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 532, in __call__
        result = self.forward(*input, **kwargs)
      File "/ascldap/users/tshead/miniconda3/lib/python3.7/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 123, in forward
        proposals, proposal_losses = self.proposal_generator(images, features, gt_instances)
      File "/ascldap/users/tshead/miniconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 532, in __call__
        result = self.forward(*input, **kwargs)
      File "/ascldap/users/tshead/miniconda3/lib/python3.7/site-packages/detectron2/modeling/proposal_generator/rpn.py", line 164, in forward
        losses = {k: v * self.loss_weight for k, v in outputs.losses().items()}
      File "/ascldap/users/tshead/miniconda3/lib/python3.7/site-packages/detectron2/modeling/proposal_generator/rpn_outputs.py", line 322, in losses
        gt_objectness_logits, gt_anchor_deltas = self._get_ground_truth()
      File "/ascldap/users/tshead/miniconda3/lib/python3.7/site-packages/detectron2/modeling/proposal_generator/rpn_outputs.py", line 262, in _get_ground_truth
        for image_size_i, anchors_i, gt_boxes_i in zip(self.image_sizes, anchors, self.gt_boxes):
    TypeError: zip argument #3 must support iteration
    INFO:detectron2.engine.hooks:Total training time: 0:00:00 (0:00:00 on hooks)
    

    The traceback seems to imply that ground truth data is missing, which made me think that the data loader was the problem. However, switching to a training loader produces a different error:

    class ValidationLoss(detectron2.engine.HookBase):
        def __init__(self, config, dataset_name):
            super(ValidationLoss, self).__init__()
            self._loader = detectron2.data.build_detection_train_loader(config, dataset_name)
            
        def after_step(self):
            for batch in self._loader:
                loss = self.trainer.model(batch)
                log.debug(f"validation loss: {loss}")
    
    INFO:detectron2.engine.train_loop:Starting training from iteration 0
    ERROR:detectron2.engine.train_loop:Exception during training:
    Traceback (most recent call last):
      File "/ascldap/users/tshead/miniconda3/lib/python3.7/site-packages/detectron2/engine/train_loop.py", line 133, in train
        self.after_step()
      File "/ascldap/users/tshead/miniconda3/lib/python3.7/site-packages/detectron2/engine/train_loop.py", line 153, in after_step
        h.after_step()
      File "<ipython-input-6-e0d2c509cc72>", line 7, in after_step
        for batch in self._loader:
      File "/ascldap/users/tshead/miniconda3/lib/python3.7/site-packages/detectron2/data/common.py", line 109, in __iter__
        for d in self.dataset:
      File "/ascldap/users/tshead/miniconda3/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 345, in __next__
        data = self._next_data()
      File "/ascldap/users/tshead/miniconda3/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 856, in _next_data
        return self._process_data(data)
      File "/ascldap/users/tshead/miniconda3/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 881, in _process_data
        data.reraise()
      File "/ascldap/users/tshead/miniconda3/lib/python3.7/site-packages/torch/_utils.py", line 394, in reraise
        raise self.exc_type(msg)
    TypeError: Caught TypeError in DataLoader worker process 0.
    Original Traceback (most recent call last):
      File "/ascldap/users/tshead/miniconda3/lib/python3.7/site-packages/torch/utils/data/_utils/worker.py", line 178, in _worker_loop
        data = fetcher.fetch(index)
      File "/ascldap/users/tshead/miniconda3/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
        data = [self.dataset[idx] for idx in possibly_batched_index]
      File "/ascldap/users/tshead/miniconda3/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in <listcomp>
        data = [self.dataset[idx] for idx in possibly_batched_index]
      File "/ascldap/users/tshead/miniconda3/lib/python3.7/site-packages/detectron2/data/common.py", line 39, in __getitem__
        data = self._map_func(self._dataset[cur_idx])
      File "/ascldap/users/tshead/miniconda3/lib/python3.7/site-packages/detectron2/utils/serialize.py", line 23, in __call__
        return self._obj(*args, **kwargs)
    TypeError: 'str' object is not callable
    
    INFO:detectron2.engine.hooks:Total training time: 0:00:00 (0:00:00 on hooks)
    

    As a sanity check, inference works just fine:

    class ValidationLoss(detectron2.engine.HookBase):
        def __init__(self, config, dataset_name):
            super(ValidationLoss, self).__init__()
            self._loader = detectron2.data.build_detection_test_loader(config, dataset_name)
            
        def after_step(self):
            for batch in self._loader:
                with detectron2.evaluation.inference_context(self.trainer.model):
                    loss = self.trainer.model(batch)
                    log.debug(f"validation loss: {loss}")
    
    INFO:detectron2.engine.train_loop:Starting training from iteration 0
    DEBUG:root:validation loss: [{'instances': Instances(num_instances=100, image_height=720, image_width=720, fields=[pred_boxes = Boxes(tensor([[4.4867e+02, 1.9488e+02, 5.1496e+02, 3.9878e+02],
            [4.2163e+02, 1.1204e+02, 6.1118e+02, 5.5378e+02],
            [8.7323e-01, 3.0374e+02, 9.2917e+01, 3.8698e+02],
            [4.3202e+02, 2.0296e+02, 5.7938e+02, 3.6817e+02],
            ...
    

    ... but that isn't what I want, of course. Any thoughts?

    Thanks in advance, Tim

  • How to detect only one class (person) from the coco pre trained model

    How to detect only one class (person) from the coco pre trained model

    Can any one tell me how to select only one class which is 'person' in my case from coco data for instance segmentation. Pre trained model. (mask_rcnn_R_50_FPN_3x.yaml)

    I want to detect 'person' from the given image only.

  • onnx model exportable support

    onnx model exportable support

    Since onnx provides almost all ops needs by maskrcnn, it would be great if model can exported to onnx and would be benefit more from TensorRT acceleration for these large models.

  • Improve documentation concerning the new config files

    Improve documentation concerning the new config files

    📚 Documentation Improvements

    In short

    Concerning: https://detectron2.readthedocs.io/en/latest/tutorials/configs.html Problem: Documentation does not seem to have been updated to reflect the new config files (.py rather than .yaml) Solution: Update the documentation

    Problem description

    FAIR recently published new Mask R-CNN baselines and this was my first introduction to the new config file that no longer relies on YAML files but on 'raw' .py files. I am trying to load the new baselines using the config files mentioned in the MODEL_ZOO (see this table). For example:

    from detectron2 import model_zoo
    model = model_zoo.get("new_baselines/mask_rcnn_regnetx_4gf_dds_FPN_400ep_LSJ.py", trained=True)
    

    This gives

    RuntimeError: new_baselines/mask_rcnn_regnetx_4gf_dds_FPN_400ep_LSJ not available in Model Zoo!
    

    I have installed Detectron2 using the installation instructions. When looking up the documentation on configs, it seems that this has not been updated to reflect the new configs and still solely mentions YAML files.

    Proposed solution

    It could be that the CONFIG_PATH_TO_URL_SUFFIX dictionary in _ModelZooUrls class still has to be updated and that this is actually a bug (see here), but I find it hard to estimate wheter this is meant behavior (i.e. the new config file should be loaded differently) or a bug due to my limited understanding of the new config files. Either way, I therefore feel like the documentation on readthedocs should be updated to reflect the change from .yaml to .py.

  • Add support for ONNX-only

    Add support for ONNX-only

    This PR is composed of different fixes to enable and end-to-end ONNX export functionality for detectron2 models

    • add_export_config API is publicly available exposed even when caffe2 is not compiled along with PyTorch (that is the new default behavior on latest PyTorch). A warning message informing users about its deprecation on future versions is also added

    • tensor.shape[0] replaces len(tensor) and for idx, img in enumerate(tensors) replaces for tmp_var1, tmp_var2 in zip(tensors, batched_imgs) so that the tracer does not lose reference to the user input on the graphs.

      • Before the changes above, the graph (see below) does not have an actual input. Instead, the input is exported as a model weight image
      • After the fix, the user images are properly acknowledged as model's input (see below) during ONNX export image
    • Added unit tests (tests/torch_export_onnx.py) for detectron2 models

    • ONNX is added as dependency for the CI to be able to run the aforementioned tests

    • Added custom symbolic functions to allow CI pipelines to succeed. The symbolics are needed because PyTorch 1.8, 1.9 and 1.10 adopted by detectron2 have several bugs. They can be removed when 1.11+ is adopted by detectron2's CI infra

    Fixes https://github.com/facebookresearch/detectron2/issues/3488 Fixes https://github.com/pytorch/pytorch/issues/69674 (PyTorch repo)

  • Added docker compose file with useful tweaks.

    Added docker compose file with useful tweaks.

    From my perspective a docker-compose file has several benefits. On the one hand, it increases the comfort and on the other hand it is a way to supply users with useful tweaks. The commited docker-compose file addresses several issues and tweaks:

    1. It fixes potential problems with dataloaders (see #384).
    2. It includes Multi-GPU and performance tweaks as suggested by NVIDIA (see https://docs.nvidia.com/deeplearning/frameworks/user-guide/index.html#caffeovr).
    3. It adds GUI support (see #379).
    4. It enables caching of downloaded models (see #382).
    5. It makes docker run with the UID of the host user.

    Of course, all this can be accomplished with a long docker command. However, a docker-compose file give a central place to gather all recommendations for running detectron2 with docker, without bloating the Dockerfile with comments.

  • Properly convert a Detectron2  model to ONNX for Deployment

    Properly convert a Detectron2 model to ONNX for Deployment

    Hello,

    I am trying to convert a Detectron2 model to ONNX format and make inference without use detectron2 dependence in inference stage.

    Even is possible to find some information about that here : https://detectron2.readthedocs.io/en/latest/tutorials/deployment.html The implementation of this task is constantly being updated and the information found in this documentation is not clear enough to carry out this task .

    Some one can help me with some Demo/Tutorial of how make it ?

    @thiagocrepaldi

    Some information:

    My model was trained using pre-trained weight from:

    'faster_rcnn_50': { 'model_path': 'COCO-Detection/faster_rcnn_R_50_FPN_3x.yaml', 'weights_path': 'model_final_280758.pkl' },

    I have 4 classes.

    Of course now i have my our weight. My model was saved in .pth forrmat.

    I used my our dataset, with image ( .png )

    Code in Python

  • How to apply mask_rcnn segmentation on a balloon video ?

    How to apply mask_rcnn segmentation on a balloon video ?

    Hi, I am going through the google colab example tutorial.

    I am trying to apply mask_rcnn segmentation on a random youtube balloon-video instead of an balloon-image to detect balloon only (one class).

    How can I assign .yaml and .pkl files that were generated using images earlier in the tutorial to a random video? thanks

    I tried the foolowing but it didn't work. I think I am having trouble assign the trained config and model files.

    !cd detectron2_repo && python demo/demo.py --config-file configs/COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml --video-input ../video-clip_b.mp4 --confidence-threshold 0.6 --output ../video-clip_b_testing1.mkv \
      --opts MODEL.WEIGHTS detectron2://COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x/137849600/model_final_f10217.pkl
    
  • Add support for Caffe2 ONNX export

    Add support for Caffe2 ONNX export

    Currently all Caffe2 export tests (under tests/test_export_caffe2.py) fail because the latest onnx releases do not have onnx.optimizer submodule anymore (instead, a new module onnxoptimizer was created from it)

    However, fuse_bn_into_conv optimization previously implemented within onnx.optimizer is already performed by torch.onnx.export too ruing ONNX export. Therefore onnx.optimizer dependency can be safely removed from detectron2 code.

    Depends on pytorch/pytorch#75718 Fixes #3488 Fixes pytorch/pytorch#69674 (PyTorch repo)

    ps: Although Caffe2 support is/will be deprecated, this PR relies on the fact that contributions are welcome as stated at docs/tutorials/deployment.md

  • AssertionError: Attribute 'thing_classes' in the metadata of 'coco_2017_train' cannot be set to a different value!

    AssertionError: Attribute 'thing_classes' in the metadata of 'coco_2017_train' cannot be set to a different value!

    ❓ Questions and Help

    Hi, I am trying to train on my dataset with just 4 classes. When I run it, I get an error as below: image The scripts are interlinked a lot and therefore a bit difficult to debug. How to resolve this?

    Thanks.

  • how to compile roi align with mps (apple m1 chip) backend support

    how to compile roi align with mps (apple m1 chip) backend support

    Hi I'm recently trying to run detectron2 with apple m1 on mac pro, I met the erro with roi_align:

      File "/Users/taowang/PycharmProjects/eql.detectron2/detectron2/layers/roi_align.py", line 20, in forward
        input, roi, spatial_scale, output_size[0], output_size[1], sampling_ratio, aligned
    RuntimeError: input.device().is_cpu() INTERNAL ASSERT FAILED at "/Users/taowang/PycharmProjects/eql.detectron2/detectron2/layers/csrc/ROIAlign/ROIAlign_cpu.cpp":408, please report a bug to PyTorch. input must be a CPU tensor
    

    the input device is on "mps" as expected, but here it seems not compatible with ROIAlign, I wonder how to compile ROIAlign to support mps backend

  • Unable to change cfg.INPUT.MAX_SIZE_TEST to more than 1422

    Unable to change cfg.INPUT.MAX_SIZE_TEST to more than 1422

    I want to use build_detection_test_loader() instead of cv2.imread() to speed up I/O when predict on large amount of images.

    The object detection model is trained by crops of images (1280 x 720), I would like to use the model directly predict on the original images (5472 x 3078). However, the output of dataloader seems to stop at 1422 no matter what bigger value of cfg.INPUT.MAX_SIZE_TEST is given.

    Instructions To Reproduce the Issue:

    1. Full runnable code
    frame_dir_path = 'demo_frames/demo_0489_frames/0-149' # can be any dir path with .jpg images
    frames_paths = glob(os.path.join(frame_dir_path, '*.jpg'))[:5]
    print(f'Image size: {cv2.imread(frames_paths[0]).shape}')
    
    def frame_paths_to_dataset_dict(image_files: List[str]) -> List[dict]:
        dataset_dicts = []
        for image_file in image_files:
            record = {"file_name": image_file}
            dataset_dicts.append(record)
        return dataset_dicts
    
    DatasetCatalog.register('inference', lambda: frame_paths_to_dataset_dict(frames_paths))
    
    cfg = get_cfg()
    
    cfg.INPUT.MAX_SIZE_TEST = 5000 # I tried several values here
    data_loader = build_detection_test_loader(cfg, 'inference')
    
    images = []
    with torch.no_grad():
        for inputs in data_loader:
            im = inputs
            images.append(im)
    images[0][0]['image'].shape
    
    1. Full logs
    Image size: (3078, 5472, 3)
    torch.Size([3, 800, 1422])
    

    Expected behavior:

    The image in inputs of dataloader iteration should either be the same with the original resolution or comply with cfg.INPUT.MAX_SIZE_TEST.

    Environment:

    2022-11-29 18:39:26 URL:https://raw.githubusercontent.com/facebookresearch/detectron2/main/detectron2/utils/collect_env.py [8391/8391] -> "collect_env.py" [1]
    ----------------------  ----------------------------------------------------------------
    sys.platform            linux
    Python                  3.8.5 (default, Jan 27 2021, 15:41:15) [GCC 9.3.0]
    numpy                   1.23.5
    detectron2              0.6 @/usr/local/lib/python3.8/dist-packages/detectron2
    Compiler                GCC 9.3
    CUDA compiler           CUDA 11.3
    detectron2 arch flags   8.0
    DETECTRON2_ENV_MODULE   <not set>
    PyTorch                 1.13.0+cu117 @/usr/local/lib/python3.8/dist-packages/torch
    PyTorch debug build     False
    GPU available           Yes
    GPU 0                   A100-SXM4-40GB (arch=8.0)
    Driver version          460.39
    CUDA_HOME               /usr/local/cuda
    Pillow                  9.3.0
    torchvision             0.14.0+cu117 @/usr/local/lib/python3.8/dist-packages/torchvision
    torchvision arch flags  3.5, 5.0, 6.0, 7.0, 7.5, 8.0, 8.6
    fvcore                  0.1.5.post20221122
    iopath                  0.1.9
    cv2                     4.6.0
    ----------------------  ----------------------------------------------------------------
    PyTorch built with:
      - GCC 9.3
      - C++ Version: 201402
      - Intel(R) Math Kernel Library Version 2020.0.0 Product Build 20191122 for Intel(R) 64 architecture applications
      - Intel(R) MKL-DNN v2.6.0 (Git Hash 52b5f107dd9cf10910aaa19cb47f3abf9b349815)
      - OpenMP 201511 (a.k.a. OpenMP 4.5)
      - LAPACK is enabled (usually provided by MKL)
      - NNPACK is enabled
      - CPU capability usage: AVX2
      - CUDA Runtime 11.7
      - NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86
      - CuDNN 8.5
      - Magma 2.6.1
      - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.7, CUDNN_VERSION=8.5.0, CXX_COMPILER=/opt/rh/devtoolset-9/root/usr/bin/c++, CXX_FLAGS= -fabi-version=11 -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wunused-local-typedefs -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Werror=cast-function-type -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.13.0, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF,
    
  • visualizer.draw_dataset_dict draws a segmentation mask for the

    visualizer.draw_dataset_dict draws a segmentation mask for the "unlabeled" semantic category

    Using this guide, I have registered a custom dataset to detectron2 for using in semantic segmentation of nuclei in histopathologic images. However, Visualizer shows me an image inversely masked.

    Instructions To Reproduce the Issue:

    1. Full runnable code or full changes you made:
    def get_patho_dicts(train_dir):
        im_paths = Path(f'{train_dir}/Tissue Images').glob('*')
        dataset_dicts = []
        for idx, im_path in enumerate(im_paths):
            height, width = cv2.imread(str(im_path)).shape[:2]
            record = {
                'file_name': str(im_path),
                'image_id': idx,
                'height': height,
                'width': width,
                'sem_seg_file_name': f'{train_dir}/Masks/{im_path.stem}.png',
                'segments_info': [ # Though this is related to panoptic segmentation based on doc, I wanted to try it too; which didn't help either.
                    {
                      'id': 255,
                      'category_id': 0,
                    },
                ],
            }
            dataset_dicts.append(record)
        return dataset_dicts
    
    for d in ['train']:
        DatasetCatalog.register('monuseg1_' + d, lambda d = d: get_patho_dicts('MoNuSeg/train'))
        MetadataCatalog.get('monuseg1_' + d).set(stuff_classes = ['nucleus'], stuff_colors=[(0,255,0)])
    
    monuseg_metadata = MetadataCatalog.get("monuseg_train")
    
    1. What exact command you run:
    dataset_dicts = get_patho_dicts('MoNuSeg/train')
    
    for d in random.sample(dataset_dicts, 1):
        print(d)
        img = cv2.imread(d['file_name'], -1)
        visualizer = Visualizer(img[:, :, ::-1], metadata=monuseg_metadata, scale=1)
        out = visualizer.draw_dataset_dict(d)
        cv2_imshow(out.get_image())
    
    1. Full logs or other relevant observations:
    • Ground truth mask: image

    • Inversely masked image: image

    Expected behavior:

    As the mask images are composed of a 1000 x 1000 x 1 array with pixels being either 0 or 255, the image should have been masked with pixels of 255. However, the other pixels are colored.

    Environment:

    Paste the output of the following command:

    ----------------------  ----------------------------------------------------------------
    sys.platform            linux
    Python                  3.7.15 (default, Oct 12 2022, 19:14:55) [GCC 7.5.0]
    numpy                   1.21.6
    detectron2              0.6 @/usr/local/lib/python3.7/dist-packages/detectron2
    Compiler                GCC 7.5
    CUDA compiler           CUDA 11.2
    detectron2 arch flags   7.5
    DETECTRON2_ENV_MODULE   <not set>
    PyTorch                 1.12.1+cu113 @/usr/local/lib/python3.7/dist-packages/torch
    PyTorch debug build     False
    GPU available           Yes
    GPU 0                   Tesla T4 (arch=7.5)
    Driver version          460.32.03
    CUDA_HOME               /usr/local/cuda
    Pillow                  7.1.2
    torchvision             0.13.1+cu113 @/usr/local/lib/python3.7/dist-packages/torchvision
    torchvision arch flags  3.5, 5.0, 6.0, 7.0, 7.5, 8.0, 8.6
    fvcore                  0.1.5.post20221122
    iopath                  0.1.9
    cv2                     4.6.0
    ----------------------  ----------------------------------------------------------------
    PyTorch built with:
      - GCC 9.3
      - C++ Version: 201402
      - Intel(R) Math Kernel Library Version 2020.0.0 Product Build 20191122 for Intel(R) 64 architecture applications
      - Intel(R) MKL-DNN v2.6.0 (Git Hash 52b5f107dd9cf10910aaa19cb47f3abf9b349815)
      - OpenMP 201511 (a.k.a. OpenMP 4.5)
      - LAPACK is enabled (usually provided by MKL)
      - NNPACK is enabled
      - CPU capability usage: AVX2
      - CUDA Runtime 11.3
      - NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86
      - CuDNN 8.3.2  (built against CUDA 11.5)
      - Magma 2.5.2
      - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.3, CUDNN_VERSION=8.3.2, CXX_COMPILER=/opt/rh/devtoolset-9/root/usr/bin/c++, CXX_FLAGS= -fabi-version=11 -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Werror=cast-function-type -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.12.1, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=OFF, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, 
    
  • make fairscale optional dependency

    make fairscale optional dependency

    Make fairscale optional. It's used rarely and has caused people to report installation issues before such as #4550. Getting fairscale to work in my environment is also nontrivial so it would be good if it's optional.

    Remove constraints on black version (fix #4005).

  • Neptune + detectron2 Integration

    Neptune + detectron2 Integration

    Hi,

    Kshiteej Kalambarkar here, a Developer Advocate at Neptune.ai

    I'm working on the detectron2 integration to help detectron2 users easily log their experiment's metadata to Neptune with a few lines of code.

    I would like to connect with you (maintainers) and know more about your product roadmap (i.e. upcoming features and breaking changes) and if you'd be interested in upstreaming it into the detectron2 repository.

    Thank you

OCR-D wrapper for detectron2 based segmentation models

ocrd_detectron2 OCR-D wrapper for detectron2 based segmentation models Introduction Installation Usage OCR-D processor interface ocrd-detectron2-segm

Dec 6, 2022
A pytorch implementation of faster RCNN detection framework (Use detectron2, it's a masterpiece)
A pytorch implementation of faster RCNN detection framework (Use detectron2, it's a masterpiece)

Notice(2019.11.2) This repo was built back two years ago when there were no pytorch detection implementation that can achieve reasonable performance.

Nov 24, 2022
NExT-QA: Next Phase of Question-Answering to Explaining Temporal Actions (CVPR2021)
NExT-QA: Next Phase of Question-Answering to Explaining Temporal Actions (CVPR2021)

NExT-QA We reproduce some SOTA VideoQA methods to provide benchmark results for our NExT-QA dataset accepted to CVPR2021 (with 1 'Strong Accept' and 2

Nov 24, 2022
You Only Look One-level Feature (YOLOF), CVPR2021, Detectron2
You Only Look One-level Feature (YOLOF), CVPR2021, Detectron2

You Only Look One-level Feature (YOLOF), CVPR2021 A simple, fast, and efficient object detector without FPN. This repo provides a neat implementation

Nov 10, 2022
Detectron2 for Document Layout Analysis
Detectron2 for Document Layout Analysis

Detectron2 trained on PubLayNet dataset This repo contains the training configurations, code and trained models trained on PubLayNet dataset using Det

Nov 21, 2022
OpenMMLab's Next Generation Video Understanding Toolbox and Benchmark
OpenMMLab's Next Generation Video Understanding Toolbox and Benchmark

Introduction English | 简体中文 MMAction2 is an open-source toolbox for video understanding based on PyTorch. It is a part of the OpenMMLab project. The m

Dec 3, 2022
A Next Generation ConvNet by FaceBookResearch Implementation in PyTorch(Original) and TensorFlow.

ConvNeXt A Next Generation ConvNet by FaceBookResearch Implementation in PyTorch(Original) and TensorFlow. A FacebookResearch Implementation on A Conv

Feb 14, 2022
Source code for the GPT-2 story generation models in the EMNLP 2020 paper "STORIUM: A Dataset and Evaluation Platform for Human-in-the-Loop Story Generation"

Storium GPT-2 Models This is the official repository for the GPT-2 models described in the EMNLP 2020 paper [STORIUM: A Dataset and Evaluation Platfor

Aug 10, 2022
Code for the paper "Next Generation Reservoir Computing"

Next Generation Reservoir Computing This is the code for the results and figures in our paper "Next Generation Reservoir Computing". They are written

Nov 9, 2022
TorchDistiller - a collection of the open source pytorch code for knowledge distillation, especially for the perception tasks, including semantic segmentation, depth estimation, object detection and instance segmentation.

This project is a collection of the open source pytorch code for knowledge distillation, especially for the perception tasks, including semantic segmentation, depth estimation, object detection and instance segmentation.

Nov 29, 2022
A Data Annotation Tool for Semantic Segmentation, Object Detection and Lane Line Detection.(In Development Stage)
A Data Annotation Tool for Semantic Segmentation, Object Detection and Lane Line Detection.(In Development Stage)

Data-Annotation-Tool How to Run this Tool? To run this software, follow the steps: git clone https://github.com/Autonomous-Car-Project/Data-Annotation

Aug 18, 2022
Hybrid CenterNet - Hybrid-supervised object detection / Weakly semi-supervised object detection
Hybrid CenterNet - Hybrid-supervised object detection / Weakly semi-supervised object detection

Hybrid-Supervised Object Detection System Object detection system trained by hybrid-supervision/weakly semi-supervision (HSOD/WSSOD): This project is

Mar 21, 2022
Yolo object detection - Yolo object detection with python

How to run download required files make build_image make download Docker versio

Jan 26, 2022
FAIR's research platform for object detection research, implementing popular algorithms like Mask R-CNN and RetinaNet.
FAIR's research platform for object detection research, implementing popular algorithms like Mask R-CNN and RetinaNet.

Detectron is deprecated. Please see detectron2, a ground-up rewrite of Detectron in PyTorch. Detectron Detectron is Facebook AI Research's software sy

Dec 5, 2022
In this project, we develop a face recognize platform based on MTCNN object-detection netcwork and FaceNet self-supervised network.
In this project, we develop a face recognize platform based on MTCNN object-detection netcwork and FaceNet self-supervised network.

模式识别大作业——人脸检测与识别平台 本项目是一个简易的人脸检测识别平台,提供了人脸信息录入和人脸识别的功能。前端采用 html+css+js,后端采用 pytorch,

Aug 2, 2022
Tools to create pixel-wise object masks, bounding box labels (2D and 3D) and 3D object model (PLY triangle mesh) for object sequences filmed with an RGB-D camera.
Tools to create pixel-wise object masks, bounding box labels (2D and 3D) and 3D object model (PLY triangle mesh) for object sequences filmed with an RGB-D camera.

Tools to create pixel-wise object masks, bounding box labels (2D and 3D) and 3D object model (PLY triangle mesh) for object sequences filmed with an RGB-D camera. This project prepares training and testing data for various deep learning projects such as 6D object pose estimation projects singleshotpose, as well as object detection and instance segmentation projects.

Nov 28, 2022
PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud, CVPR 2019.
PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud, CVPR 2019.

PointRCNN PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud Code release for the paper PointRCNN:3D Object Proposal Generation a

Nov 29, 2022
Object tracking and object detection is applied to track golf puts in real time and display stats/games.

Putting_Game Object tracking and object detection is applied to track golf puts in real time and display stats/games. Works best with the Perfect Prac

Dec 29, 2021