Detectron2 源码分析
Detectron2 源码分析Content:安装环境代码结构训练流程预测流程重点代码讲解Detectron2已集成模型参考链接1. 安装环境安装参考 INSTALL.md目前Detectron2只支持在Linux和macOS上安装，不支持Windows安装。安装anaconda$ wget https://repo.anaconda.com/archive/Anaconda3-2020.02-L
·
Detectron2 源码分析
Content:
- 安装环境
- 代码结构
- 训练流程
- 预测流程
- 重点代码讲解
- Detectron2已集成模型
- 参考链接
1. 安装环境
-
安装参考 INSTALL.md
目前Detectron2只支持在Linux和macOS上安装，不支持Windows安装。
-
安装anaconda
$ wget https://repo.anaconda.com/archive/Anaconda3-2020.02-Linux-x86_64.sh $ chmod 755 Anaconda3-2020.02-Linux-x86_64.sh $ ./Anaconda3-2020.02-Linux-x86_64.sh $ source ~/.bashrc -
安装cuda ,下载地址
$ wget http://developer.download.nvidia.com/compute/cuda/10.1/Prod/local_installers/cuda_10.1.243_418.87.00_linux.run $ sudo sh cuda_10.1.243_418.87.00_linux.run -
安装pytorch
# 参考：https://pytorch.org/get-started/locally/ # 无GPU命令 $ pip install torch==1.5.1+cpu torchvision==0.6.1+cpu -f https://download.pytorch.org/whl/torch_stable.html -
安装其他依赖
由于安装detectron2需要编译C++代码，所以必须安装gcc编译器。
$ echo "y" | sudo apt-get install build-essential cmake gcc vim gitcd $ pip install 'git+https://github.com/facebookresearch/fvcore' $ pip install opencv-python $ pip install pycocotools>=2.0.1 -
安装Detectron2
python -m pip install 'git+https://github.com/wincooler/detectron2.git'
-
安装调试环境
在开发环境中，最好使用IDE进行可视化调试，推荐试用VSCode
- 安装VSCode
- python在VSCode中如何调试 。在 .vscode/launch.json 中配置
{ // Use IntelliSense to learn about possible attributes. // Hover to view descriptions of existing attributes. // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387 "version": "0.2.0", "configurations": [ { "name": "Python: Current File", "type": "python", "request": "launch", "program": "${file}", "console": "integratedTerminal", // demo.py args key point "args":[ "--config-file" ,"/home/xyz/work/git/detectron2/configs/COCO-Keypoints/keypoint_rcnn_R_50_FPN_1x.yaml", "--input", "/home/xyz/work/images/kp_434230.jpg", "/home/xyz/work/images/kp_491613.jpg", "--output", "/home/xyz/work/images/output_infer_kp", "--opts", "MODEL.WEIGHTS", "detectron2://COCO-Keypoints/keypoint_rcnn_R_50_FPN_1x/137261548/model_final_04e291.pkl", "MODEL.DEVICE", "cpu" ], //train_net_detect.py "args":[ "--config-file" , "/home/xyz/work/git/detectron2/configs/COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_1x.yaml", "OUTPUT_DIR","/home/xyz/work/test_d2/train_net_detect", "MODEL.DEVICE", "cpu" , "SOLVER.IMS_PER_BATCH" , "2" , "SOLVER.BASE_LR" , "0.0025" ], "justMyCode": false , //Debug时，是否调进库代码 } ] }
- 代码规范推荐pep8, 可在VSCode中安装pylint扩展 ,在 .vscode\settings.json中配置。
{ "python.pythonPath": "C:\\ProgramData\\Anaconda3\\envs\\d2\\python.exe", //指定python环境 "python.envFile": "E:\\work\\github\\PyDL2\\src\\.vscode\\dev.env", //指定环境变量目录 "python.linting.enabled": true, //是否检测代码规范 "python.linting.pycodestyleEnabled": true, //指定代码规范类型 "python.linting.pycodestyleArgs": ["--ignore=E303"], //忽略指定的错误类型 "python.linting.pylintArgs": ["--max-line-length=100"], //指定行最大字符数 "python.linting.pycodestyleArgs": [ "--max-line-length=100"], //指定行最大字符数 }
2. 代码结构
(base) xyz@ubuntu:~/work/git/detectron2$ tree
.
├── configs # 存放配置文件的文件夹，由于模型中参数过多，使用yaml文件来统一管理。
│ ├── Base-RCNN-C4.yaml
│ ├── Base-RCNN-DilatedC5.yaml
│ ├── Base-RCNN-FPN.yaml
│ ├── Base-RetinaNet.yaml
│ ├── Cityscapes
│ │ └── mask_rcnn_R_50_FPN.yaml
│ ├── COCO-Detection
│ │ ├── faster_rcnn_R_101_C4_3x.yaml
│ │ ├── faster_rcnn_R_101_DC5_3x.yaml
│ │ ├── faster_rcnn_R_101_FPN_3x.yaml
│ │ ├── faster_rcnn_R_50_C4_1x.yaml
│ │ ├── faster_rcnn_R_50_C4_3x.yaml
│ │ ├── faster_rcnn_R_50_DC5_1x.yaml
│ │ ├── faster_rcnn_R_50_DC5_3x.yaml
│ │ ├── faster_rcnn_R_50_FPN_1x.yaml
│ │ ├── faster_rcnn_R_50_FPN_3x.yaml
│ │ ├── faster_rcnn_X_101_32x8d_FPN_3x.yaml
│ │ ├── fast_rcnn_R_50_FPN_1x.yaml
│ │ ├── retinanet_R_101_FPN_3x.yaml
│ │ ├── retinanet_R_50_FPN_1x.yaml
│ │ ├── retinanet_R_50_FPN_3x.yaml
│ │ ├── rpn_R_50_C4_1x.yaml
│ │ └── rpn_R_50_FPN_1x.yaml
│ ├── COCO-InstanceSegmentation
│ │ ├── mask_rcnn_R_101_C4_3x.yaml
│ │ ├── mask_rcnn_R_101_DC5_3x.yaml
│ │ ├── mask_rcnn_R_101_FPN_3x.yaml
│ │ ├── mask_rcnn_R_50_C4_1x.yaml
│ │ ├── mask_rcnn_R_50_C4_3x.yaml
│ │ ├── mask_rcnn_R_50_DC5_1x.yaml
│ │ ├── mask_rcnn_R_50_DC5_3x.yaml
│ │ ├── mask_rcnn_R_50_FPN_1x_giou.yaml
│ │ ├── mask_rcnn_R_50_FPN_1x.yaml
│ │ ├── mask_rcnn_R_50_FPN_3x.yaml
│ │ └── mask_rcnn_X_101_32x8d_FPN_3x.yaml
│ ├── COCO-Keypoints
│ │ ├── Base-Keypoint-RCNN-FPN.yaml
│ │ ├── keypoint_rcnn_R_101_FPN_3x.yaml
│ │ ├── keypoint_rcnn_R_50_FPN_1x.yaml
│ │ ├── keypoint_rcnn_R_50_FPN_3x.yaml
│ │ └── keypoint_rcnn_X_101_32x8d_FPN_3x.yaml
│ ├── COCO-PanopticSegmentation
│ │ ├── Base-Panoptic-FPN.yaml
│ │ ├── panoptic_fpn_R_101_3x.yaml
│ │ ├── panoptic_fpn_R_50_1x.yaml
│ │ └── panoptic_fpn_R_50_3x.yaml
│ ├── Detectron1-Comparisons
│ │ ├── faster_rcnn_R_50_FPN_noaug_1x.yaml
│ │ ├── keypoint_rcnn_R_50_FPN_1x.yaml
│ │ ├── mask_rcnn_R_50_FPN_noaug_1x.yaml
│ │ └── README.md
│ ├── LVISv0.5-InstanceSegmentation
│ │ ├── mask_rcnn_R_101_FPN_1x.yaml
│ │ ├── mask_rcnn_R_50_FPN_1x.yaml
│ │ └── mask_rcnn_X_101_32x8d_FPN_1x.yaml
│ ├── LVISv1-InstanceSegmentation
│ │ ├── mask_rcnn_R_101_FPN_1x.yaml
│ │ ├── mask_rcnn_R_50_FPN_1x.yaml
│ │ └── mask_rcnn_X_101_32x8d_FPN_1x.yaml
│ ├── Misc
│ │ ├── cascade_mask_rcnn_R_50_FPN_1x.yaml
│ │ ├── cascade_mask_rcnn_R_50_FPN_3x.yaml
│ │ ├── cascade_mask_rcnn_X_152_32x8d_FPN_IN5k_gn_dconv.yaml
│ │ ├── mask_rcnn_R_50_FPN_1x_cls_agnostic.yaml
│ │ ├── mask_rcnn_R_50_FPN_1x_dconv_c3-c5.yaml
│ │ ├── mask_rcnn_R_50_FPN_3x_dconv_c3-c5.yaml
│ │ ├── mask_rcnn_R_50_FPN_3x_gn.yaml
│ │ ├── mask_rcnn_R_50_FPN_3x_syncbn.yaml
│ │ ├── panoptic_fpn_R_101_dconv_cascade_gn_3x.yaml
│ │ ├── scratch_mask_rcnn_R_50_FPN_3x_gn.yaml
│ │ ├── scratch_mask_rcnn_R_50_FPN_9x_gn.yaml
│ │ ├── scratch_mask_rcnn_R_50_FPN_9x_syncbn.yaml
│ │ └── semantic_R_50_FPN_1x.yaml
│ ├── PascalVOC-Detection
│ │ ├── faster_rcnn_R_50_C4.yaml
│ │ └── faster_rcnn_R_50_FPN.yaml
│ └── quick_schedules
│ ├── cascade_mask_rcnn_R_50_FPN_inference_acc_test.yaml
│ ├── cascade_mask_rcnn_R_50_FPN_instant_test.yaml
│ ├── fast_rcnn_R_50_FPN_inference_acc_test.yaml
│ ├── fast_rcnn_R_50_FPN_instant_test.yaml
│ ├── keypoint_rcnn_R_50_FPN_inference_acc_test.yaml
│ ├── keypoint_rcnn_R_50_FPN_instant_test.yaml
│ ├── keypoint_rcnn_R_50_FPN_normalized_training_acc_test.yaml
│ ├── keypoint_rcnn_R_50_FPN_training_acc_test.yaml
│ ├── mask_rcnn_R_50_C4_GCV_instant_test.yaml
│ ├── mask_rcnn_R_50_C4_inference_acc_test.yaml
│ ├── mask_rcnn_R_50_C4_instant_test.yaml
│ ├── mask_rcnn_R_50_C4_training_acc_test.yaml
│ ├── mask_rcnn_R_50_DC5_inference_acc_test.yaml
│ ├── mask_rcnn_R_50_FPN_inference_acc_test.yaml
│ ├── mask_rcnn_R_50_FPN_instant_test.yaml
│ ├── mask_rcnn_R_50_FPN_pred_boxes_training_acc_test.yaml
│ ├── mask_rcnn_R_50_FPN_training_acc_test.yaml
│ ├── panoptic_fpn_R_50_inference_acc_test.yaml
│ ├── panoptic_fpn_R_50_instant_test.yaml
│ ├── panoptic_fpn_R_50_training_acc_test.yaml
│ ├── README.md
│ ├── retinanet_R_50_FPN_inference_acc_test.yaml
│ ├── retinanet_R_50_FPN_instant_test.yaml
│ ├── rpn_R_50_FPN_inference_acc_test.yaml
│ ├── rpn_R_50_FPN_instant_test.yaml
│ ├── semantic_R_50_FPN_inference_acc_test.yaml
│ ├── semantic_R_50_FPN_instant_test.yaml
│ └── semantic_R_50_FPN_training_acc_test.yaml
├── datasets # 获取数据集的脚本
│ ├── coco
│ │ └── annotations
│ │ ├── instances_minival2014_100.json
│ │ ├── instances_train2017.json
│ │ ├── instances_val2017_100.json
│ │ ├── instances_val2017.json
│ │ ├── person_keypoints_minival2014_100.json
│ │ └── person_keypoints_val2017_100.json
│ ├── prepare_cocofied_lvis.py
│ ├── prepare_for_tests.sh
│ ├── prepare_panoptic_fpn.py
│ └── README.md
├── demo # 执行inference的demo
│ ├── demo.py # demo 入口
│ ├── predictor.py # 对图片/视频检测做了封装，可以借鉴，避免重复造轮子
│ └── README.md
├── detectron2 # detectron2核心代码
│ ├── checkpoint # 模型权重文件的相关处理
│ │ ├── c2_model_loading.py # 处理Caffe2与Detectron模型权重的转换
│ │ ├── catalog.py # 对预训练模型的存取
│ │ ├── detection_checkpoint.py # #读进模型文件(*.pkl)，存在内存中
│ │ └── __init__.py
│ ├── config # 配置训练参数的相关处理
│ │ ├── compat.py # 处理配置参数的兼容问题
│ │ ├── config.py # 定义CfgNode类
│ │ ├── defaults.py # 总的默认初始化参数
│ │ └── __init__.py
│ ├── data # 数据集的相关处理
│ │ ├── build.py # 构建 train/test的Pytorch DataLoader
│ │ ├── catalog.py # 对数据集的描述信息metadata及数据集本身的管理
│ │ ├── common.py
│ │ ├── dataset_mapper.py
│ │ ├── datasets
│ │ │ ├── builtin_meta.py # 注册内置数据集元数据metadata
│ │ │ ├── builtin.py # 注册内置数据集
│ │ │ ├── cityscapes.py
│ │ │ ├── coco.py # 提供加载coco格式数据集的方法
│ │ │ ├── __init__.py
│ │ │ ├── lvis.py
│ │ │ ├── lvis_v0_5_categories.py
│ │ │ ├── lvis_v1_categories.py
│ │ │ ├── pascal_voc.py
│ │ │ ├── README.md
│ │ │ └── register_coco.py # 提供注册coco格式数据集的方法
│ │ ├── detection_utils.py # 提供图像读取，转换等处理方法
│ │ ├── __init__.py
│ │ ├── samplers
│ │ │ ├── distributed_sampler.py # 继承了pytorch采样器
│ │ │ ├── grouped_batch_sampler.py # 继承了pytorch采样器
│ │ │ └── __init__.py
│ │ └── transforms # 数据转化/增强
│ │ ├── augmentation_impl.py
│ │ ├── augmentation.py
│ │ ├── __init__.py
│ │ └── transform.py
│ ├── engine # 训练流程引擎，提供训练流程框架
│ │ ├── defaults.py # 提供各种方法:默认命令行参数，默认启动，默认预测器，默认训练器等。
│ │ ├── hooks.py # 提供训练过程中调用的8种钩子方法，比如CallbackHook，IterationTimer，PeriodicWriter，PeriodicCheckpointer，LRScheduler等
│ │ ├── __init__.py
│ │ ├── launch.py # 提供分布式启动入口方法
│ │ └── train_loop.py # 提供训练流程框架
│ ├── evaluation # 提供各种模型评估方法
│ │ ├── cityscapes_evaluation.py
│ │ ├── coco_evaluation.py
│ │ ├── evaluator.py
│ │ ├── fast_eval_api.py
│ │ ├── __init__.py
│ │ ├── lvis_evaluation.py
│ │ ├── panoptic_evaluation.py
│ │ ├── pascal_voc_evaluation.py
│ │ ├── rotated_coco_evaluation.py
│ │ ├── sem_seg_evaluation.py
│ │ └── testing.py
│ ├── export # 转换detectron2模型为caffe2,onnx模型, 参考：https://detectron2.readthedocs.io/tutorials/deployment.html
│ │ ├── api.py
│ │ ├── c10.py
│ │ ├── caffe2_export.py
│ │ ├── caffe2_inference.py
│ │ ├── caffe2_modeling.py
│ │ ├── __init__.py
│ │ ├── patcher.py
│ │ ├── README.md
│ │ └── shared.py
│ ├── __init__.py
│ ├── layers # 提供各种网络层，包括一些C++/Cuda代码，在detectron setup的时候会被编译
│ │ ├── batch_norm.py
│ │ ├── blocks.py
│ │ ├── csrc
│ │ │ ├── box_iou_rotated
│ │ │ │ ├── box_iou_rotated_cpu.cpp
│ │ │ │ ├── box_iou_rotated_cuda.cu
│ │ │ │ ├── box_iou_rotated.h
│ │ │ │ └── box_iou_rotated_utils.h
│ │ │ ├── cocoeval
│ │ │ │ ├── cocoeval.cpp
│ │ │ │ └── cocoeval.h
│ │ │ ├── cuda_version.cu
│ │ │ ├── deformable
│ │ │ │ ├── deform_conv_cuda.cu
│ │ │ │ ├── deform_conv_cuda_kernel.cu
│ │ │ │ └── deform_conv.h
│ │ │ ├── nms_rotated
│ │ │ │ ├── nms_rotated_cpu.cpp
│ │ │ │ ├── nms_rotated_cuda.cu
│ │ │ │ └── nms_rotated.h
│ │ │ ├── README.md
│ │ │ ├── ROIAlign
│ │ │ │ ├── ROIAlign_cpu.cpp
│ │ │ │ ├── ROIAlign_cuda.cu
│ │ │ │ └── ROIAlign.h
│ │ │ ├── ROIAlignRotated
│ │ │ │ ├── ROIAlignRotated_cpu.cpp
│ │ │ │ ├── ROIAlignRotated_cuda.cu
│ │ │ │ └── ROIAlignRotated.h
│ │ │ └── vision.cpp
│ │ ├── deform_conv.py
│ │ ├── __init__.py
│ │ ├── mask_ops.py
│ │ ├── nms.py
│ │ ├── roi_align.py
│ │ ├── roi_align_rotated.py
│ │ ├── rotated_boxes.py
│ │ ├── shape_spec.py
│ │ └── wrappers.py
│ ├── modeling # 提供模型构建方法
│ │ ├── anchor_generator.py
│ │ ├── backbone # 提供网络骨架
│ │ │ ├── backbone.py
│ │ │ ├── build.py
│ │ │ ├── fpn.py
│ │ │ ├── __init__.py
│ │ │ └── resnet.py
│ │ ├── box_regression.py
│ │ ├── __init__.py
│ │ ├── matcher.py
│ │ ├── meta_arch # 提供网络架构
│ │ │ ├── build.py
│ │ │ ├── __init__.py
│ │ │ ├── panoptic_fpn.py
│ │ │ ├── rcnn.py # 把 class GeneralizedRCNN 实例化一个对象注册进 META_ARCH_REGISTRY
│ │ │ ├── retinanet.py
│ │ │ └── semantic_seg.py
│ │ ├── poolers.py
│ │ ├── postprocessing.py
│ │ ├── proposal_generator
│ │ │ ├── build.py
│ │ │ ├── __init__.py
│ │ │ ├── proposal_utils.py
│ │ │ ├── rpn.py
│ │ │ └── rrpn.py
│ │ ├── roi_heads
│ │ │ ├── box_head.py
│ │ │ ├── cascade_rcnn.py
│ │ │ ├── fast_rcnn.py
│ │ │ ├── __init__.py
│ │ │ ├── keypoint_head.py
│ │ │ ├── mask_head.py
│ │ │ ├── roi_heads.py
│ │ │ └── rotated_fast_rcnn.py
│ │ ├── sampling.py
│ │ └── test_time_augmentation.py
│ ├── model_zoo # 提供预训练的模型
│ │ ├── __init__.py
│ │ └── model_zoo.py
│ ├── solver # 构建学习率调整器
│ │ ├── build.py
│ │ ├── __init__.py
│ │ └── lr_scheduler.py
│ ├── structures # 提供图片/标注的各种数据结构
│ │ ├── boxes.py
│ │ ├── image_list.py
│ │ ├── __init__.py
│ │ ├── instances.py
│ │ ├── keypoints.py
│ │ ├── masks.py
│ │ └── rotated_boxes.py
│ └── utils # 提供常用的utils方法
│ ├── analysis.py
│ ├── collect_env.py
│ ├── colormap.py
│ ├── comm.py
│ ├── env.py
│ ├── events.py
│ ├── __init__.py
│ ├── logger.py
│ ├── memory.py
│ ├── README.md
│ ├── registry.py
│ ├── serialize.py
│ ├── video_visualizer.py
│ └── visualizer.py
├── dev
│ ├── linter.sh
│ ├── packaging
│ │ ├── build_all_wheels.sh
│ │ ├── build_wheel.sh
│ │ ├── gen_install_table.py
│ │ ├── gen_wheel_index.sh
│ │ ├── pkg_helpers.bash
│ │ └── README.md
│ ├── parse_results.sh
│ ├── README.md
│ ├── run_inference_tests.sh
│ └── run_instant_tests.sh
├── docker # 提供docker文件，以便部署到docker环境中
│ ├── docker-compose.yml # 目前docker-compose不支持GPU，所以该yaml文件目前是无效的
│ ├── Dockerfile # 可借鉴的Dockerfile文件
│ ├── Dockerfile-circleci
│ └── README.md
├── docs # 项目文档源码，生成后可阅读文档： https://detectron2.readthedocs.io/
│ ├── conf.py
│ ├── index.rst
│ ├── Makefile
│ ├── modules
│ │ ├── checkpoint.rst
│ │ ├── config.rst
│ │ ├── data.rst
│ │ ├── engine.rst
│ │ ├── evaluation.rst
│ │ ├── export.rst
│ │ ├── index.rst
│ │ ├── layers.rst
│ │ ├── modeling.rst
│ │ ├── model_zoo.rst
│ │ ├── solver.rst
│ │ ├── structures.rst
│ │ └── utils.rst
│ ├── notes
│ │ ├── benchmarks.md
│ │ ├── changelog.md
│ │ ├── compatibility.md
│ │ ├── contributing.md -> ../../.github/CONTRIBUTING.md
│ │ └── index.rst
│ ├── README.md
│ ├── requirements.txt
│ ├── _static
│ │ └── css
│ │ └── custom.css
│ └── tutorials
│ ├── builtin_datasets.md -> ../../datasets/README.md
│ ├── configs.md
│ ├── data_loading.md
│ ├── datasets.md
│ ├── deployment.md
│ ├── evaluation.md
│ ├── extend.md
│ ├── getting_started.md -> ../../GETTING_STARTED.md
│ ├── index.rst
│ ├── install.md -> ../../INSTALL.md
│ ├── models.md
│ ├── README.md
│ ├── training.md
│ └── write-models.md
├── GETTING_STARTED.md # 启动开始文档
├── INSTALL.md # 安装文档
├── LICENSE
├── MODEL_ZOO.md # model zoo 说明文档
├── projects # 提供了facebook正在研究的4个projects
│ ├── DensePose
│ │ ├── apply_net.py
│ │ ├── configs
│ │ │ ├── Base-DensePose-RCNN-FPN.yaml
│ │ │ ├── densepose_rcnn_R_101_FPN_DL_s1x.yaml
│ │ │ ├── densepose_rcnn_R_101_FPN_DL_WC1_s1x.yaml
│ │ │ ├── densepose_rcnn_R_101_FPN_DL_WC2_s1x.yaml
│ │ │ ├── densepose_rcnn_R_101_FPN_s1x_legacy.yaml
│ │ │ ├── densepose_rcnn_R_101_FPN_s1x.yaml
│ │ │ ├── densepose_rcnn_R_101_FPN_WC1_s1x.yaml
│ │ │ ├── densepose_rcnn_R_101_FPN_WC2_s1x.yaml
│ │ │ ├── densepose_rcnn_R_50_FPN_DL_s1x.yaml
│ │ │ ├── densepose_rcnn_R_50_FPN_DL_WC1_s1x.yaml
│ │ │ ├── densepose_rcnn_R_50_FPN_DL_WC2_s1x.yaml
│ │ │ ├── densepose_rcnn_R_50_FPN_s1x_legacy.yaml
│ │ │ ├── densepose_rcnn_R_50_FPN_s1x.yaml
│ │ │ ├── densepose_rcnn_R_50_FPN_WC1_s1x.yaml
│ │ │ ├── densepose_rcnn_R_50_FPN_WC2_s1x.yaml
│ │ │ ├── evolution
│ │ │ │ ├── Base-RCNN-FPN-MC.yaml
│ │ │ │ ├── densepose_R_101_FPN_1x_Atop10_toP.yaml
│ │ │ │ ├── densepose_R_101_FPN_DL_1x_Atop10_toP.yaml
│ │ │ │ ├── densepose_R_101_FPN_DL_WC1_1x_Atop10_toP.yaml
│ │ │ │ ├── densepose_R_101_FPN_DL_WC1M_1x_Atop10_toP.yaml
│ │ │ │ ├── densepose_R_101_FPN_WC1_1x_Atop10_toP.yaml
│ │ │ │ ├── densepose_R_101_FPN_WC1M_1x_Atop10_toP.yaml
│ │ │ │ ├── densepose_R_50_FPN_1x_Atop10_toP.yaml
│ │ │ │ ├── densepose_R_50_FPN_DL_1x_Atop10_toP.yaml
│ │ │ │ ├── densepose_R_50_FPN_DL_WC1_1x_Atop10_toP.yaml
│ │ │ │ ├── densepose_R_50_FPN_DL_WC1M_1x_Atop10_toP.yaml
│ │ │ │ ├── densepose_R_50_FPN_WC1_1x_Atop10_toP.yaml
│ │ │ │ ├── densepose_R_50_FPN_WC1M_1x_Atop10_toP.yaml
│ │ │ │ └── faster_rcnn_R_50_FPN_1x_MC.yaml
│ │ │ └── quick_schedules
│ │ │ ├── densepose_rcnn_R_50_FPN_DL_instant_test.yaml
│ │ │ ├── densepose_rcnn_R_50_FPN_inference_acc_test.yaml
│ │ │ ├── densepose_rcnn_R_50_FPN_instant_test.yaml
│ │ │ ├── densepose_rcnn_R_50_FPN_training_acc_test.yaml
│ │ │ ├── densepose_rcnn_R_50_FPN_TTA_inference_acc_test.yaml
│ │ │ ├── densepose_rcnn_R_50_FPN_WC1_instant_test.yaml
│ │ │ └── densepose_rcnn_R_50_FPN_WC2_instant_test.yaml
│ │ ├── densepose
│ │ │ ├── config.py
│ │ │ ├── data
│ │ │ │ ├── build.py
│ │ │ │ ├── combined_loader.py
│ │ │ │ ├── dataset_mapper.py
│ │ │ │ ├── datasets
│ │ │ │ │ ├── builtin.py
│ │ │ │ │ ├── chimpnsee.py
│ │ │ │ │ ├── coco.py
│ │ │ │ │ ├── dataset_type.py
│ │ │ │ │ └── __init__.py
│ │ │ │ ├── inference_based_loader.py
│ │ │ │ ├── __init__.py
│ │ │ │ ├── structures.py
│ │ │ │ ├── transform
│ │ │ │ │ ├── image.py
│ │ │ │ │ └── __init__.py
│ │ │ │ ├── utils.py
│ │ │ │ └── video
│ │ │ │ ├── frame_selector.py
│ │ │ │ ├── __init__.py
│ │ │ │ └── video_keyframe_dataset.py
│ │ │ ├── densepose_coco_evaluation.py
│ │ │ ├── densepose_head.py
│ │ │ ├── engine
│ │ │ │ ├── __init__.py
│ │ │ │ └── trainer.py
│ │ │ ├── evaluator.py
│ │ │ ├── __init__.py
│ │ │ ├── modeling
│ │ │ │ └── test_time_augmentation.py
│ │ │ ├── roi_head.py
│ │ │ ├── utils
│ │ │ │ ├── dbhelper.py
│ │ │ │ ├── logger.py
│ │ │ │ └── transform.py
│ │ │ └── vis
│ │ │ ├── base.py
│ │ │ ├── bounding_box.py
│ │ │ ├── densepose.py
│ │ │ └── extractor.py
│ │ ├── dev
│ │ │ ├── README.md
│ │ │ ├── run_inference_tests.sh
│ │ │ └── run_instant_tests.sh
│ │ ├── doc
│ │ │ ├── GETTING_STARTED.md
│ │ │ ├── images
│ │ │ │ ├── res_bbox_dp_contour.jpg
│ │ │ │ ├── res_bbox_dp_segm.jpg
│ │ │ │ ├── res_bbox_dp_u.jpg
│ │ │ │ ├── res_bbox_dp_v.jpg
│ │ │ │ ├── vis_bbox_dp_i.jpg
│ │ │ │ ├── vis_bbox_dp_pts.jpg
│ │ │ │ ├── vis_bbox_dp_segm.jpg
│ │ │ │ ├── vis_bbox_dp_u.jpg
│ │ │ │ └── vis_bbox_dp_v.jpg
│ │ │ ├── MODEL_ZOO.md
│ │ │ ├── TOOL_APPLY_NET.md
│ │ │ └── TOOL_QUERY_DB.md
│ │ ├── query_db.py
│ │ ├── README.md
│ │ ├── tests
│ │ │ ├── common.py
│ │ │ ├── test_combine_data_loader.py
│ │ │ ├── test_frame_selector.py
│ │ │ ├── test_image_resize_transform.py
│ │ │ ├── test_model_e2e.py
│ │ │ ├── test_setup.py
│ │ │ ├── test_structures.py
│ │ │ └── test_video_keyframe_dataset.py
│ │ └── train_net.py
│ ├── PointRend
│ │ ├── configs
│ │ │ ├── InstanceSegmentation
│ │ │ │ ├── Base-PointRend-RCNN-FPN.yaml
│ │ │ │ ├── pointrend_rcnn_R_50_FPN_1x_cityscapes.yaml
│ │ │ │ ├── pointrend_rcnn_R_50_FPN_1x_coco.yaml
│ │ │ │ └── pointrend_rcnn_R_50_FPN_3x_coco.yaml
│ │ │ └── SemanticSegmentation
│ │ │ ├── Base-PointRend-Semantic-FPN.yaml
│ │ │ ├── pointrend_semantic_R_101_FPN_1x_cityscapes.yaml
│ │ │ └── pointrend_semantic_R_50_FPN_1x_coco.yaml
│ │ ├── point_rend
│ │ │ ├── coarse_mask_head.py
│ │ │ ├── color_augmentation.py
│ │ │ ├── config.py
│ │ │ ├── dataset_mapper.py
│ │ │ ├── __init__.py
│ │ │ ├── point_features.py
│ │ │ ├── point_head.py
│ │ │ ├── roi_heads.py
│ │ │ └── semantic_seg.py
│ │ ├── README.md
│ │ └── train_net.py
│ ├── README.md
│ ├── TensorMask
│ │ ├── configs
│ │ │ ├── Base-TensorMask.yaml
│ │ │ ├── tensormask_R_50_FPN_1x.yaml
│ │ │ └── tensormask_R_50_FPN_6x.yaml
│ │ ├── README.md
│ │ ├── setup.py
│ │ ├── tensormask
│ │ │ ├── arch.py
│ │ │ ├── config.py
│ │ │ ├── __init__.py
│ │ │ └── layers
│ │ │ ├── csrc
│ │ │ │ ├── SwapAlign2Nat
│ │ │ │ │ ├── SwapAlign2Nat_cuda.cu
│ │ │ │ │ └── SwapAlign2Nat.h
│ │ │ │ └── vision.cpp
│ │ │ ├── __init__.py
│ │ │ └── swap_align2nat.py
│ │ ├── tests
│ │ │ ├── __init__.py
│ │ │ └── test_swap_align2nat.py
│ │ └── train_net.py
│ └── TridentNet
│ ├── configs
│ │ ├── Base-TridentNet-Fast-C4.yaml
│ │ ├── tridentnet_fast_R_101_C4_3x.yaml
│ │ ├── tridentnet_fast_R_50_C4_1x.yaml
│ │ └── tridentnet_fast_R_50_C4_3x.yaml
│ ├── README.md
│ ├── train_net.py
│ └── tridentnet
│ ├── config.py
│ ├── __init__.py
│ ├── __pycache__
│ │ ├── config.cpython-37.pyc
│ │ ├── __init__.cpython-37.pyc
│ │ ├── trident_backbone.cpython-37.pyc
│ │ ├── trident_conv.cpython-37.pyc
│ │ ├── trident_rcnn.cpython-37.pyc
│ │ └── trident_rpn.cpython-37.pyc
│ ├── trident_backbone.py
│ ├── trident_conv.py
│ ├── trident_rcnn.py
│ └── trident_rpn.py
├── README.md
├── setup.cfg
├── setup.py # detectron2 安装程序
├── tests # 单元测试代码
│ ├── data
│ │ ├── __init__.py
│ │ ├── test_coco_evaluation.py
│ │ ├── test_coco.py
│ │ ├── test_detection_utils.py
│ │ ├── test_rotation_transform.py
│ │ ├── test_sampler.py
│ │ └── test_transforms.py
│ ├── __init__.py
│ ├── layers
│ │ ├── __init__.py
│ │ ├── test_mask_ops.py
│ │ ├── test_nms_rotated.py
│ │ ├── test_roi_align.py
│ │ └── test_roi_align_rotated.py
│ ├── modeling
│ │ ├── __init__.py
│ │ ├── test_anchor_generator.py
│ │ ├── test_box2box_transform.py
│ │ ├── test_fast_rcnn.py
│ │ ├── test_matcher.py
│ │ ├── test_model_e2e.py
│ │ ├── test_roi_heads.py
│ │ ├── test_roi_pooler.py
│ │ └── test_rpn.py
│ ├── README.md
│ ├── structures
│ │ ├── __init__.py
│ │ ├── test_boxes.py
│ │ ├── test_imagelist.py
│ │ ├── test_instances.py
│ │ └── test_rotated_boxes.py
│ ├── test_checkpoint.py
│ ├── test_config.py
│ ├── test_export_caffe2.py
│ ├── test_model_analysis.py
│ ├── test_model_zoo.py
│ └── test_visualizer.py
└── tools # 提供的训练，模型转换工具
├── analyze_model.py
├── benchmark.py
├── convert-torchvision-to-d2.py
├── deploy
│ ├── caffe2_converter.py # 转换detectron2模型为caffe2, onnx或torchscript
│ ├── caffe2_mask_rcnn.cpp
│ ├── CMakeLists.txt
│ ├── README.md
│ └── torchscript_traced_mask_rcnn.cpp
├── plain_train_net.py # 训练入口例子
├── README.md
├── train_net.py # 训练入口例子
├── visualize_data.py
└── visualize_json_results.py
94 directories, 515 files
3. 训练流程
见：tools\train_net.py 和 tools\plain_train_net.py
4. 预测流程
见： demo\demo.py
5. 重点代码讲解
文件解读:
-
.circleci\config.yml
- CI配置文件，参考使用 CircleCI 2.0 进行持续集成/持续部署, 官网
- 涉及安装python, detectron2, nvidia driver, 跑 unittest等。
- 跑unittest命令,参考Python必会的单元测试框架 —— unittest
python -m unittest discover -v -s tests -
.vscode\settings.json
{ "python.pythonPath": "C:\\ProgramData\\Anaconda3\\envs\\d2\\python.exe", //指定python环境 "python.envFile": "E:\\work\\github\\PyDL2\\src\\.vscode\\dev.env", //指定环境变量目录 "python.linting.enabled": true, //是否检测代码规范 "python.linting.pycodestyleEnabled": true, //指定代码规范类型 "python.linting.pycodestyleArgs": ["--ignore=E303"], //忽略指定的错误类型 "python.linting.pylintArgs": ["--max-line-length=100"], //指定行最大字符数 "python.linting.pycodestyleArgs": [ "--max-line-length=100"], //指定行最大字符数 } -
.vscode/launch.json
{ // Use IntelliSense to learn about possible attributes. // Hover to view descriptions of existing attributes. // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387 "version": "0.2.0", "configurations": [ { "name": "Python: Current File", "type": "python", "request": "launch", "program": "${file}", "console": "integratedTerminal", // demo.py args key point "args":[ "--config-file" ,"/home/xyz/work/git/detectron2/configs/COCO-Keypoints/keypoint_rcnn_R_50_FPN_1x.yaml", "--input", "/home/xyz/work/images/kp_434230.jpg", "/home/xyz/work/images/kp_491613.jpg", "--output", "/home/xyz/work/images/output_infer_kp", "--opts", "MODEL.WEIGHTS", "detectron2://COCO-Keypoints/keypoint_rcnn_R_50_FPN_1x/137261548/model_final_04e291.pkl", "MODEL.DEVICE", "cpu" ], //train_net_detect.py "args":[ "--config-file" , "/home/xyz/work/git/detectron2/configs/COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_1x.yaml", "OUTPUT_DIR","/home/xyz/work/test_d2/train_net_detect", "MODEL.DEVICE", "cpu" , "SOLVER.IMS_PER_BATCH" , "2" , "SOLVER.BASE_LR" , "0.0025" ], "justMyCode": false , //Debug时，是否调进库代码 } ] } -
configs\Cityscapes\*
-
configs\COCO-Detection\*
COCO目标检测测评指标
cocoapi
coco官网 -
configs\COCO-InstanceSegmentation
-
configs\COCO-Keypoints
-
configs\COCO-PanopticSegmentation
-
configs\Detectron1-Comparisons
-
configs\LVISv0.5-InstanceSegmentation
-
configs\LVISv1-InstanceSegmentation
-
configs\Misc
-
configs\PascalVOC-Detection
-
configs\quick_schedules
-
datasets\README.md
- 讲述内置数据集builtin datasets ,数据集存放目录由环境变量
DETECTRON2_DATASETS指定,export DETECTRON2_DATASETS=/path/to/datasets,If left unset, the default is./datasetsrelative to your current working directory. - model zoo ,包含了各种baseline metrics
- Some of the builtin tests (
dev/run_*_tests.sh) uses a tiny version of the COCO dataset, 测试模型
which you can download with./prepare_for_tests.sh. 下载数据集
- 讲述内置数据集builtin datasets ,数据集存放目录由环境变量
-
datasets\prepare_for_tests.sh
- 下载内置数据集 coco json描述
root@hp6-exx:/home/appuser/detectron2_repo/datasets# ./prepare_for_tests.sh --2020-07-23 02:21:02-- https://dl.fbaipublicfiles.com/detectron2/annotations/coco/instances_val2017_100.json root@hp6-exx:/home/appuser/detectron2_repo/datasets# ls coco/annotations/ instances_minival2014_100.json person_keypoints_minival2014_100.json instances_val2017_100.json person_keypoints_val2017_100.json -
datasets\prepare_cocofied_lvis.py
-
datasets\prepare_panoptic_fpn.py
-
detectron2/_ init_.py
- 调用了 setup_environment 启动detectron2就会被执行，初始化环境，程序生命周期内只执行一次，通过全局变量_ENV_SETUP_DONE来控制
- 调用了 torch/_ init_.py
-
/home/xyz/.local/lib/python3.7/site-packages/detectron2/utils/env.py
- setup_environment
- Perform environment setup work. The default setup is a no-op, but this
function allows the user to specify a Python source file or a module in
the $DETECTRON2_ENV_MODULE environment variable, that performs
custom setup work that may be necessary to their computing environment. - 检查了是否使用opencv
- 检测torch, fvcore,yaml的版本
- Perform environment setup work. The default setup is a no-op, but this
- setup_custom_environment , 设置了环境变量DETECTRON2_ENV_MODULE，可加载自定义的模块,放置一些启动初始化的代码:log的配置，可通过socketio发出去，放入"detectron2.utils.env.custom_module"
- setup_environment
-
/home/xyz/.local/lib/python3.7/site-packages/detectron2/utils/collect_env.py
- def collect_env_info():
--------------------- -------------------------------------------------------------------- sys.platform linux Python 3.7.6 (default, Jan 8 2020, 19:59:22) [GCC 7.3.0] numpy 1.18.1 detectron2 0.1.3 @/home/xyz/.local/lib/python3.7/site-packages/detectron2 Compiler GCC 7.5 CUDA compiler not available DETECTRON2_ENV_MODULE <not set> PyTorch 1.5.1+cpu @/home/xyz/.local/lib/python3.7/site-packages/torch PyTorch debug build False GPU available False Pillow 7.0.0 torchvision 0.6.1+cpu @/home/xyz/.local/lib/python3.7/site-packages/torchvision fvcore 0.1.1 cv2 4.3.0 --------------------- -------------------------------------------------------------------- PyTorch built with: - GCC 7.3 - C++ Version: 201402 - Intel(R) Math Kernel Library Version 2019.0.5 Product Build 20190808 for Intel(R) 64 architecture applications - Intel(R) MKL-DNN v0.21.1 (Git Hash 7d2fd500bc78936d1d648ca713b901012f470dbc) - OpenMP 201511 (a.k.a. OpenMP 4.5) - NNPACK is enabled - CPU capability usage: AVX2 - Build settings: BLAS=MKL, BUILD_TYPE=Release, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -fopenmp -DNDEBUG -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DUSE_INTERNAL_THREADPOOL_IMPL -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, USE_CUDA=0, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=OFF, USE_NNPACK=ON, USE_OPENMP=ON, USE_STATIC_DISPATCH=OFF, -
detectron2/utils/visualizer.py
__all__ = ["ColorMode", "VisImage", "Visualizer"]把预测结果或annotation的数据画到图上。
-
/detectron2/checkpoint/catalog.py
- class ModelCatalog(object): # Store mappings from names to third-party models.
- class ModelCatalogHandler(PathHandler): # Resolve URL like catalog://.
- class Detectron2Handler(PathHandler): # Resolve anything that’s in Detectron2 model zoo.
PathManager.register_handler(ModelCatalogHandler())
PathManager.register_handler(Detectron2Handler()) - 注册 detectron2:// 进PathManager，可下载该前缀的模型
-
detectron2/data/catalog.py
- class DatasetCatalog(object): ，register, get, list , clear, remove 数据集
- class Metadata(types.SimpleNamespace): , 存储数据集的metadata
- class MetadataCatalog: , MetadataCatalog provides access to “Metadata” of a given dataset.
# callable 判断是否是可调用的函数 assert callable(func), "You must register a function with `DatasetCatalog.register`!" -
detectron2/data/datasets/register_coco.py
- def register_coco_instances(name, metadata, json_file, image_root): , 从json_file中加载coco数据集
assert isinstance(image_root, (str, os.PathLike)), image_root #判断字符串是否是路径
- def register_coco_instances(name, metadata, json_file, image_root): , 从json_file中加载coco数据集
-
detectron2/data/detection_utils.py
- Common data processing utilities that are used in a typical object detection data pipeline.
- 暴露方法
__all__ = [ "SizeMismatchError", # 加载图片时，图片尺寸可能与annotation的不同 "convert_image_to_rgb", "check_image_size", "transform_proposals", "transform_instance_annotations", "annotations_to_instances", "annotations_to_instances_rotated", "build_augmentation", "build_transform_gen", "create_keypoint_hflip_indices", "filter_empty_instances", "read_image", ] # Convert PIL image to numpy array of target format. def convert_PIL_to_numpy(image, format): # Convert an image from given format to RGB. def convert_image_to_rgb(image, format): # Applies the exif orientation correctly. 按exif信息转换图片方向 def _apply_exif_orientation(image): # Read an image into the given format. Will apply rotation and flipping if the image has such exif information. 按文件路径用PIL读取图片，经按exif转换图片方向，转为指定的format格式的numpy def read_image(file_name, format=None): # Raise an error if the image does not match the size specified in the dict. def check_image_size(dataset_dict, image): # Apply transformations to the proposals in dataset_dict, if any. def transform_proposals(dataset_dict, image_shape, transforms, *, proposal_topk, min_box_size=0): def transform_instance_annotations( annotation, transforms, image_size, *, keypoint_hflip_indices=None ): def transform_keypoint_annotations(keypoints, transforms, image_size, keypoint_hflip_indices=None): def annotations_to_instances(annos, image_size, mask_format="polygon"): def annotations_to_instances_rotated(annos, image_size): def filter_empty_instances(instances, by_box=True, by_mask=True, box_threshold=1e-5): def create_keypoint_hflip_indices(dataset_names): def gen_crop_transform_with_instance(crop_size, image_size, instance): def check_metadata_consistency(key, dataset_names): def build_augmentation(cfg, is_train): -
detectron2/config/defaults.py , 616行
- Convention about Training / Test specific parameters, 训练/测试参数.
- 几个重要的参数
# Path (a file path, or URL like detectron2://.., https://..) to a checkpoint file to be loaded to the model. You can find available models in the model zoo. # 模型的路径，即可本地文件路径，也可以 detectron2://, https://开头的URL _C.MODEL.WEIGHTS = "" # Tf True, when working on datasets that have instance annotations, the # training dataloader will filter out images without associated annotations cfg.DATALOADER.FILTER_EMPTY_ANNOTATIONS -
/home/xyz/.local/lib/python3.7/site-packages/fvcore/common/file_io.py
- def get_cache_dir , $FVCORE_CACHE, 否则 ~/.torch/fvcore_cache
def get_cache_dir(cache_dir: Optional[str] = None) -> str: if cache_dir is None: cache_dir = os.path.expanduser( os.getenv("FVCORE_CACHE", "~/.torch/fvcore_cache") ) return cache_dir -
/home/xyz/.local/lib/python3.7/site-packages/fvcore/common/download.py
- def download ,下载文件
-
demo\demo.py
-
是一个做inference demo
-
使用说明见：GETTING_STARTED.md
-
设置进程启动方式,见Python并行开发
import multiprocessing as mp mp.set_start_method("spawn", force=True)# 初始化 DefaultPredictor(cfg): 按cfg构建模型 demo = VisualizationDemo(cfg) -
一次成功运行日志
(base) xyz@ubuntu:~/work/git/detectron2$ cd /home/xyz/work/git/detectron2 ; env /home/xyz/anaconda3/bin/python /home/xyz/.vscode/extensions/ms-python.python-2020.7.96456/pythonFiles/lib/python/debugpy/launcher 39641 -- /home/xyz/work/git/detectron2/demo/demo.py --config-file /home/xyz/work/git/detectron2/configs/COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml --input /home/xyz/work/images/ma.jpg --opts MODEL.WEIGHTS detectron2://COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x/137849600/model_final_f10217.pkl MODEL.DEVICE cpu [07/24 16:43:30 detectron2]: Arguments: Namespace(confidence_threshold=0.5, config_file='/home/xyz/work/git/detectron2/configs/COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml', input=['/home/xyz/work/images/ma.jpg'], opts=['MODEL.WEIGHTS', 'detectron2://COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x/137849600/model_final_f10217.pkl', 'MODEL.DEVICE', 'cpu'], output=None, video_input=None, webcam=False) [07/24 16:43:31 fvcore.common.checkpoint]: Loading checkpoint from detectron2://COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x/137849600/model_final_f10217.pkl [07/24 16:43:31 fvcore.common.file_io]: URL https://dl.fbaipublicfiles.com/detectron2/COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x/137849600/model_final_f10217.pkl cached in /home/xyz/.torch/fvcore_cache/detectron2/COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x/137849600/model_final_f10217.pkl [07/24 16:43:31 fvcore.common.checkpoint]: Reading a file from 'Detectron2 Model Zoo' [07/24 16:43:38 detectron2]: /home/xyz/work/images/ma.jpg: detected 25 instances in 6.69s -
-
demo\predictor.py
- 预测Inference图片
-
detectron2\engine\defaults.py
- 实现 def default_argument_parser ,创建一个适合detectron2的ArgumentParser，设定了一些必要的参数，比如：–config-file, –eval-only, –num-gpus , –num-machines , –machine-rank, –dist-url , opts
- 实现 def default_setup
- 创建了一个detectron2 logger
- 打印日志: environment, cmdline arguments, and config
- 保存 config 到输出目录
- 实现 class DefaultPredictor
- Load checkpoint from
cfg.MODEL.WEIGHTS. - Always take BGR image as the input and apply conversion defined by
cfg.INPUT.FORMAT. - Apply resizing defined by
cfg.INPUT.{MIN,MAX}_SIZE_TEST. - Take one input image and produce a single output, instead of a batch.
- Load checkpoint from
- 实现 class DefaultTrainer(SimpleTrainer) : init :根据cfg构建模型，设为eval，而不是默认的train, 获取checkpointer(下载并加载权重weights文件pkl); call : 输入图片，获得预测结果。
- IMS_PER_BATCH 应该是 world_size的整数倍,world_size与num_workders有关系。
assert ( cfg.SOLVER.IMS_PER_BATCH % old_world_size == 0 ), "Invalid REFERENCE_WORLD_SIZE in config!"- def auto_scale_workers 自动修改了 cfg.SOLVER.IMS_PER_BATCH, cfg.SOLVER.BASE_LR, cfg.SOLVER.MAX_ITER ,cfg.SOLVER.REFERENCE_WORLD_SIZE 等参数
-
/home/xyz/.local/lib/python3.7/site-packages/detectron2/evaluation/coco_evaluation.py
-
class COCOEvaluator(DatasetEvaluator):
Evaluate AR for object proposals, AP for instance detection/segmentation, AP
for keypoint detection outputs using COCO’s metrics.
See http://cocodataset.org/#detection-eval and
http://cocodataset.org/#keypoints-eval to understand its metrics.
In addition to COCO, this evaluator is able to support any bounding box detection,
instance segmentation, or keypoint detection dataset.
-
-
detectron2\engine\train_loop.py
- 实现 class HookBase
- 钩子
hook.before_train() for iter in range(start_iter, max_iter): hook.before_step() trainer.run_step() hook.after_step() hook.after_train() - 实现 class TrainerBase ,一个训练基类，def run_step 没有实现
- 实现 class SimpleTrainer(TrainerBase) ,一个简单的大众化训练
- 实现 class HookBase
-
detectron2\modeling\backbone\build.py
按cfg.MODEL.BACKBONE.NAME构建一个backbone, 包含resnet_fpn_backbone,retinanet_resnet_fpn_backbone, resnet_backbone, trident_resnet_backbone
def build_backbone(cfg, input_shape=None): -
detectron2\modeling\backbone\backbone.py
提供网络骨架(backbone)基类
class Backbone(nn.Module, metaclass=ABCMeta): -
detectron2\modeling\backbone\resnet.py
class ResNet(Backbone): """ Implement :paper:`ResNet`. """ @BACKBONE_REGISTRY.register() def build_resnet_backbone(cfg, input_shape): -
/home/xyz/.local/lib/python3.7/site-packages/detectron2/modeling/backbone/fpn.py
__all__ = ["build_resnet_fpn_backbone", "build_retinanet_resnet_fpn_backbone", "FPN"] class FPN(Backbone): """ This module implements :paper:`FPN`. It creates pyramid features built on top of some input feature maps. """ -
projects\DensePose\densepose\data\datasets\builtin.py
from .chimpnsee import register_dataset as register_chimpnsee_dataset from .coco import BASE_DATASETS as BASE_COCO_DATASETS from .coco import DATASETS as COCO_DATASETS from .coco import register_datasets as register_coco_datasets DEFAULT_DATASETS_ROOT = "datasets" register_coco_datasets(COCO_DATASETS, DEFAULT_DATASETS_ROOT) register_coco_datasets(BASE_COCO_DATASETS, DEFAULT_DATASETS_ROOT) register_chimpnsee_dataset(DEFAULT_DATASETS_ROOT) -
projects\DensePose\densepose\data\datasets\coco.py
@dataclass #python 3.7以上支持的数据类型 ，Coco数据集的一个信息描述 class CocoDatasetInfo: name: str images_root: str annotations_fpath: str DATASETS = [ CocoDatasetInfo( name="densepose_coco_2014_train", images_root="coco/train2014", annotations_fpath="coco/annotations/densepose_train2014.json", ), ] # Loads a JSON file with annotations in COCO instances format. def load_coco_json(annotations_json_file: str, image_root: str, dataset_name: str): -
detectron2\engine\hooks.py
训练过程中使用的一些钩子
__all__ = [ "CallbackHook", "IterationTimer", #记录Iteration的间隔时间 "PeriodicWriter", "PeriodicCheckpointer", #定期保存checkpoint "LRScheduler", # 动态修改lr "AutogradProfiler", "EvalHook", #evalation "PreciseBN", ] -
detectron2/engine/launch.py
- Launch multi-gpu or distributed training.
def launch(main_func, num_gpus_per_machine, num_machines=1, machine_rank=0, dist_url=None, args=()): """ Launch multi-gpu or distributed training. This function must be called on all machines involved in the training. It will spawn child processes (defined by ``num_gpus_per_machine`) on each machine. Args: main_func: a function that will be called by `main_func(*args)` num_gpus_per_machine (int): number of GPUs per machine num_machines (int): the total number of machines machine_rank (int): the rank of this machine dist_url (str): url to connect to for distributed jobs, including protocol e.g. "tcp://127.0.0.1:8686". Can be set to "auto" to automatically select a free port on localhost args (tuple): arguments passed to main_func """ world_size = num_machines * num_gpus_per_machine if world_size > 1: # https://github.com/pytorch/pytorch/pull/14391 # TODO prctl in spawned processes if dist_url == "auto": assert num_machines == 1, "dist_url=auto not supported in multi-machine jobs." port = _find_free_port() dist_url = f"tcp://127.0.0.1:{port}" if num_machines > 1 and dist_url.startswith("file://"): logger = logging.getLogger(__name__) logger.warning( "file:// is not a reliable init_method in multi-machine jobs. Prefer tcp://" ) mp.spawn( _distributed_worker, nprocs=num_gpus_per_machine, args=(main_func, world_size, num_gpus_per_machine, machine_rank, dist_url, args), daemon=False, ) else: main_func(*args) -
detectron2\modeling\meta_arch\build.py
根据register的网络模型名字，加载模型进内存.
from detectron2.utils.registry import Registry META_ARCH_REGISTRY = Registry("META_ARCH") # noqa F401 isort:skip def build_model(cfg): """ Build the whole model architecture, defined by ``cfg.MODEL.META_ARCHITECTURE``. Note that it does not load any weights from ``cfg``. """ meta_arch = cfg.MODEL.META_ARCHITECTURE model = META_ARCH_REGISTRY.get(meta_arch)(cfg) model.to(torch.device(cfg.MODEL.DEVICE)) return model -
detectron2\modeling\meta_arch\rcnn.py
把 class GeneralizedRCNN 实例化一个对象注册进 META_ARCH_REGISTRY
from .build import META_ARCH_REGISTRY @META_ARCH_REGISTRY.register() class GeneralizedRCNN(nn.Module): """ Generalized R-CNN. Any models that contains the following three components: 1. Per-image feature extraction (aka backbone) 2. Region proposal generation 3. Per-region feature extraction and prediction """ @configurable def __init__( self, *, backbone: Backbone, proposal_generator: nn.Module, roi_heads: nn.Module, pixel_mean: Tuple[float], pixel_std: Tuple[float], input_format: Optional[str] = None, vis_period: int = 0, ): """ NOTE: this interface is experimental. Args: backbone: a backbone module, must follow detectron2's backbone interface proposal_generator: a module that generates proposals using backbone features roi_heads: a ROI head that performs per-region computation pixel_mean, pixel_std: list or tuple with #channels element, representing the per-channel mean and std to be used to normalize the input image input_format: describe the meaning of channels of input. Needed by visualization vis_period: the period to run visualization. Set to 0 to disable. """ super().__init__() self.backbone = backbone self.proposal_generator = proposal_generator self.roi_heads = roi_heads self.input_format = input_format self.vis_period = vis_period if vis_period > 0: assert input_format is not None, "input_format is required for visualization!" self.register_buffer("pixel_mean", torch.Tensor(pixel_mean).view(-1, 1, 1)) self.register_buffer("pixel_std", torch.Tensor(pixel_std).view(-1, 1, 1)) assert ( self.pixel_mean.shape == self.pixel_std.shape ), f"{self.pixel_mean} and {self.pixel_std} have different shapes!" -
detectron2\checkpoint\detection_checkpoint.py
-
实现 class DetectionCheckpointer
Same as :class:
Checkpointer, but is able to handle models in detectron & detectron2 model zoo, and apply conversions for legacy models.
加载模型 pklclass DetectionCheckpointer(Checkpointer) def _load_file(self, filename): #读进模型文件(*.pkl)，存在内存中 def _load_model(self, checkpoint)
-
-
site-packages/fvcore/common/checkpoint.py
加载/保存模型权重文件；训练结束后，保存权重模型 model_final
class PeriodicCheckpointer: def step(self, iteration: int, **kwargs: Any) -> None: self.checkpointer.save("model_final", **additional_state) class Checkpointer(object): def save(self, name: str, **kwargs: Dict[str, str]) -> None: # 保存模型 with PathManager.open(save_file, "wb") as f: torch.save(data, f) def load(self, path: str, checkpointables: Optional[List[str]] = None) -> object: # 加载模型 -
detectron2\solver\build.py
- 实现 def build_optimizer ,build_lr_scheduler
def build_optimizer(cfg: CfgNode, model: torch.nn.Module) -> torch.optim.Optimizer: """ Build an optimizer from config. """ def build_lr_scheduler( cfg: CfgNode, optimizer: torch.optim.Optimizer ) -> torch.optim.lr_scheduler._LRScheduler: """ Build a LR scheduler from config. """ - def build_optimizer(cfg: CfgNode, model: torch.nn.Module) -> torch.optim.Optimizer:
- 实现 def build_optimizer ,build_lr_scheduler
-
detectron2/config/config.py
-
实现 class cfgNode()
-
def get_cfg() , 从defaults 中初始化了所有train/test的参数
-
def configurable(init_func)
def configurable(init_func): """ Decorate a class's __init__ method so that it can be called with a CfgNode object using the class's from_config classmethod. Examples: :: class A: @configurable def __init__(self, a, b=2, c=3): pass @classmethod def from_config(cls, cfg): # Returns kwargs to be passed to __init__ return {"a": cfg.A, "b": cfg.B} a1 = A(a=1, b=2) # regular construction a2 = A(cfg) # construct with a cfg a3 = A(cfg, b=3, c=4) # construct with extra overwrite """
-
-
docs\tutorials\datasets.md
-
如何用detectron2的data loaders 加载自定义的数据集
def get_coco_d_instances_meta(json_file): logger = logging.getLogger("detectron2.trainer") from fvcore.common.timer import Timer from pycocotools.coco import COCO timer = Timer() coco_api = COCO(json_file) if timer.seconds() > 1: logger.info("Loading {} takes {:.2f} seconds.".format(json_file, timer.seconds())) cat_ids = sorted(coco_api.getCatIds()) cats = coco_api.loadCats(cat_ids) D_CATEGORIES = [{"color": c["color"] if "color" in c else [0,0,142], "isthing": 1, "id":c["id"] , "name": c["name"]} for c in sorted(cats, key=lambda x: x["id"])] thing_ids = [k["id"] for k in D_CATEGORIES if k["isthing"] == 1] thing_colors = [k["color"] for k in D_CATEGORIES if k["isthing"] == 1] # Mapping from the incontiguous COCO-D category id to an id in [0, 4] thing_dataset_id_to_contiguous_id = {k: i for i, k in enumerate(thing_ids)} thing_classes = [k["name"] for k in D_CATEGORIES if k["isthing"] == 1] ret = { "thing_dataset_id_to_contiguous_id": thing_dataset_id_to_contiguous_id, "thing_classes": thing_classes, "thing_colors": thing_colors, } logger.info(f"coco_d_instances_meta: {ret}") return ret def register_dataset_detect(): _PREDEFINED_SPLITS_D = { "Train": ( "test_d2/dataset/defect_coco/Train", "test_d2/dataset/defect_coco/Train/train.json"), "Validation": ("test_d2/dataset/defect_coco/Validation" , "test_d2/dataset/defect_coco/Validation/val.json"), } for dataset_name, (image_root, json_file) in _PREDEFINED_SPLITS_D.items(): # Assume pre-defined datasets live in `./datasets`. metadata = get_coco_d_instances_meta(json_file) register_coco_instances(dataset_name, metadata, json_file, image_root)
-
-
site-packages\fvcore\common\registry.py
定义Registry ，提供 name -> object 的mapping. 比如：
from detectron2.utils.registry import Registry META_ARCH_REGISTRY = Registry("META_ARCH") @META_ARCH_REGISTRY.register() class GeneralizedRCNN(nn.Module): meta_arch = cfg.MODEL.META_ARCHITECTURE model = META_ARCH_REGISTRY.get(meta_arch)(cfg) -
docs\tutorials\write-models.md
各种注册，开发者可用写各种类或函数，通过Registry方法，可嵌入到detectron2的代码执行流程中。如此就可在开发者自己的project中编写代码，并在detectron2库中适当的时机被调用。而不必修改detectron2代码。detectron2包含的Registry有：-
ANCHOR_GENERATOR
- class DefaultAnchorGenerator(nn.Module):
- class RotatedAnchorGenerator(nn.Module):
-
BACKBONE
- def build_resnet_fpn_backbone(cfg, input_shape: ShapeSpec):
- def build_retinanet_resnet_fpn_backbone(cfg, input_shape: ShapeSpec):
- def build_resnet_backbone(cfg, input_shape):
- def build_trident_resnet_backbone(cfg, input_shape):
-
META_ARCH
- class PanopticFPN(nn.Module):
- class GeneralizedRCNN(nn.Module):
- class ProposalNetwork(nn.Module):
- class RetinaNet(nn.Module):
- class SemanticSegmentor(nn.Module):
- class TensorMask(nn.Module):
-
SEM_SEG_HEADS
- class SemSegFPNHead(nn.Module):
- class PointRendSemSegHead(nn.Module):
-
RPN_HEAD
- class StandardRPNHead(nn.Module):
-
PROPOSAL_GENERATOR
- class RPN(nn.Module):
- class RRPN(RPN):
- class TridentRPN(RPN):
-
ROI_BOX_HEAD
- class FastRCNNConvFCHead(nn.Module):
-
ROI_HEADS
- class CascadeROIHeads(StandardROIHeads):
- class Res5ROIHeads(ROIHeads):
- class StandardROIHeads(ROIHeads):
- class RROIHeads(StandardROIHeads):
- class DensePoseROIHeads(StandardROIHeads): projects\DensePose\densepose\roi_head.py
- class PointRendROIHeads(StandardROIHeads):
- class TridentRes5ROIHeads(Res5ROIHeads):
- class TridentStandardROIHeads(StandardROIHeads):
-
ROI_KEYPOINT_HEAD
- class KRCNNConvDeconvUpsampleHead(BaseKeypointRCNNHead):
-
ROI_MASK_HEAD
- class MaskRCNNConvUpsampleHead(BaseMaskRCNNHead):
- class CoarseMaskHead(nn.Module):
-
ROI_DENSEPOSE_HEAD
- class DensePoseDeepLabHead(nn.Module):
- class DensePoseV1ConvXHead(nn.Module):
-
POINT_HEAD
- class StandardPointHead(nn.Module):
-
-
detectron2/layers/wrappers.py
封装替代torch的一些类和函数class Conv2d(torch.nn.Conv2d): """ A wrapper around :class:`torch.nn.Conv2d` to support empty inputs and more features. """ if TORCH_VERSION > (1, 4): ConvTranspose2d = torch.nn.ConvTranspose2d else: class ConvTranspose2d(torch.nn.ConvTranspose2d): """ A wrapper around :class:`torch.nn.ConvTranspose2d` to support zero-size tensor. """ if TORCH_VERSION > (1, 4): BatchNorm2d = torch.nn.BatchNorm2d else: class BatchNorm2d(torch.nn.BatchNorm2d): """ A wrapper around :class:`torch.nn.BatchNorm2d` to support zero-size tensor. """ if TORCH_VERSION > (1, 5): Linear = torch.nn.Linear else: class Linear(torch.nn.Linear): """ A wrapper around :class:`torch.nn.Linear` to support empty inputs and more features. Because of https://github.com/pytorch/pytorch/issues/34202 """ def interpolate(input, size=None, scale_factor=None, mode="nearest", align_corners=None): """ A wrapper around :func:`torch.nn.functional.interpolate` to support zero-size tensor. """ -
/home/xyz/.local/lib/python3.7/site-packages/detectron2/layers/init.py
Detectron2 提供的模型 layers
# Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved from .batch_norm import FrozenBatchNorm2d, get_norm, NaiveSyncBatchNorm from .deform_conv import DeformConv, ModulatedDeformConv from .mask_ops import paste_masks_in_image from .nms import batched_nms, batched_nms_rotated, nms, nms_rotated from .roi_align import ROIAlign, roi_align from .roi_align_rotated import ROIAlignRotated, roi_align_rotated from .shape_spec import ShapeSpec from .wrappers import BatchNorm2d, Conv2d, ConvTranspose2d, cat, interpolate, Linear, nonzero_tuple from .blocks import CNNBlockBase __all__ = [k for k in globals().keys() if not k.startswith("_")] -
detectron2/layers/csrc/vision.cpp
为Detectron2提供的C++ extension.
PYBIND11_MODULE(TORCH_EXTENSION_NAME, m) { m.def("get_compiler_version", &get_compiler_version, "get_compiler_version"); m.def("get_cuda_version", &get_cuda_version, "get_cuda_version"); m.def("box_iou_rotated", &box_iou_rotated, "IoU for rotated boxes"); m.def("deform_conv_forward", &deform_conv_forward, "deform_conv_forward"); m.def( "deform_conv_backward_input", &deform_conv_backward_input, "deform_conv_backward_input"); m.def( "deform_conv_backward_filter", &deform_conv_backward_filter, "deform_conv_backward_filter"); m.def( "modulated_deform_conv_forward", &modulated_deform_conv_forward, "modulated_deform_conv_forward"); m.def( "modulated_deform_conv_backward", &modulated_deform_conv_backward, "modulated_deform_conv_backward"); m.def("nms_rotated", &nms_rotated, "NMS for rotated boxes"); m.def("roi_align_forward", &ROIAlign_forward, "ROIAlign_forward"); m.def("roi_align_backward", &ROIAlign_backward, "ROIAlign_backward"); m.def( "roi_align_rotated_forward", &ROIAlignRotated_forward, "Forward pass for Rotated ROI-Align Operator"); m.def( "roi_align_rotated_backward", &ROIAlignRotated_backward, "Backward pass for Rotated ROI-Align Operator"); m.def("COCOevalAccumulate", &COCOeval::Accumulate, "COCOeval::Accumulate"); m.def( "COCOevalEvaluateImages", &COCOeval::EvaluateImages, "COCOeval::EvaluateImages"); pybind11::class_<COCOeval::InstanceAnnotation>(m, "InstanceAnnotation") .def(pybind11::init<uint64_t, double, double, bool, bool>()); pybind11::class_<COCOeval::ImageEvaluation>(m, "ImageEvaluation") .def(pybind11::init<>()); } -
setup.py
自定义C++/CUDA扩展，供Python调用，参考CUSTOM C++ AND CUDA EXTENSIONS
6. Detectron2已集成模型
见：MODEL_ZOO.md
7. 参考链接
更多推荐
14
0- 0
已为社区贡献1条内容
所有评论(0)