一、准备

1.1 SlowFast代码

官网地址:

https://github.com/facebookresearch/SlowFast

1.2 SlowFast环境准备

很多教程都说按照官方的 Installation来,本人尝试了很多次,很多包压根就装不上,如果有相同经历的朋友可以跟着我的步骤来,亲身经历且有效。

1.2.1 创建虚拟环境

 conda create -n SlowFast python=3.8

以下模块的安装均采用离线方式,git到本机然后传到服务器,再进行安装。

1.2.2 安装Pytorch

conda install pytorch==1.8.0 torchvision==0.9.0 torchaudio==0.8.0 cudatoolkit=11.1 -c pytorch -c conda-forge

1.2.3 安装cocoapi

(1)下载源码:https://github.com/philferriere/cocoapi

(2)编译源码: sudo /home/wzhou/anaconda3/envs/SlowFast/bin/python setup.py install

1.2.4 安装fvcore

(1)下载源码:https://github.com/facebookresearch/fvcore

(2)编译源码:python setup.py build --force develop

1.2.5 安装detectron2

是所有模块中安装最麻烦的一步

(1)下载源码:https://github.com/facebookresearch/detectron2

(2)将下载的包detectron2复制一份到site-packages下

cp -r detectron2/ /home/wzhou/anaconda3/envs/SlowFast/lib/python3.8/site-packages

(3)修改以下内容(共6处)参考链接

第一处:修改detectron2的setup.py中最末尾的部分

第二处:修改site-packages\torch\utils\cpp_extension.py

(site-packages所在路径:/home/wzhou/anaconda3/envs/SlowFast/lib/python3.8/site-packages

第三处:修改site-packages\torch\include\torch\csrc\jit\runtime\argumenta_spec.h

第四处:detectron2文件夹下:\detectron2\layers\csrc\ROIAlignRotated\ROIAlignRotated_cuda.cu,将所有的ceil改为ceilf

第五处:detectron2文件夹下:\detectron2\detectron2\layers\csrc\deformable\deform_conv_cuda_kernel.cu,将所有的floor改为floorf

第六处:将detectron2\layers\csrc\deformable下三个文件的AT_CHEK全换成TORCH_CHECK

(4)编译源码:  sudo /home/wzhou/anaconda3/envs/SlowFast/bin/python setup.py install

1.2.6 安装slowfast

(1)下载源码:https://github.com/facebookresearch/SlowFast

  (2)   修改文件:  将setup.py里的PIL改为pillow

(3)编译源码: sudo /home/wzhou/anaconda3/envs/SlowFast/bin/python setup.py install

1.2.7 安装 pythorchvideo

 (1)下载源码:https://github.com/facebookresearch/pytorchvideo

 (2)编译源码:pip install -e .

二、运行测试

2.1 制作label

建一个json文件,我命名为ava.json,保存至目录:demo\AVA\ava.json, 内容如下:

{"bend/bow (at the waist)": 0, "crawl": 1, "crouch/kneel": 2, "dance": 3, 
  "fall down": 4, "get up": 5, "jump/leap": 6, "lie/sleep": 7, "martial art": 8, 
  "run/jog": 9, "sit": 10, "stand": 11, "swim": 12, "walk": 13, "answer phone": 14, 
  "brush teeth": 15, "carry/hold (an object)": 16, "catch (an object)": 17, "chop": 18, 
  "climb (e.g., a mountain)": 19, "clink glass": 20, "close (e.g., a door, a box)": 21, 
  "cook": 22, "cut": 23, "dig": 24, "dress/put on clothing": 25, "drink": 26, 
  "drive (e.g., a car, a truck)": 27, "eat": 28, "enter": 29, "exit": 30, "extract": 31, 
  "fishing": 32, "hit (an object)": 33, "kick (an object)": 34, "lift/pick up": 35, 
  "listen (e.g., to music)": 36, "open (e.g., a window, a car door)": 37, 
  "paint": 38, "play board game": 39, "play musical instrument": 40, 
  "play with pets": 41, "point to (an object)": 42, "press": 43, 
  "pull (an object)": 44, "push (an object)": 45, "put down": 46, 
  "read": 47, "ride (e.g., a bike, a car, a horse)": 48, "row boat": 49, 
  "sail boat": 50, "shoot": 51, "shovel": 52, "smoke": 53, "stir": 54, 
  "take a photo": 55, "text on/look at a cellphone": 56, "throw": 57, 
  "touch (an object)": 58, "turn (e.g., a screwdriver)": 59, "watch (e.g., TV)": 60,
  "work on a computer": 61, "write": 62, "fight/hit (a person)": 63, 
  "give/serve (an object) to (a person)": 64, "grab (a person)": 65, 
  "hand clap": 66, "hand shake": 67, "hand wave": 68, "hug (a person)": 69, 
  "kick (a person)": 70, "kiss (a person)": 71, "lift (a person)": 72, 
  "listen to (a person)": 73, "play with kids": 74, "push (another person)": 75, 
  "sing to (e.g., self, a person, a group)": 76, "take (an object) from (a person)": 77, 
  "talk to (e.g., self, a person, a group)": 78, "watch (a person)": 79}

2.2 更改配置文件

修改SlowFast/demo/AVA/SLOWFAST_32x2_R101_50_50.yaml

TRAIN:
  ENABLE: False
  DATASET: ava
  BATCH_SIZE: 16
  EVAL_PERIOD: 1
  CHECKPOINT_PERIOD: 1
  AUTO_RESUME: True
  CHECKPOINT_FILE_PATH: '/home/wzhou/way/llwang/SFast/configs/AVA/c2/SLOWFAST_32x2_R101_50_50.pkl'  #path to pretrain model
  CHECKPOINT_TYPE: pytorch
DATA:
  NUM_FRAMES: 32
  SAMPLING_RATE: 2
  TRAIN_JITTER_SCALES: [256, 320]
  TRAIN_CROP_SIZE: 224
  TEST_CROP_SIZE: 256
  INPUT_CHANNEL_NUM: [3, 3]
DETECTION:
  ENABLE: True
  ALIGNED: False
AVA:
  BGR: False
  DETECTION_SCORE_THRESH: 0.8
  TEST_PREDICT_BOX_LISTS: ["person_box_67091280_iou90/ava_detection_val_boxes_and_labels.csv"]
SLOWFAST:
  ALPHA: 4
  BETA_INV: 8
  FUSION_CONV_CHANNEL_RATIO: 2
  FUSION_KERNEL_SZ: 5
RESNET:
  ZERO_INIT_FINAL_BN: True
  WIDTH_PER_GROUP: 64
  NUM_GROUPS: 1
  DEPTH: 101
  TRANS_FUNC: bottleneck_transform
  STRIDE_1X1: False
  NUM_BLOCK_TEMP_KERNEL: [[3, 3], [4, 4], [6, 6], [3, 3]]
  SPATIAL_DILATIONS: [[1, 1], [1, 1], [1, 1], [2, 2]]
  SPATIAL_STRIDES: [[1, 1], [2, 2], [2, 2], [1, 1]]
NONLOCAL:
  LOCATION: [[[], []], [[], []], [[6, 13, 20], []], [[], []]]
  GROUP: [[1, 1], [1, 1], [1, 1], [1, 1]]
  INSTANTIATION: dot_product
  POOL: [[[2, 2, 2], [2, 2, 2]], [[2, 2, 2], [2, 2, 2]], [[2, 2, 2], [2, 2, 2]], [[2, 2, 2], [2, 2, 2]]]
BN:
  USE_PRECISE_STATS: False
  NUM_BATCHES_PRECISE: 200
SOLVER:
  MOMENTUM: 0.9
  WEIGHT_DECAY: 1e-7
  OPTIMIZING_METHOD: sgd
MODEL:
  NUM_CLASSES: 80
  ARCH: slowfast
  MODEL_NAME: SlowFast
  LOSS_FUNC: bce
  DROPOUT_RATE: 0.5
  HEAD_ACT: sigmoid
TEST:
  ENABLE: False
  DATASET: ava
  BATCH_SIZE: 8
DATA_LOADER:
  NUM_WORKERS: 2
  PIN_MEMORY: True

NUM_GPUS: 1
NUM_SHARDS: 1
RNG_SEED: 0
#OUTPUT_DIR: .
#TENSORBOARD:
#  MODEL_VIS:
#    TOPK: 2
DEMO:
  ENABLE: True
  LABEL_FILE_PATH: "/home/wzhou/way/llwang/SFast/demo/AVA/ava.json"
  INPUT_VIDEO: "/home/wzhou/way/llwang/SFast/Vinput/fight_006.mp4"
  OUTPUT_FILE: "/home/wzhou/way/llwang/SFast/Voutput/fight_006.mp4"

  DETECTRON2_CFG: "COCO-Detection/faster_rcnn_R_50_FPN_3x.yaml"
  DETECTRON2_WEIGHTS: detectron2://COCO-Detection/faster_rcnn_R_50_FPN_3x/137849458/model_final_280758.pkl

注意:这两个文件是我自己建的

2.3 下载权重模型 

下载模型SLOWFAST_32x2_R101_50_50 .pkl 到SlowFast/configs/AVA/c2目录下,下载页面

 

 2.4 运行检测

cd SlowFast

python tools/run_net.py --cfg demo/AVA/SLOWFAST_32x2_R101_50_50.yaml

三、错误总结

        项目运行出来后,感觉也没有那么难搞了,但也确实花费我挺长时间的,就把我遇到的一些问题记录下来,其中不乏有python版本不匹配,以及下载的模块不是官方给定的,导致文件的缺失等等,也方便以后复盘。

3.1 error1

解决:  注释掉(没有任何依据,非常害怕后续的操作报错)

 后续貌似并没有出现什么问题。

3.2 error2

 解决:参考博客

 3.3 error3

 解决:注释掉这4个地方

 3.4 error4

 解决:原因是下载的detectron2不是官方的,导致model_zoo.py里的函数不全有缺失,重新下载官方提供的detectron2模块即可。

四、参考链接

博客1

博客2

博客3

博客4

Logo

更多推荐