Depth-aware CNN for RGB-D Segmentation [Arxiv]

Installation

Install Pytorch, dominate, TensorBoardX.

The depth-aware convolution and depth-aware average pooling operations are under folder models/ops, to build them, simply use sh make.sh to compile.

Training

#!./scripts/train.sh
python train.py \
--name nyuv2_VGGdeeplab_depthconv \
--dataset_mode nyuv2 \
--flip --scale --crop --colorjitter \
--depthconv \
--list dataset/lists/nyuv2/train.lst \
--vallist dataset/lists/nyuv2/val.lst

Pretrained Model can be found here.

Testing

#!./scripts/test.sh
python test.py \
--name nyuv2_VGGdeeplab_depthconv \
--dataset_mode nyuv2 \
--list dataset/lists/nyuv2/test.lst \
--depthconv \
--how_many 0

Need to put FloorPlan1_physics under the root directory.

python test.py --list "../FloorPlan1_physics/images.hdf5" --dataset_mode ai2thor --pretrained_model checkpoints/label2city/model

Citation

If you find this work useful, please consider citing:

    @inproceedings{wang2018depthconv,
        title={Depth-aware CNN for RGB-D Segmentation},
        author={Wang, Weiyue and Neumann, Ulrich},
        booktitle={ECCV},
        year={2018}
    }

Acknowledgemets

The visulization code is borrowed from pytorch-CycleGAN-and-pix2pix. Here is a pytorch implementation of DeepLab.