Depth calculation error with the StereoDepthSensor

[henryhcliu]

System:
Python: 3.11.0
Platform: Linux-5.15.0-130-generic-x86_64-with-glibc2.39
sapien: 3.0.1

Describe the bug
When running the demo to generate the depth image from the two IR sensors, the result shows all 0 entries.

To Reproduce
Steps to reproduce the behavior (use pastebin for code):
Run the code at https://github.com/haosulab/SAPIEN/blob/dev/manualtest/stereodepth.py

Expected behavior
The real depth image for the rendered IR sensors.

Screenshots
The original plot:

After bug fixing by

    def compute_depth(self, bbox_start: tuple = None, bbox_size: tuple = None):
        # left_cuda = self._cam_ir_l.get_picture_cuda("Color")
        # right_cuda = self._cam_ir_r.get_picture_cuda("Color")
        left_ir, right_ir = self.get_ir()
        left_cuda = (left_ir * 255).astype(np.uint8)
        right_cuda = (right_ir * 255).astype(np.uint8)
        self._ss.compute(left_cuda, right_cuda, bbox_start, bbox_size)

Additional context
Please fix the official code's bug accordingly.

[fbxiang]

I believe your code would first download the images to the CPU and then upload them to the GPU to compute depth. This would probably be quite a bit slower.
I cannot reproduce this issue on my machine, but maybe you can do some more investigations here. For example, how does the images from get_picture_cuda look like? Are they the same as the ones fetched from get_ir?