Same dataset, 100% for "single object" but 0% for "multi object"

[MohamadJaber1]

Hello everyone,

Thank you @btekin for sharing with us this amazing work. I have succeeded in implementing your paper with single object for my robotic manipulation task. However, I have difficulties with multiple object pose estimation.

I would like to tell you what I am suspecting in and please let me know if there is any other changes for the multi object besides the ones I will mention below.

• The ground truth bounding box is different than that of single object. It will be shown below, please let me know if thats true and how it should be fixed?
• In the main cfg file (as of occlusion.data), I keeping the directories of one object with its labels and then relying on the get_add_objs(--) function for calling the other object and training on its images as well. Is it right?

"More clarification":
For simplicity, I have generated a small dataset of two objects at the same scene with one pose only and for each object dataset, I annontated its respective object in the scene. After training for several epochs, I am showing the output from the validation set where its converge well.

This model is trained for detecting and estimating the pose of a cat.

And this one is trained for detecting and estimating the pose of a cam.

Now, I am using Python 3.6.8, and PyTorch 1.0.1.
I have update the following lines in these files according to #30 and btekin/singleshotpose

• train_multi.py (configs)
• region_multi.py
• darknet_multi.py
• dataset_multi.py
• image_multi.py
  changed:
  

  singleshotpose/multi_obj_pose_estimation/image_multi.py

  Lines 24 to 27 in 766d5ec

  	if objname == 'ape':
  	add_objs = ['can', 'cat', 'duck', 'glue', 'holepuncher', 'iron', 'phone'] # eggbox
  	elif objname == 'benchvise':
  	add_objs = ['ape', 'can', 'cat', 'driller', 'duck', 'glue', 'holepuncher']

  ...
  to:
  if objname == 'cat':
  add_objs = ['cam']
  elif objname == 'cam':
  add_objs = ['cat']

Regarding cfg files, I have main one same as occlusion.data but for two objects. And two individual cfg as ape.data.
The result of the model is:
Testing "cat"

Testing "cam"

[lmhlll]

hello, How do I display image results?
"Python valid. Py" does not show the resulting image.

[MohamadJaber1]

hello, How do I display image results?
"Python valid. Py" does not show the resulting image.

https://github.com/microsoft/singleshotpose/blob/766d5eced25df6f0c438c6414c07b8b686c0558e/valid.ipynb

[lmhlll]

hello, How do I display image results?
"Python valid. Py" does not show the resulting image.

https://github.com/microsoft/singleshotpose/blob/766d5eced25df6f0c438c6414c07b8b686c0558e/valid.ipynb

Thank you! 2019/08/28

[MrLuer]

hi， I want my robot grab object，but I dont know how to get the position of object relative to camera，for detail see #118
if you can reply I will very appreciate @MohamadJaber1

[MohamadJaber1]

hi， I want my robot grab object，but I dont know how to get the position of object relative to camera，for detail see #118
if you can reply I will very appreciate @MohamadJaber1

Of course you can. This project (singleshotpose) by @btekin will allow you to get the 6 DoF object pose (position + orientation) with respect to the camera frame. After that you will apply some transformation w.r.t. world coordinate system (robotic environment). I advise to have some literature review before starting and good luck.

[xiaoxiongmaoxuexi]

@MohamadJaber1 Thank you for sharing. I have encountered the same problem as you. Have you solved this problem. Thank you for your answer.

Same dataset, 100% for "single object" but 0% for "multi object"