Question about 2D motion representation

[OrigamiStationery]

Hello! Thank you for your marvelous work!
MAS represent a 2D motion as a (T,J,2) joint sequences. However, a vector that includes rotation and velocity is widely applied in the 3D motion representation, for example the 263-D vector for a HumanML3D-type (T,22,3) joints. I wonder why in this work, this motion representation method that can help the network learn better was not derived to 2D motions.

[roykapon]

Hello @OrigamiStationery, this is a great question! First, we did not test this option so it could actually be a viable addition. However, the main issue with 2D motions is that they do not preserve bone lengths (for inatance, if a bone faces straight towards the camera it would appear to have a length of 0)... This means that we would not gain much from this representation as we would also need to represent the bone lengths. You are more than welcome to try this and see if helps!

Your question also sparked another idea in my head: We could use this representaition for the optimizing 3D motion. This will force the bone lengths to be consistent and prevent collapses. However, it is not guaranteed to work since in early stages of the motion optimization the 3d motion is not coherent. We might try this in the future as it is a simple addition.

Thank you @OrigamiStationery for the interest and the intriguing questions!