I wonder why the extrinsics are set as 0.01 and 4.0 in gen_images_portrait4d.py#L241?

[lvZic]

I konw the fixed extrinsics with (0,01. 4.0) helps to align the head and BG, however the generated image seems not align to the origin source image.

Portrait-4D/portrait4d/gen_images_portrait4d.py

Line 241 in da5eec6

extrinsics[1,3] = 0.01

[YuDeng]

Hi, this parameters are set empirically and can be adjusted.

[lvZic]

Hi, this parameters are set empirically and can be adjusted.

I know the this parameters can be adjusted.
But how the parameters should be set in order that the result image can match the source image. In my test, the head(including neck and shoulder) in result image's position moves slightly from the one in source image.
I set the blending mask by head mask without neck, while the result shows inconsitent between head and neck. I know the neck pose impats all the head region(face, neck and shoulder), so I wonder if there is a way in your method that can generate results with just head animation without shoulder, just like liveportrait?

[lvZic]

e.g: c = params_app[:,:25]
intrinsics = c[:,16:]

                if use_neck:
                    extrinsics = c[0:,:16].reshape(4,4)
                    # extrinsics = torch.eye(4).to(device)
                    # extrinsics[1,3] = 0.01#0.01
                    # extrinsics[2,3] = 4.2 #4.2
                    c = torch.cat([extrinsics.reshape(1,-1), intrinsics.reshape(1,-1)], dim=-1)

                else:
                    pose_params_mot[:,:3] *= 0

The above codes using source_img's extrinsics, while the result still exists mismatch between result and source.

[lvZic]

Hi, this parameters are set empirically and can be adjusted.

In my opinion, the result generated are geometrically correspinding to tri-plane axis, in which the head are cented in 128x128( the preprocess crop and align strategy do the samething). So the generated result are must be centered in image without manually extrinsics.
The origin extrinsics cannot be used, as neither source nor driving extrinsic can proj the eg3d axis to 2d image.