Skip to content

Wrong output when decoding big values #2

@Shadlock0133

Description

@Shadlock0133

When testing encoding then decoding values with Rice coding, I got failures on big positive values. Here is how I fixed it, tests pass, but I'm not sure if it's correct.

In encode_value, x >> 30 should be x >> 31 to preserve sign in lowest bit.

In decode_value, last line is Ok((x as i32 >> 1) ^ ((x << 31) as i32 >> 31)). When shifting values right, high bit get sign-extended, inverting sign from original value. To fix that, cast need to be moved after shift (x >> 1) as i32.

Lastly, in encode_value, (2 * x) can be replaced with (x << 1), which works the same, and doesn't panic in debug on big values.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions