-
Notifications
You must be signed in to change notification settings - Fork 13.8k
ggml-hexagon: fix swiglu failure at test-backend-ops
#17344
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
…d_inf for improved overflow handling
| static const float kMaxExp = 88.02f; // log(INF) | ||
|
|
||
| const HVX_Vector max_exp = hvx_vec_splat_fp32(kMaxExp); | ||
| const HVX_Vector inf = hvx_vec_splat_fp32(kInf); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
thought we can move this init out of the for loop below.
| uint32_t w[VLEN_FP32]; | ||
| __fp16 fp16[VLEN_FP16]; | ||
| float fp32[VLEN_FP32]; | ||
| } __attribute__((aligned(VLEN), packed)) HVX_VectorAlias; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
its safe to use the gcc ext since in htp we're using clang.
| const float limit = ((const float *) (op_params))[3]; | ||
|
|
||
| const int nc = (src1_valid) ? ne0 : ne0 / 2; | ||
| const int nc = (src1_valid) ? ne00 : ne00 / 2; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
looks we should use src0->ne[0] here instead of dst->ne[0]
| // neg by setting the fp32 sign bit | ||
| HVX_Vector mask = Q6_V_vsplat_R(0x80000000); | ||
| return Q6_V_vor_VV(v, mask); | ||
| return Q6_V_vxor_VV(v, mask); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
using xor here to fix when x < 0
Fixes some failure on the Hexagon
swiglu/siluop and improves correctness of the GGML Hexagon backend.Summary of changes
hvx_vec_exp_fp32_guard,hvx_vec_inverse_fp32_guard) and use them in the Hexagon backend.NaN/Infhandling inexp,inverse, and related helper func.Before
After