Skip to content

fix: wrap dequantize_row_i2_s to_float callback to fix UB on ARM#469

Open
Mindev27 wants to merge 1 commit intomicrosoft:mainfrom
Mindev27:fix/i2s-to-float-wrapper
Open

fix: wrap dequantize_row_i2_s to_float callback to fix UB on ARM#469
Mindev27 wants to merge 1 commit intomicrosoft:mainfrom
Mindev27:fix/i2s-to-float-wrapper

Conversation

@Mindev27
Copy link

@Mindev27 Mindev27 commented Mar 15, 2026

The to_float callback for GGML_TYPE_I2_S casts dequantize_row_i2_s to ggml_to_float_t, but the function signatures do not match. As a result, i2_scale is never passed and ARM reads garbage from the register.

This PR adds a wrapper that extracts i2_scale from the tensor data and passes it correctly.

Based on analysis by @betovildoza in #468. Tested build on M4 Pro; no regression observed on the dotprod path

The change is inside the vendored llama.cpp submodule, so the diff won't show on GitHub. Checkout the branch to review, or see below:

// Added before type_traits table in ggml/src/ggml.c:
+static void dequantize_row_i2_s_wrapper(const void * vx, float * y, int64_t n) {
+    const uint8_t * x = (const uint8_t *) vx;
+    const float i2_scale = ((const float *)(x + (n / 4)))[0];
+    dequantize_row_i2_s(x, y, n, i2_scale);
+}

// In type table for GGML_TYPE_I2_S:
-.to_float = (ggml_to_float_t) dequantize_row_i2_s,
+.to_float = dequantize_row_i2_s_wrapper,

Partially addresses #468

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant