fix(qwen3): quantize Model before loading safetensors for 4-bit weights #2

Merged
peterswimm merged 1 commit from fix/qwen3-quantize-before-load into main 2026-07-16 20:01:28 +00:00
Owner

Cherry-picks an already-authored, already-verified fix (originally commit 3b607a72, 2026-05-14) that never made it to main — the branch it lived on diverged and was never merged.

Bug

Qwen3 checkpoints that declare quantization in config.json (the mlx-community/Qwen3-*-{4,8}bit family) build Model::new with every projection/embedding as MaybeQuantized::Original. load_qwen3_model then loads packed [..., dim/8] safetensors weights straight into those Original slots, which expect [..., dim]. The result is a tensor with the wrong last dimension, and the first RMSNorm in the forward pass crashes:

[rms_norm] (*weight) must have the same size as the last dimension of x but has 1024 elements

Fix

Before load_safetensors, if config.json declares quantization, call Model::try_into_quantized(group_size, bits) first — converting every MaybeQuantized field to its Quantized variant so the packed weights land in slots that actually invoke mlx_dequantize.

Verified

  • mlx-community/Qwen3-0.6B-4bit: previously crashed on the first generate step with the exact error above; now generates real tokens.
  • Re-verified today via the downstream ritual_engine integration tests (mlx_concat_smoke, mlx_smoke) against a real local model snapshot — both now pass end-to-end (tokenizer → generate loop → decode), where they previously failed identically with the same rms_norm crash whether using ConcatKeyValueCache or the custom RitualLayerCache, confirming the bug was in this model path, not in downstream KV-cache wrapping.

Rebased cleanly onto current main (cherry-pick, no conflicts). main already independently picked up the .inner-qualified key fallback fix from the same branch (via a different commit), so this PR only adds the Qwen3-specific piece that's still missing.

Cherry-picks an already-authored, already-verified fix (originally commit 3b607a72, 2026-05-14) that never made it to `main` — the branch it lived on diverged and was never merged. ## Bug Qwen3 checkpoints that declare `quantization` in `config.json` (the `mlx-community/Qwen3-*-{4,8}bit` family) build `Model::new` with every projection/embedding as `MaybeQuantized::Original`. `load_qwen3_model` then loads packed `[..., dim/8]` safetensors weights straight into those `Original` slots, which expect `[..., dim]`. The result is a tensor with the wrong last dimension, and the first `RMSNorm` in the forward pass crashes: ``` [rms_norm] (*weight) must have the same size as the last dimension of x but has 1024 elements ``` ## Fix Before `load_safetensors`, if `config.json` declares `quantization`, call `Model::try_into_quantized(group_size, bits)` first — converting every `MaybeQuantized` field to its `Quantized` variant so the packed weights land in slots that actually invoke `mlx_dequantize`. ## Verified - `mlx-community/Qwen3-0.6B-4bit`: previously crashed on the first generate step with the exact error above; now generates real tokens. - Re-verified today via the downstream `ritual_engine` integration tests (`mlx_concat_smoke`, `mlx_smoke`) against a real local model snapshot — both now pass end-to-end (tokenizer → generate loop → decode), where they previously failed identically with the same rms_norm crash whether using `ConcatKeyValueCache` or the custom `RitualLayerCache`, confirming the bug was in this model path, not in downstream KV-cache wrapping. Rebased cleanly onto current `main` (cherry-pick, no conflicts). `main` already independently picked up the `.inner`-qualified key fallback fix from the same branch (via a different commit), so this PR only adds the Qwen3-specific piece that's still missing.
fix(qwen3): quantize Model before loading safetensors for 4-bit weights
Some checks failed
validate / checks (pull_request) Failing after 3s
validate / tests (macos-15, 1.85.0) (pull_request) Failing after 4s
validate / tests (macos-15, stable) (pull_request) Failing after 2s
5e78a4a157
When a Qwen3 checkpoint declares quantization in config.json (the
mlx-community/Qwen3-*-{4,8}bit family), Model::new builds the model
with every projection / embedding wrapped in MaybeQuantized::Original.
load_qwen3_model then calls load_safetensors against those Original
slots — silently loading packed [vocab_size, hidden_size/8] tensors
into Embedding slots whose Module::forward returns a tensor with the
wrong last dimension. The first RMSNorm in the forward path then
errors with:

  [rms_norm] (*weight) must have the same size as the last dimension
  of x but has 2048 elements

(or 1024 for Qwen3-0.6B) because the actual x carries the un-unpacked
packed dimension rather than hidden_size.

Fix:
- Add `quantization: Option<QuantizationConfig>` to ModelArgs, populated
  via serde from config.json's `quantization` block (Optional so bf16
  checkpoints continue to deserialize unchanged).
- Before load_safetensors, if `quantization` is present, call
  `Model::try_into_quantized(group_size, bits)` on the freshly-built
  model. That converts every MaybeQuantized<nn::Linear / nn::Embedding>
  field to MaybeQuantized::Quantized so the packed weights now land in
  slots whose forward path actually invokes mlx_dequantize.

Verified end-to-end:
- mlx-community/Qwen3-0.6B-bf16: still loads + generates (no regression
  on the unquantized path).
- mlx-community/Qwen3-1.7B-4bit: previously crashed on layer 0
  input_layernorm; now reaches the generate loop and produces tokens.

Test reproducer (downstream): see ritual_engine `mlx_concat_smoke`
integration test pinned at this rev — was 1 failed, now 1 passed.
peterswimm deleted branch fix/qwen3-quantize-before-load 2026-07-16 20:01:28 +00:00
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
Toilville/mlx-rs-mirror!2
No description provided.