Tools & Apps

I noticed the 2 Hessian Refinement of MACaQ can be applied to any GGUF without any patches... and tried it. 13% better quants

Supported Published November 13, 2026 Reviewed by AIOTruth November 14, 2026
7.6 AIOTruth Research Value Score

Summary

A hobbyist on r/LocalLLaMA published a refined GGUF build of a Qwen model on Hugging Face. They took two of the three techniques described in the MACaQ quantization paper and applied them without the third. The first measures how much damage compression does to each layer and picks quantization settings based on that. The second is a refinement step. The poster reports lower KL divergence than the matching Unsloth quant, the same speed, and a slightly better coding benchmark score.

The poster says the method works on existing GGUF files with no patches to inference software, so users only need to swap the file. They also say it applies to text, image, and speech models. The third technique, a random Hadamard transform, was left out because it adds an inference step that slows output unless the inference code is carefully reworked.

Why it matters

Better small quants change which models people run locally and which they run through hosted APIs. That matters for anyone trying to be cited or recommended by AI, because local assistants answer from what is stored in their weights, with no live retrieval. If a cheaper quant keeps more of the full model's behavior, the facts and associations a brand built up in training survive compression with less loss. A brand's representation in an open model then depends partly on quantization choices made by third parties like Unsloth or individual uploaders. Brands that track how open models describe them should test the quantized versions people actually download, since those may answer differently from the full model.

Source

What was checked

  • canonical URL taken from the page itself
  • read the item page (200)
  • https://github.com/ggml-org/llama.cpp: does not mention the item, tier 6 (Public source code or repository)
  • https://huggingface.co/docs: does not mention the item, tier 5 (Original dataset with disclosed methods)
  • https://huggingface.co/docs/hub/gguf: does not mention the item, tier 5 (Original dataset with disclosed methods)
  • https://huggingface.co/docs/inference-providers/index: supports the item, tier 5 (Original dataset with disclosed methods)
  • https://huggingface.co/docs/hub/model-cards: does not mention the item, tier 5 (Original dataset with disclosed methods)
  • https://huggingface.co/christophedevriese/Qwen3.8-27B-Hessian-Refined-GGUF: supports the item, tier 5 (Original dataset with disclosed methods)

Limitations

  • Everything checked traces back to a single domain, so this is one party's account.
  • Reddit engagement is a discovery signal here, not evidence that the claim is correct.

AIOTruth judgment

The item scores high on usefulness and originality: it takes a published method apart, finds a subset that runs on current tooling with no changes, and releases a working file anyone can download. The evidence status is Supported because the Hugging Face model page is a first-party artifact with the method disclosed. Relevance is low because this is model compression work, not a direct discovery or citation topic. The limits are real: the poster ran everything on one consumer GPU with a small test set, says the coding gain is a single data point, and all checked material traces back to the same author. Reddit engagement shows interest, not correctness.

How this score was calculated

DimensionWeightScoreWhat produced it
AIO relevance30%4geo; 2 scope question(s) matched
Usefulness30%102 artifact(s), 6 actionable marker(s), 2 measured figure(s)
Evidence25%8Original dataset with disclosed methods; 2 verified source(s) across 1 domain(s)
Originality15%9carries original testing or data; closest archive match 0

aioRelevance x 0.30 + usefulness x 0.30 + evidence x 0.25 + originality x 0.15. The rubric is published in full on the Editorial Method page. Scoring is deterministic: the same item scores the same on every run.

Treat the claim cautiously Open the source

Found something wrong here? AIOTruth corrects material errors openly. Have something we should review? Submit a find.

Run a check on AIOInsights