Models / Granite-4.0-Hgenerated from capability-matrix.json

Granite-4.0-H

IBM's Granite 4.0-H: Mamba-2 layers plus attention and experts.

chatexperts: sparse +sharedsafetensors, GGUF

Granite 4.0-H Tiny

Get it

fits on the CPU on MacBook Pro, M1 Pro, 16 GBfits on the CPU on RTX 2070 SUPER, 8 GBfits on Ryzen 7 3700X, CPU only
goinfer-chat pull granite-4.0-h-tiny
repo
ibm-granite/granite-4.0-h-tiny-GGUF
file
granite-4.0-h-tiny-Q8_0.gguf
quant
q8_0
size
7.4 GB
sha256
3528ba7c7ece5cb9…

Good for, and what it needs

A Mamba-2/attention hybrid MoE, if you want to exercise that path.

~8 GB resident at --quant int4; see docs/quantization.md on MoE and low-bit. Tokenizer: pre="dbrx", measured cl100k-shaped (tokenizer/gguf.go) — no PreTokenizerDecline

Tool calling

not yet measured

Decisions · /v1/systemone

Label scoring, unmeasured.

What hasn't been shown

  • Runs on the CPU only today. Not eligible to live on a GPU.
  • Tool calling hasn't been measured on Granite 4.0-H Tiny.
  • No speed measured on any machine.
  • No graded speed on the M1 Pro 16 GB.
  • No graded speed on the RTX 2070 SUPER.
  • No graded speed on the Ryzen 7 3700X.
  • Never measured on a machine like yours, so no speed is shown.

How sure we are

Against the released model
100.0% picks the same next token as HuggingFace
0.99566 closest the raw scores get at their worst position (cosine, 1.00000 is identical; bar starts at 0.95)

real-oracle 100.0%/0.99566 · what parity-gated means

Measured speed · decode, tokens per second

your pick · MacBook Pro, M1 Pro, 16 GBMetal
not measured
your pick · RTX 2070 SUPER, 8 GBCUDA
not measured
your pick · Ryzen 7 3700X, CPU onlyCPU
not measured

Ollama v0.32.5 at its defaults, same machine, same session. 128 tokens of context unless noted. Full method in benchmarks.

Architecture, for the curious
model_type
granitemoehybrid
design
state-space hybrid (Mamba-2)
experts
sparse +shared
attention window
none
QK-norm
no
RoPE
full
norm
RMSNorm, pre-norm
activation
SwiGLU
tied head
no
modality
text
GPU-resident
no
registry description: IBM Granite-4.0-H: Mamba-2 + attention hybrid + MoE-on-every-layer