LLM Architecture Cards

What is inside an open-weight model, in one page, without downloading its weights. Each card is built from the model\x27s Hugging Face config: the model is instantiated on torch\x27s meta device (shapes only), parameters are counted per module, one decoder layer is drawn as a block diagram with its weight shapes, and the total is cross-checked against the safetensors metadata on the Hub. Memory numbers are arithmetic, not measurements.

Cards

Model	Attention	FFN	Params · bf16	KV / token	Hub check
Qwen/Qwen2.5-7B-Instruct the local model in the Booking Agent Demo	GQA 7× · 28 / 4 · head 128	dense · 18944 · SiLU	7.62B · 14.2 GiB	56 KiB	matches
openai/privacy-filter the browser model in the PII Detector Demo	GQA 7× · 14 / 2 · head 64 · sliding window 128	MoE 128 experts · top-4	1.40B · 2.6 GiB	4 KiB	matches
Qwen/Qwen3.5-9B	GQA 4× · 16 / 4 · head 256 · 24 linear + 8 full	dense · 12288 · SiLU	8.95B · 16.7 GiB	128 KiB	Hub 9.65B, +0.7B not instantiated (MTP head)
Qwen/Qwen3.8-27B	GQA 6× · 24 / 4 · head 256 · 48 linear + 16 full	dense · 17408 · SiLU	26.9B · 50.1 GiB	256 KiB	Hub 27.78B, +0.9B not instantiated
ibm-granite/granite-embedding-278m-multilingual	encoder, MHA	dense	278M · 0.5 GiB	—	matches

Make your own

git clone https://github.com/kaihuan-huang/llm-arch-viz && cd llm-arch-viz
uv venv .venv --python 3.12 && uv pip install --python .venv/bin/python torch transformers huggingface_hub
.venv/bin/python llm_arch_viz.py Qwen/Qwen2.5-7B-Instruct            # -> out/Qwen__Qwen2.5-7B-Instruct.html
.venv/bin/python llm_arch_viz.py Qwen/Qwen2.5-7B-Instruct --json     # facts + memory as JSON, for checking a hypothesis

Why meta device: from_config under torch.device("meta") allocates no storage, so a 27B model takes seconds and zero bytes; only config.json is fetched. The layer diagram orders modules by name (input norm → attention → post-attention norm → MLP), so a model with unusual names may draw out of order; the module tree is exact.