openai/privacy-filter
← PII Detector Demo · OpenAIPrivacyFilterForTokenClassification · meta device, no weights downloaded
| architecture | OpenAIPrivacyFilterForTokenClassification |
|---|
| model_type | openai_privacy_filter |
|---|
| attention | GQA 7x · heads 14 · kv heads 2 · head_dim 64 |
|---|
| position | RoPE · θ=150000 · scaling yarn |
|---|
| norm | RMSNorm · eps 1e-05 |
|---|
| ffn | MoE 128 experts · top-4 · inter 640 · silu |
|---|
| embedding | vocab 200,064 · tie False |
|---|
| context | max_position_embeddings 131072 · sliding window 128 |
|---|
Dimensions
| hidden | 640 |
|---|
| layers | 8 |
|---|
| heads | 14 |
|---|
| kv_heads | 2 |
|---|
| head_dim | 64 |
|---|
| intermediate | 640 |
|---|
| vocab | 200,064 |
|---|
| ctx | 131,072 |
|---|
Parameters
≈1.40B
1,399,486,865 instantiated · Hub safetensors 1.40B (matches)
| model.layers | 1.27B | 90.8% |
|---|
| model.embed_tokens | 128.0M | 9.1% |
|---|
| score | 21.2K | 0.0% |
|---|
| model.norm | 640 | 0.0% |
|---|
| model.rotary_emb | 0 | 0.0% |
|---|
| dropout | 0 | 0.0% |
|---|
Memory (bf16 unless noted)
| params_total | 1,399,486,865 |
|---|
| weights_bf16_gib | 2.61 |
|---|
| weights_int8_gib | 1.30 |
|---|
| weights_int4_gib | 0.65 |
|---|
| kv_cache_bytes_per_token | 4,096 |
|---|
| kv_cache_gib_at_ctx | 0.12 |
|---|
| ctx_used | 32,768 |
|---|
KV cache = 2 · layers · kv_heads · head_dim · 2 bytes per token
One decoder layer · [0..7] ×8
Module tree (click to expand)
model OpenAIPrivacyFilterForTokenClassification 1.40B 100.0%
model OpenAIPrivacyFilterModel 1.40B 100.0%
- embed_tokens Embedding 128.0M 9.1% 200064×640
layers ModuleList 1.27B 90.8%
[0..7] ×8 OpenAIPrivacyFilterEncoderLayer 158.9M 11.4%
self_attn OpenAIPrivacyFilterAttention 1.3M 0.1% 14
- q_proj Linear 574.3K 0.0% 896×640, 896
- k_proj Linear 82.0K 0.0% 128×640, 128
- v_proj Linear 82.0K 0.0% 128×640, 128
- o_proj Linear 574.1K 0.0% 640×896, 640
mlp OpenAIPrivacyFilterMLP 157.6M 11.3%
- router OpenAIPrivacyFilterTopKRouter 82.0K 0.0% 128×640, 128
- experts OpenAIPrivacyFilterExperts 157.5M 11.3% 128×640×1280, 128×1280
- input_layernorm OpenAIPrivacyFilterRMSNorm 640 0.0% 640
- post_attention_layernorm OpenAIPrivacyFilterRMSNorm 640 0.0% 640
- norm OpenAIPrivacyFilterRMSNorm 640 0.0% 640
- rotary_emb OpenAIPrivacyFilterRotaryEmbedding 0 0.0%
- dropout Dropout 0 0.0%
- score Linear 21.2K 0.0% 33×640, 33
How this page was made: the model is instantiated from its Hugging Face config on torch's meta device (shapes only, no weights downloaded), parameters are counted per module, and the total is cross-checked against the safetensors metadata on the Hub. Memory figures are arithmetic, not measurements.