[Megathread] GLM-5.3-Flash - former ox-alpha

Megathread for discussing the release of GLM-5.3-Flash. Quants Fine-Tunes & Abliterations Chat Templates Inference Server Support & Configuration Experiences, Benchmarks & Model Comparisons We'll try to clean up future duplicates around the release and point them here. Highlights GLM-5.3-Flash is the first natively multimodal model in the GLM-5 series, and the first open-weight release of the glm5_next architecture. Z.ai's pitch: outperforms GLM-5.2 at one-tenth the price while approaching Claude Opus 4.8 on coding and agentic benchmarks. It introduces: Hybrid Sparse + Linear Attention: 45 layers laid out as repeating blocks of 3x KDA linear attention followed by 1x DeepSeek-style sparse attention (34 linear / 11 sparse layers). The sparse layers use a lightning indexer (32 heads, dim 128) with a top-k budget of 2048 tokens, sharply reducing long-context serving cost. Manifold-Constrained Hyper-Connections (mHC): widened residual streams with manifold-constrained mixing between layers, adopted to further improve scaling efficiency. Natively Multimodal: a 24-layer ViT (448px, patch 14, 2x2 spatial merge) with temporal patching, so image and video tokens are in the vocabulary. Trained on a 30T-token multimodal corpus. MTP head shipped in the weights: 1 next-N prediction layer; the official vLLM recipe uses it with 5 speculative tokens. FP8 first: the main repo is FP8 (e4m3, dynamic activation scaling). A separate official BF16 repo exists. Model Overview Type: Causal Language Model with Vision Encoder ( Glm5NextForConditionalGeneration ) Training Stage: Pre-training (30T multimodal tokens) & Post-training License: MIT Language Model Number of Parameters: 320B with 18B activated Hidden Dimension: 4096 Vocabulary: 154,880 Number of Layers: 45 (first 3 dense MLP, remaining 42 MoE) Hidden Layout: 11 x (3 x (KDA Linear Attention -> MoE) -> 1 x (Sparse Attention -> MoE)), plus 1 trailing linear layer KDA Linear Attention (34 layers): Number of Heads: 64 Head Dimension: 128 Sparse Attention (11 layers, DeepSeek-style): Number of Attention Heads: 64 QK / V Head Dimension: 256 / 256 Indexer: 32 heads, head dim 128 Budget: top-2048 tokens Mixture of Experts: Number of Experts: 288 routed + 1 shared Number of Activated Experts: 8 routed + 1 shared Expert Intermediate Dimension: 2048 Dense Intermediate Dimension (layers 0-2): 12288 mHC: enabled MTP: 1 layer Context Length: 1,048,576 tokens ( max_position_embeddings ); evaluated at 300K text / 164K vision Vision Encoder Depth: 24 layers, hidden 1024, 16 heads Image Size: 448 x 448, Patch Size: 14 Spatial Merge: 2 x 2, Temporal Patch: 2 (video) Output Projection: 4096 (LM hidden) Weights zai-org/GLM-5.3-Flash - FP8 (e4m3, dynamic), 62 shards, ~331 GB zai-org/GLM-5.3-Flash-BF16 - BF16, 120 shards, ~640 GB [ARCHITECTURE IMAGE - upload via the editor here, or use the link below] Architecture diagram Benchmarks [BENCHMARKS IMAGE - upload via the editor here, or use the link below] Benchmark chart Recommended sampling parameters From generation_config.json and the eval setup on the model card: Default: temperature=1.0, top_p=0.95 (max generation length 163,840 for evals) Agentic coding (NL2Repo): temperature=1.0, top_p=1.0 DeepSWE: temperature=0.95, top_p=1.0 Vision (BabyVision): temperature=1.0, top_p=0.95 Inference vLLM (official recipe, vLLM 0.27.0+, FlashInfer 0.6.17+ for NoPE sparse MLA, Hopper and newer): vllm serve zai-org/GLM-5.3-Flash \ --tensor-parallel-size 4 \ --kv-cache-dtype fp8 \ --speculative-config '{"method":"mtp","num_speculative_tokens":5}' \ --tool-call-parser glm47 \ --reasoning-parser glm45 \ --enable-auto-tool-choice \ --served-model-name zai-org/GLM-5.3-Flash SGLang: official cookbook has verified configs for H100/H200/B200/B300/GB200/GB300 (TP4/EP4), with adaptive MTP for low-latency and --mm-feature-transport cpu to offload vision features. Also supported at launch: TokenSpeed, KTransformers (CPU/GPU hybrid tutorial linked below). Official Links HF: huggingface.co/zai-org/GLM-5.3-Flash HF (BF16): huggingface.co/zai-org/GLM-5.3-Flash-BF16 MS: modelscope.cn/models/ZhipuAI/GLM-5.3-Flash Repo: github.com/zai-org/GLM-5 Blog: z.ai/blog/glm-5.3-flash Docs: docs.z.ai/guides/llm/glm-5.3-flash Paper: arxiv.org/abs/2602.15763 vLLM: recipes.vllm.ai/zai-org/GLM-5.3-Flash SGLang: docs.sglang.io/cookbook/autoregressive/GLM/GLM-5.3-Flash TokenSpeed: lightseek.org/tokenspeed/recipes/models KTransformers: github.com/kvcache-ai/ktransformers/blo...en/kt-kernel/GLM-5.3-Flash-Tutorial.md Discord: discord.gg/QR7SARHRxK Popular Unsloth GGUF: huggingface.co/unsloth/GLM-5.3-Flash-GGUF Unsloth FP8: huggingface.co/unsloth/GLM-5.3-Flash-FP8 AtomicChat GGUF: huggingface.co/AtomicChat/GLM-5.3-Flash-GGUF