Tag

context-window

Covered in AI Engineering Models & Benchmarks, where the background and the sources for this subject live.

AI EngineeringSep 23, 2026

Why does vLLM 0.30.0 cut some models' max context length?

vLLM 0.30.0, released September 22, 2026, fixed a bug where vendor YaRN aliases like deepseek_yarn and telechat3-yarn double-applied the scaling factor, PR #56446. TeleChat3-36B-Thinking's derived max_model_len drops from 131072 to 32768, sarvam-105b's from 5242880 to 131072, both exactly 4x and 40x, and the old, larger figures risked indexing past the cache.

3 min read

Models & BenchmarksSep 2, 2026

What does a million-token context window actually buy you?

A large context window raises the ceiling on how much a model can see at once, but accuracy is not uniform across that window. Retrieval quality, latency and cost still dominate real workloads. Long context is best treated as a way to stop chunking small documents rather than as a replacement for retrieval.

5 min read