Tag
Covered in AI EngineeringModels & Benchmarks, where the background and the sources for this subject live.
TypeSafe publishes four use cases for Jev and two rate limits: 250,000 tokens per second and 1,200 requests a minute. Below 12,500 tokens per request the request limit binds first, so a small-state workload reaches a fraction of the token ceiling. Map-reducing a petabyte stays out of reach.
4 min read
openai-python v3.15.0 adds a prewarm field to the Responses API's prompt cache options. Setting it true forces generate to false, writing the cache with no output, at a TTL fixed to 30 minutes. Claude has offered the same effect since launch via a max_tokens=0 request, with a 5 minute default or a paid 1 hour TTL.
4 min read
TypeSafe AI launched Jev on 15 September 2026 as a System One model, priced at $0.042 per million input tokens with output free, and reports 70ms to 500ms against 3 to 329 seconds for frontier models. The economics are the strong part of the claim. The psychology metaphor carries less weight.
4 min read
Prompt caching stores a prefix of your prompt so later requests reuse it instead of reprocessing it. Reads are far cheaper than base input tokens, but writing the cache costs a premium and entries expire. It pays whenever a large stable prefix is reused several times inside the TTL, and loses on one-shot traffic.
6 min read