Radar / AI infrastructure / ttok 1.0
ttok 1.0: Simon Willison's token-counting CLI
CLI tool that counts tokens in text using OpenAI's tiktoken library; the 1.0 release switches the default tokenizer from GPT-4 to GPT-5/GPT-6.
Why it matters
A tiny, practical utility every LLM developer uses, refreshed after two quiet years: new --list-models command, and the default tokenizer now matches GPT-5/GPT-6 (backed by a cross-model experiment showing all seven GPT-5.x/6.x models tokenize identically).
What you could build with it
A startup selling LLM cost analytics could bundle ttok-style exact token counts into its dashboards so finance teams see why their GPT-6 bills moved, instead of relying on model-reported estimates.
Does it hold up?
Too early to judge as a 1.0: it is a small utility from a trusted open-source author and the 1.0 change is a sane default-tokenizer update, but there is no usage evidence beyond the release notes yet.
Built with ttok 1.0
- HotON.ai: ttok 1.0 release notehoton.ai · Release recap dated Oct 9 linking the 1.0 tag and Simon Willison's blog post explaining the GPT-5/6 default-tokenizer switch.
Learn more
First spotted on github: source.
More AI infrastructure
DeepGEMM: DeepSeek's open GPU BLAS kernel libraryDeepSeek's DeepGEMM, a clean and efficient open-source BLAS kernel library for NVIDIA GPUs (up to 1550 TFLOPS on…infra · JEV 0.77Modular open-sources MAX inference server, Mojo stdlib and accelerator kernelsThe core of Modular's unified AI deployment stack is now open: the MAX inference server (OpenAI-compatible endpoints)…infra · JEV 0.76OmniRouteFree MIT AI gateway: one endpoint, 290+ providers and 500+ models with quota-aware auto-fallback.infra · JEV 0.75Context Mode: context-window optimization for AI coding agentsMCP server for AI coding agents that sandboxes verbose tool output and persists session memory, cutting context use by…infra · JEV 0.74StrataOpen-source inference engine that runs the 125B-parameter Qwen3.8-Flash-Next MoE on a single consumer GPU with 12GB+…infra · JEV 0.74Microsoft releases 301,000 Copilot coding-agent tracesMicrosoft open-sourced 301,026 GitHub Copilot coding-agent sessions (9.3M model calls, 8.7M tool calls) with timings…infra · JEV 0.71
Get the week's best AI launches, plus 3 ideas worth building
One email every Saturday. Ranked by traction, not hype. Free.