Brief
llama-index 0.14.16 adds rate limiters and non-blocking retries around LLM and embedding calls
Release notes dated 2026-03-10 for llama-index-core 0.14.16 add a token-bucket rate limiter for LLM and embedding API calls, plus a sliding-window limiter for strict per-minute caps. The same set carries async and retry fixes; the notes give no defaults, benchmarks or migration guidance.
The headline change is throttling at the call site. llama-index-core 0.14.16 adds a token-bucket rate limiter for LLM and embedding API calls, and a separate SlidingWindowRateLimiter aimed at strict per-minute caps. Both sit in the workflow step where a pipeline fans out requests to a model provider.
Reliability work accompanies it. The release replaces a blocking run_async_tasks call with asyncio.gather, changes async retry backoff so it no longer blocks the event loop, and makes the OpenAI embeddings retry decorator respect the Retry-After header. Partial-failure handling in SubQuestionQueryEngine is also fixed, and SimpleObjectNodeMapping gains a RestrictedUnpickler referencing CWE-502.
For operators, the practical question is whether in-process limiting now substitutes for the concurrency caps and per-key budgets many teams maintain outside the framework. That depends on bucket size, refill behaviour and whether the limiter is shared across processes — none of which these notes state.
Our reading
This desk cares because rate limiting and retry semantics are the two places where an agent framework stops being a library and starts shaping production traffic. Teams running multi-step pipelines against metered providers should treat the limiter as a candidate replacement for homemade throttles, but only after checking how it behaves under concurrent workers. Anyone relying on partial results…
What to do or watch
Pin the version, then run a load test that (a) confirms the limiter holds your provider's per-minute cap with your real concurrency, and (b) checks that a retry inside an async workflow no longer stalls the event loop. The unresolved question is whether the limiter is per-process or shared, and what default bucket it uses — the notes do not say.
Source details and supporting facts
Each line is stated by the page named above it.
Stated by github.com
- llama-index-core 0.14.16 adds a token-bucket rate limiter for LLM and embedding API calls (#20712).
- The release adds a SlidingWindowRateLimiter for strict per-minute caps (#20799).
- llama-index-embeddings-openai 0.5.2 respects the Retry-After header in the OpenAI retry decorator (#20813).
- The release changes async retry backoff to avoid blocking the event loop (#20764).
- The release replaces blocking run_async_tasks with asyncio.gather (#20795).
- The release fixes partial-failure handling in SubQuestionQueryEngine (#20905).
- The release adds a RestrictedUnpickler to SimpleObjectNodeMapping (CWE-502) (#20857).
- The release notes are dated 2026-03-10.
Sources
- LlamaIndex releasesText stored 17 September 2026
How this story was checked. Written from the 1 page listed above, stored 17 September 2026; claims checked against that stored text on 17 September 2026.
What that means
- 8 of 8 reported statements were confirmed against the page that carries them; the rest were removed rather than published.
- Figures in the text were required to appear in the stored source text: yes. Identifiers: yes.
- The check reads stored text only: no claim rests on a fresh look that did not happen.
- Where the reporting was silent, the text says so instead of filling the gap.