4 posts tagged with "Performance"

v2.7 is about knowing when your RAG pipeline gets worse before your users do. You can draft a test set from your own documents, score it from code or the command line, compare two configurations, and fail a pull request when answers get less grounded, with a new GitHub Action. It also adds retrievers for PostgreSQL with pgvector and for Qdrant, a generated API reference with corrected TypeScript types, and published, reproducible benchmarks.

v2.6 makes the library easier to adopt alongside what you already run and cheaper to run in production. checkAnswer grounds answers from LangChain, LlamaIndex, Mastra or the Vercel AI SDK. A new /testing entry runs pipelines offline in tests, and opt-in caching skips the model calls for repeated questions. The CLI's query command can now show how it got its answer, and both the package and the CLI load several times faster. The current version is 2.6.2: 2.6.1 published the updated README to npm, and 2.6.2 fixes npm install on machines without husky (see the 2.6.2 post).

How we transformed a slow, costly RAG system into a high-performance production service by optimizing every layer of the stack. This case study shares specific techniques, measurements, and code that reduced P95 latency from 2000ms to 800ms.

Optimizing RAG pipeline performance requires understanding bottlenecks across embedding generation, vector search, and LLM inference. This guide provides proven techniques for achieving production-grade performance at scale.