3 posts tagged with "Developer Tools"

v2.7 is about knowing when your RAG pipeline gets worse before your users do. You can draft a test set from your own documents, score it from code or the command line, compare two configurations, and fail a pull request when answers get less grounded, with a new GitHub Action. It also adds retrievers for PostgreSQL with pgvector and for Qdrant, a generated API reference with corrected TypeScript types, and published, reproducible benchmarks.

v2.6 makes the library easier to adopt alongside what you already run and cheaper to run in production. checkAnswer grounds answers from LangChain, LlamaIndex, Mastra or the Vercel AI SDK. A new /testing entry runs pipelines offline in tests, and opt-in caching skips the model calls for repeated questions. The CLI's query command can now show how it got its answer, and both the package and the CLI load several times faster. The current version is 2.6.2: 2.6.1 published the updated README to npm, and 2.6.2 fixes npm install on machines without husky (see the 2.6.2 post).

v2.5.0 is a consolidation release. The CLI now does real work end to end, every pipeline stage can be cancelled, grounding metrics measure what they claim to, and modules that were simulations are now clearly separated in an /experimental entry point. Several of these changes alter scores and defaults, so please read the upgrade notes before you update.