RAG Pipeline Utils v3.2: Install Only What You Use, Cheaper Models per Query, OpenTelemetry and the Edge

v3.2.0 splits the library into four packages you can install on their own, routes each query to the cheapest model that can answer it, records runs as OpenTelemetry spans, reports the time to first token, and runs on Cloudflare Workers and other edge runtimes. Installs are 15 MB smaller. Nothing changes for existing code.

npm install @devilsdev/rag-pipeline-utils@^3.2.0

Install only what you use

The library is now four packages, and @devilsdev/rag-pipeline-utils re-exports all of them:

Package	What it has
@devilsdev/rag-core	createRagPipeline, MemoryRetriever, hybrid search, rerankers, caches, costs and routing
@devilsdev/rag-connectors	OpenAI, Anthropic, Cohere and Ollama connectors, LocalEmbedder, Qdrant, pgvector, the AI SDK
@devilsdev/rag-eval	The evaluation metrics, datasets and reports, checkAnswer, citations
@devilsdev/rag-guardrails	detectInjection and the guards

If you only score answers from another framework, npm install @devilsdev/rag-eval installs about 6 MB. The full package is 33 MB.

const { computeFaithfulness, checkAnswer } = require("@devilsdev/rag-eval");

These are the same objects as the full package's, not copies: require("@devilsdev/rag-pipeline-utils").CitationTracker === require("@devilsdev/rag-eval").CitationTracker, so you can mix imports, and instanceof works across them. All five packages release together at the same version. See Packages.

The full package also got smaller. The CLI's prompts, spinner, colours and tables are now bundled into the CLI instead of installed as dependencies, which takes a clean install from 48 MB to 33 MB, with 51 fewer packages.

Send each query to the cheapest model that can answer it

Most questions a RAG service gets are lookups that a small model answers as well as a large one. createModelRouter is an llm that classifies each query and picks the cheapest model suited to its class:

const {
  AnthropicConnector,
  createModelRouter,
} = require("@devilsdev/rag-pipeline-utils");

const llm = createModelRouter({
  routes: [
    {
      model: "claude-haiku-4-5",
      llm: new AnthropicConnector({ model: "claude-haiku-4-5" }),
      handles: ["simple"],
    },
    { model: "claude-sonnet-5-5", llm: new AnthropicConnector() },
  ],
});

await llm.route("How many vacation days do I get?");
// { class: "simple", model: "claude-haiku-4-5", estimatedCost: 0.0016 }
await llm.route("Why do contractors get fewer days than employees?");
// { class: "complex", model: "claude-sonnet-5-5", ... }

The estimate prices this query's prompt and expected answer at each model's price, from the built-in table (now with Claude Haiku 4.5 and Opus 5.5) or your own pricing. A model with no price is an error when the router is created, so it can't win by looking free. Pass the router as llm, and each result reports the decision as result.route. fallback: true hands a failed query to the next cheapest model. The CLI has the same router as the router llm in .ragrc.json, and query --trace prints which model answered. See Routing queries to cheaper models.

OpenTelemetry spans

createRagPipeline({ telemetry: true }) records each run as a span, with a child span for each stage: embeddings, retrieval, reranking and the model call. The stage spans follow the OpenTelemetry GenAI conventions, so the tracing tools that understand those show the model, token counts and timings. Install @opentelemetry/api, an optional peer dependency, next to the SDK you already use. Spans never record what users typed unless you set recordContent: true. See OpenTelemetry.

Time to first token

For a streamed answer, the wait users notice is how long the first word takes. The stream's last chunk now carries it:

for await (const chunk of await pipeline.run({
  query: "How many vacation days do I get?",
  options: { stream: true },
})) {
  if (chunk.done) console.log(chunk.timing); // { firstTokenMs: 412.7, totalMs: 1630.2 }
}

firstTokenMs counts from the moment run() was called, so retrieval and reranking are included. The OpenTelemetry spans carry it too, rag-pipeline query --streaming --trace prints it, and rag-pipeline ai benchmark --streaming reports its percentiles.

Cloudflare Workers and edge runtimes

The pipeline runs on Cloudflare Workers with the nodejs_compat flag. CI runs it in workerd, the Workers runtime, on every change, along with the new Cloudflare Worker example. Getting there fixed Claude and Ollama streaming, which could garble a character split between two network chunks, and QdrantRetriever, which Workers rejected.

Where Node.js APIs don't exist at all, import @devilsdev/rag-pipeline-utils/edge. It has the pipeline, the Anthropic, Cohere and Ollama connectors, the AI SDK adapter, MemoryRetriever and QdrantRetriever, routing, grounding, guardrails and evaluation, in one module of about 140 KB compressed. It suits Workers without the flag, Deno, Netlify Edge Functions and Vercel Edge Functions. Note that Vercel now recommends its Node.js runtime over Edge, and Next.js 16.3 drops runtime = 'edge', so on Next.js use the main entry. See Edge runtimes.

Also in 3.2.0

• An MCP server example. examples/mcp-server lets Claude Desktop, Claude Code and other MCP clients search your indexed documents.
• TypeScript. With strict on, the built-in retrievers and rerankers are now accepted as createRagPipeline components; before, they failed to compile.
• A smaller package. The CLI runs from a bundle, so the package no longer ships src/: 165 files instead of 373.

Upgrading

There are no breaking changes. The full list is in the CHANGELOG.