RAG Pipeline Utils v3.6: Start a Project in One Command, Pinecone, and Fewer False Alarms

v3.6.0 widens the way in. npm create rag-pipeline gives you a working app that answers from your documents and checks every answer, with Next.js, Mastra or Cloudflare Workers. Pinecone joins pgvector and Qdrant. The answer check raises fewer false alarms, measured against the Claude benchmarks, including where it got worse. And if you deploy to Cloudflare Workers, this release fixes the main entry there. Nothing breaks for existing code.

npm create rag-pipeline@latest my-app -- --template next

Start a project in one command

Each starter answers questions from a folder of documents, lists the passages it used, and checks every sentence of the answer against them. It runs before you add an API key, returning the matching passages, so you can try retrieval first.

Template	What you get
next	A Next.js app: a question page and POST /api/ask, with any AI SDK model wrapped in groundingMiddleware
mastra	A Mastra agent with a search_documents tool, a groundedness scorer (no model calls) and guardrails
workers	A Cloudflare Worker: GET /ask as JSON or a token stream, with citations and a groundedness score, on the /edge build

From the next starter, with a key:

{
  "answer": "Full-time employees receive twenty days of paid vacation per year. Unused days carry over until the end of March.",
  "sources": [{ "id": "vacation", "source": "vacation.md" }],
  "grounding": { "grounded": true, "groundedness": 1, "unsupported": [] }
}

Every change to the library builds and runs each starter in CI: next build, tsc and mastra build, wrangler deploy --dry-run, then a question with no key. Before this release they also ran against Claude, and each gave a grounded answer. Starter templates

Pinecone

PineconeRetriever talks to Pinecone's REST API with the built-in fetch, so there's nothing to install, and it runs on the edge build too.

const retriever = new PineconeRetriever({ index: "docs" }); // reads PINECONE_API_KEY
const pipeline = createRagPipeline({ embedder, retriever, llm });

Writing it turned up two things worth knowing about Pinecone today:

• It now has two kinds of index. Pinecone creates document indexes now, and those reject the vectors API most guides still show. The retriever reads the index description and uses the Documents API or the vectors API to match. An index that embeds text itself is refused with the reason, since it has no field for your vectors.
• Pinecone is eventually consistent. A write isn't visible to a query straight away. The retriever waits until its own writes are indexed, by comparing the log sequence number Pinecone returns for each write with the one on each query, so a query right after store() sees the new documents.

It passes the same retriever contract as pgvector and Qdrant: in CI against an in-memory Pinecone that is eventually consistent like the real one, and before this release against real indexes of both kinds. Pinecone

Fewer false alarms, measured

The Claude benchmarks in 3.5 found the lexical check flagging correct claims for reasons that had nothing to do with the facts. For example, "The protein binds to the receptor" failed against "The protein binds to the receptor but does not activate it", because of a negation in a clause the claim leaves out. 3.6 handles:

• a negation in a clause the claim leaves out;
• two negated clauses in a sentence;
• reference numbers ("Document 3");
• identifiers, compared whole;
• month and numeric dates;
• German, Spanish and French negation words.

The fixes were made against cases written for this project, committed before any change. The published sets were then run on the frozen code:

• SciFact citations: every claim the check flagged was reviewed by hand. False alarms fell from 78 to 43 across five Claude models, and the 3 real problems were still caught.
• RAGTruth: sentences wrongly flagged fell from 1,633 to 1,474, and with a Claude judge by about a quarter, at the cost of a few more misses.
• Answer-level scores barely move, because an answer stays flagged while any one of its sentences is.

That run also found a regression of our own: a claim naming "IL-3" passed against a source naming "IL-2". It was fixed, with test cases first, and the sets were run once more. The review also found two new false-alarm patterns from the clause rule. We left them as known limitations rather than tune the check on the published sets. All of it, with 3.5.0 and 3.6 side by side, is on the Benchmarks page.

Because verdicts can change, re-save the report you compare against with rag-pipeline eval --baseline after upgrading.

Fixed: the main entry in Cloudflare Workers

If you deploy to Workers, upgrade. Installed from npm, the main entry has crashed in a deployed Worker since 3.2.0, and a project without openai failed to bundle. CI didn't catch it because it tested a shortcut to the CommonJS build in a checkout that had openai installed. 3.6 points Wrangler at the CommonJS build and keeps optional packages from breaking a bundle. CI now installs the packed package into a project of its own and runs a Worker from it. On 3.2 to 3.5, import @devilsdev/rag-pipeline-utils/edge instead.

Also

• @devilsdev/rag-mcp-server is listed in the official MCP Registry as io.github.DevilsDev/rag-mcp-server, and each release now updates the listing.
• Before every release, eleven developer journeys run against the packed package and the live Claude API (npm run test:consumer). The Benchmarks page lists every Claude API call behind its numbers, with token counts and cost.

The scoped packages, @devilsdev/rag-mcp-server and the new create-rag-pipeline are released at 3.6.0 too. Full details: CHANGELOG.