Moss: Sub-10ms Search SDK for AI Agents (Free, 2026)

Moss is a sub-10ms semantic search SDK (P50 3.1ms) for AI agents and voice apps. No vector DB needed. Runs in-browser, edge, or cloud. Free tier at moss.dev.

Moss is a retrieval SDK for AI agents (YC F25, 2024) that runs semantic search in-process at sub-10ms latency (P50 3.1ms on 100K documents), cutting token usage by 70-90% versus network-based RAG. Works in browser, edge, or cloud via JavaScript, Python, and TypeScript SDKs. Free tier available; best for voice agents and real-time copilots where retrieval speed matters.

Moss is a semantic search runtime (YC F25, founded 2024, San Francisco) delivering sub-10ms vector retrieval for AI agents and voice copilots without a separate vector database. It bundles search and embedding into a single SDK running in-browser, on-device, edge, or cloud. Developers integrate it via JavaScript, Python, or TypeScript SDKs. A free tier is available at moss.dev; 130+ projects are live as of 2026.

Maker: Moss (YC F25) · Protocol: REST · Auth: api key

Compatible agents: LangChain, LlamaIndex, DSPy, Pipecat, LiveKit, Vapi, ElevenLabs, Strands Agents, Claude Desktop (via unofficial MCP server), Any LLM with function-calling support

Required runtime: Node.js >= 16 (for JavaScript/TypeScript SDK), Python 3.9+ (for Python SDK), Moss Cloud project_id and project_key (free at moss.dev)

About Moss

Moss is a sub-10ms semantic search runtime built by Sri Raghu Malireddi and Harsha Nalluru (YC F25, San Francisco, founded 2024). Unlike a standalone vector database, Moss is a retrieval library you embed inside your own application: it runs queries entirely in-process with no network round trip to an external service. As of June 2026, 130+ projects are live, 300+ indexes have been created, and 50+ companies are actively testing the product, with adoption growing 200% month-over-month. The architecture works in two layers. Moss Cloud handles document ingestion, embedding, storage, and distribution. You register at moss.dev to receive a project_id and project_key, then pass them to the SDK. At startup the runtime downloads your vector index over HTTPS, holds it in memory, and serves every query locally. Once an index is loaded, no query leaves your process, which is the source of the sub-10ms latency. Benchmarks on 100,000 documents (Apple M4 Pro, 750 measured queries, top_k=5) show a median (P50) latency of 3.1ms and P99 of 5.4ms, including both embedding inference and end-to-end retrieval. Moss ships SDKs for JavaScript and TypeScript (runs in browser via WebAssembly and in Node.js), Python, Elixir, and C. Framework integrations include LangChain, DSPy, LlamaIndex, Pipecat, LiveKit, Vapi, ElevenLabs, and Strands Agents. Any agent that can call a library function can use Moss for retrieval. The WebAssembly build allows fully offline AI interfaces without a backend server. An unofficial MCP server (github.com/CoderOMaster/moss-mcp) lets Claude Desktop and other MCP clients manage indexes and query Moss through the Model Context Protocol. Moss offers a free tier: sign up at moss.dev and receive a project_id and project_key at no cost. Paid tiers exist for larger workloads, but exact pricing is not published publicly; contact Moss directly for scale pricing. The core runtime is open source and hosted at github.com/usemoss/moss. The team has shipped SDKs across four languages and integrations with eight agent and voice frameworks within the first year. The GitHub organization at github.com/usemoss lists 15 repositories. The product is expanding from voice AI and developer tool pilots into broader conversational AI and real-time copilot applications where retrieval latency is a critical constraint.

Key Features

• Sub-10ms Latency: Benchmarks on 100,000 documents show a P50 latency of 3.1ms and P99 of 5.4ms, including both embedding inference and end-to-end retrieval on an Apple M4 Pro.
• No Vector Database Required: Moss packages the entire index as a single artifact that lives in Moss Cloud and loads into the SDK runtime in memory, removing the need to manage a separate Pinecone, Qdrant, or Weaviate cluster.
• Runs Everywhere: A single SDK targets browsers (via WebAssembly), Node.js, edge runtimes, mobile devices, and cloud servers without code changes, letting the same retrieval logic run client-side or server-side.
• Hybrid Retrieval: Combines dense vector (semantic) search with keyword scoring in a single call, improving recall for queries that mix natural-language intent with specific product names or identifiers.
• Built-in Embeddings: Moss includes embedding models such as moss-minilm bundled with the SDK, so developers do not need to wire up a separate embedding API call before indexing or querying documents.
• 70-90% Token Savings: By retrieving the top-k relevant passages instead of sending full documents to the LLM context window, production pilots report cutting token usage by 70-90% compared to standard RAG pipelines.

Use Cases

• Voice Agent FAQ Retrieval: A voice agent calls client.query() to retrieve the top 3 matching FAQ answers in under 5ms, keeping the conversation flowing without a perceptible pause between the user question and the spoken response.
• Offline Browser Copilot: A JavaScript copilot loads the Moss WebAssembly runtime and a pre-built index, then serves semantic search entirely in the browser with no backend server, enabling privacy-first or air-gapped AI assistants.
• Real-Time Customer Support Agent: A support copilot indexes product documentation and queries Moss for relevant passages with each user message, cutting token usage by 70-90% compared to sending full documents to the LLM context window.
• On-Device Mobile AI Search: A mobile app bundles the Moss index and runtime on the device, giving users instant semantic search over personal notes or knowledge bases without a network connection.
• RAG Pipeline Latency Reduction: Teams migrating from Pinecone or Qdrant replace the external vector DB call with a local Moss runtime call, reducing retrieval latency from 80-300ms round trips down to 3-5ms while also cutting API costs.

Install

npm install @moss/sdk

Requirements

• Node.js >= 16 or Python 3.9+ (depending on SDK language)
• Sign up at moss.dev to obtain a free project_id and project_key
• Index your documents using client.createIndex() before querying

Actions

Create Index

Creates a new semantic search index from an array of documents and optionally specifies an embedding model.

import { MossClient } from '@moss/sdk';

const client = new MossClient({ projectId: 'proj_abc123', projectKey: 'key_xyz789' });

await client.createIndex('product-faq', [
  { id: 'doc1', text: 'Returns are accepted within 30 days of purchase.' },
  { id: 'doc2', text: 'Shipping takes 3 to 5 business days for standard delivery.' },
  { id: 'doc3', text: 'Contact support at help@example.com for order issues.' }
], { modelId: 'moss-minilm' });

• indexName (string) — required: Unique name for the index within the project.
• documents (array) — required: Array of document objects, each with an id (string) and text (string) field. Optionally include an embedding array for pre-computed vectors.
• options (object): Configuration object. Accepts modelId (string) to specify the embedding model, e.g. 'moss-minilm'.

Load Index

Downloads a pre-built index from Moss Cloud into the local runtime memory, enabling sub-10ms queries without further network calls.

await client.loadIndex('product-faq');

• indexName (string) — required: Name of the index to load from Moss Cloud into local memory.

Query

Performs a semantic search on a loaded index and returns the top-k most relevant documents, with optional hybrid keyword scoring.

const results = await client.query('product-faq', 'how long does shipping take', { topK: 3 });

for (const result of results) {
  console.log(result.id, result.score, result.text);
}

• indexName (string) — required: Name of the index to search against. Must be loaded first with loadIndex().
• queryText (string) — required: The natural-language query string to embed and search.
• options (object): Options object. Accepts topK (number, default 5) for the number of results to return, and embedding (array) to supply a pre-computed query vector.

Add Documents

Appends new documents to an existing index and updates the index on Moss Cloud without rebuilding from scratch.

await client.addDocuments('product-faq', [
  { id: 'doc4', text: 'We offer free shipping on orders over $50.' },
  { id: 'doc5', text: 'Gift wrapping is available at checkout for $5.' }
]);

• indexName (string) — required: Name of the existing index to append documents to.
• documents (array) — required: Array of document objects, each with an id (string) and text (string). Ids must be unique within the index.

List Indexes

Returns the names and metadata of all indexes in the current Moss Cloud project.

const indexes = await client.listIndexes();

for (const idx of indexes) {
  console.log(idx.name, idx.documentCount, idx.createdAt);
}

How to Invoke

Developers call SDK methods directly in application code: client.createIndex(), client.loadIndex(), client.query(), and client.addDocuments(). Queries run locally in-process after the index is loaded from Moss Cloud. An unofficial MCP server also exposes Moss index management via MCP tools for Claude Desktop and other MCP clients.

Pricing

Free tier available: sign up at moss.dev for a project_id and project_key at no cost. Paid tiers exist for larger indexes and higher query volumes, but exact pricing is not published. Contact Moss directly for scale pricing.

Strengths

• P50 latency of 3.1ms makes real-time voice agent retrieval practical without perceptible delay.
• WebAssembly build runs in the browser with no backend server, enabling fully offline or privacy-first AI applications.
• SDKs in four languages (JS/TS, Python, Elixir, C) plus eight framework integrations cover most production AI stacks.
• No vector database infrastructure to provision, scale, or tune: one free-tier signup gives a working retrieval layer in minutes.

Weaknesses

• Paid tier pricing is not public, making cost estimation for large workloads impossible without a direct sales conversation.
• The product is early-stage (YC F25, 2024), so long-term reliability guarantees, SLAs, and enterprise support terms are not yet documented.
• Holding indexes in-process increases application memory footprint, which can be a constraint on low-RAM devices or edge runtimes.

View the official Moss skill page