Chunking

What it means

Breaking large documents into smaller pieces of information.

Code example

import { RecursiveCharacterTextSplitter } from "langchain/text_splitter";
const splitter = new RecursiveCharacterTextSplitter({
  chunkSize: 500,
  chunkOverlap: 50,
});
const chunks = await splitter.splitText(longDocument);

Why it matters

Smaller chunks make retrieval more focused and provide relevant context without sending entire documents to the model.

When to use it

• Preparing documents for RAG
• Splitting long text for indexing