RAG Text Chunker

Split Markdown or text into retrieval-ready chunks by headings, words or characters — with overlap and metadata, in your browser

    Input

    Try a sample:

    Sections split at heading boundaries; any section larger than the max is further split by characters.

    Have HTML? Convert to Markdown first

    Chunks (4)

    4 chunks · ≈86 tokens total

    #0DevTools API Reference
    65 chars · ≈17 tok
    # DevTools API Reference

Base URL: https://api.devtools.tools/v1
    #1DevTools API Reference > Authentication
    88 chars · ≈22 tok
    ## Authentication

All requests require a bearer token sent in the Authorization header.
    #2DevTools API Reference > Errors
    86 chars · ≈22 tok
    ## Errors

Errors use standard HTTP status codes and a JSON body with a message field.
    #3DevTools API Reference > Errors > Rate limiting
    99 chars · ≈25 tok
    ### Rate limiting

Clients are limited to 100 requests per minute. Exceeding the limit returns 429.
    Retrieval-Augmented Generation (RAG) and long-context LLM workflows both depend on splitting documents into chunks that are small enough to embed or rank, yet big enough to keep their meaning. Naive word-count splitters cut a sentence or a code block in half and destroy context. The RAG Text Chunker splits intelligently: a Markdown-header strategy breaks the document at heading boundaries (preserving the heading hierarchy as each chunk's source path), while fixed-word and fixed-character strategies give you deterministic sizes with configurable overlap so adjacent chunks share context at the edges. Every chunk carries metadata — its index, the source heading it came from, and an estimated token count — and you can copy a single chunk, copy them all, or export the full set as JSON ready for an embeddings pipeline. Chunking runs entirely locally in your browser. Have raw HTML? Convert it to Markdown first with the HTML to Markdown tool, then chunk it here.

    Examples

    Split by Markdown headings

    # API Reference
Base URL: …
## Authentication
Use a bearer token.
## Errors
Returns JSON.

→ 3 chunks:
  [0] API Reference
  [1] API Reference > Authentication
  [2] API Reference > Errors

    Each heading starts a new chunk; the source path keeps the hierarchy so retrieval knows where a chunk lived.

    Fixed size with overlap

    Strategy: fixed-words · max 200 · overlap 30

→ adjacent chunks share their last/first 30 words, smoothing context transitions.

    Overlap prevents a concept that straddles a boundary from being split across two unrelated chunks.

    Export for embeddings

    [
  { "index": 0, "heading": "Intro", "content": "…", "tokens": 128 },
  { "index": 1, "heading": "Setup", "content": "…", "tokens": 95 }
]

    Download the chunks as JSON — index, heading, content and token estimate — ready for a vector database.

    Examples

    Recursive chunking with overlap

    Input
    Strategy: recursive
Target size: 500 chars
Overlap: 50 chars
    Output
    [ { "chunk_id": "chk_001", "index": 0, "content": "...",
    "char_count": 498, "estimated_token_count": 124,
    "source_header": "# Installation" }, ... ]

    Recursive splitting keeps paragraph and sentence boundaries intact, with 50 chars of overlap carried into the next chunk.

    About this tool

    Text Chunker splits Markdown or plain text into LLM- and retrieval-friendly pieces. Choose a strategy — by headings, by word count, by character count, or recursive (paragraph → sentence → character) — set an overlap so context bleeds between chunks, and export the result as structured JSON ready for a vector store or RAG pipeline.

    Each chunk carries metadata: an id, its index, the character count, an estimated token count, and the source heading it came from. The recursive strategy respects natural boundaries so you don't slice a sentence in half, and a hard cap keeps very large inputs manageable. Everything runs locally; nothing is uploaded.

    How to use

    1. Paste your text

      Drop in Markdown or plain text — documentation, a transcript, an article, or any long passage you need to split.

    2. Pick a strategy and size

      Choose headings, words, characters, or recursive, then set the target size and an optional overlap between chunks.

    3. Review the chunks

      Chunks render with index, char/token counts and source headers, and overlap is highlighted so you can see the seam.

    4. Export as JSON

      Download a versioned JSON array of chunks (id, index, content, counts, header) for your indexing pipeline.

    Use cases

    Build a RAG index

    Split documentation into overlapping chunks and export the JSON for embedding and retrieval.

    Fit long context into a model

    Break a document that exceeds a context window into model-sized pieces you can feed sequentially.

    Prepare eval datasets

    Produce consistent, metadata-tagged chunks as inputs for retrieval or summarization experiments.

    Common mistakes

    Mistake:Setting overlap larger than the chunk size.

    Fix:Overlap should be a fraction of the chunk size — too much duplicates content across chunks and wastes tokens.

    Mistake:Choosing character splitting for prose.

    Fix:Use the recursive strategy for natural text so chunks break on paragraph and sentence boundaries instead of mid-word.

    Mistake:Treating the token estimate as exact.

    Fix:The per-chunk token count is an approximation; use your model's tokenizer for billing-critical counts.

    Frequently asked questions

    References & standards

    Related tools

    Context Assembler

    Combine files and folders into one LLM-ready context block with XML tags, a project tree, ignore filters and a token estimate — like repomix or files-to-prompt, in your browser.

    Prompt Template Builder

    Write reusable prompt templates with {{variables}}, auto-fill the values, and copy the compiled prompt. Save your library of prompts locally — no AI calls, everything in your browser.

    Token Cost Calculator

    Compare LLM API costs across models with cache, batch, reasoning and traffic simulation — count tokens with a real tokenizer, refresh prices live, and export CSV/JSON/Markdown