# `TextChunker.Strategies.RecursiveChunk`
[🔗](https://github.com/revelrylabs/text_chunker_ex/blob/main/lib/text_chunker/strategies/recursive_chunk/recursive_chunk.ex#L1)

Handles recursive text splitting, aiming to adhere to configured size and overlap requirements.
Employs a flexible separator-based approach to break down text into manageable chunks, while generating metadata for each produced chunk.

**Terminology**

* **Split:** An intermediate text fragment produced by breaking text on a separator. A split may still exceed `chunk_size` and require further splitting.
* **Chunk:** A final, correctly-sized text segment returned to the caller.

Both are represented by `%TextChunker.Chunk{}` structs internally; the names refer to their role in the algorithm.

**Key Features:**

* **Size-Guided Chunking:** Prioritizes splitting text into semantic blocks while respecting the maximum `chunk_size`.
* **Context Preservation:** Maintains `chunk_overlap` to minimize information loss at chunk boundaries.
* **Separator Handling:** Selects the most appropriate delimiter (e.g., line breaks, spaces) based on the text content.
* **Metadata Generation:** Creates `%TextChunker.Chunk{}` structs containing the split text and its original byte range.

**Algorithm Overview**

1. **Separator Prioritization:**  Establishes a list of potential separators (e.g., line breaks, spaces), ordered by their expected relevance to the text structure.
2. **Recursive Splitting:**
  *  Iterates through the separator list.
  *  Attempts to split the text using the current separator.
  *  If a split is successful, recursively applies the algorithm to any resulting splits that still exceed the `chunk_size`.
3. **Chunk Assembly:**
  *  Combines splits into chunks, aiming to get as close to the `chunk_size` as possible.
  *  Employs `chunk_overlap` to ensure smooth transitions between chunks.
4. **Metadata Generation:**  Tracks byte ranges for each chunk for potential reassembly of the original text.

# `split`

```elixir
@spec split(
  binary(),
  keyword()
) :: [TextChunker.Chunk.t()]
```

Internal recursive chunking strategy. Use `TextChunker.split/2` for public API.

Splits text using prioritized separators, respecting `chunk_size` limits while
maintaining `chunk_overlap` for context preservation.

## Options

* `:chunk_size` (integer) - Maximum chunk size
* `:chunk_overlap` (integer) - Overlap between chunks
* `:format` (atom) - Text format for separator selection
* `:get_chunk_size` (function) - Size calculation function (required)

---

*Consult [api-reference.md](api-reference.md) for complete listing*