Why fixed-size splitting fails on structured documents, and what to do instead.
Splitting documents every thousand characters is the default in every tutorial, and it is the reason a great many retrieval systems answer badly on real corporate documents.
Fixed-size splitting breaks meaning
It cuts tables in half, separates a heading from the text it governs, and splits a procedure between step four and step five. The resulting chunk is retrievable and useless, because the sentence that made it meaningful is in a different chunk.
Split on structure
Use the document's own boundaries: headings, sections, list items, table rows, slide breaks. A chunk that corresponds to a real section is self-contained by design. For long sections, split at paragraph boundaries within them rather than at a character count.
Carry context into the chunk
Prefix each chunk with its document title and heading path, so a fragment about eligibility still says which policy it belongs to. Modest overlap between adjacent chunks helps for prose; for tables and structured data, keep the header row with every chunk instead.
Different document types need different strategies in the same system. A single chunking function applied to contracts, spreadsheets and slide decks is a guaranteed source of wrong answers.
Want this for your business?
Let's talk about how we can help you build and grow.
