Chunk Documents Around Meaning Before Tuning Size

A practical approach

Step 1

Inspect several real documents and identify their natural sections. Keep titles, source identifiers and relevant hierarchy as metadata.

Step 2

Create a simple baseline, then compare a structure-aware alternative. Make sure tables, lists and short policy exceptions survive the transformation.

Step 3

For each test question, inspect the retrieved passage before judging the generated answer. A missing condition is a retrieval problem even when the prose sounds convincing.

Worked example

A cancellation policy has a general rule followed by an exception for rescheduled events. Splitting the exception away from the rule can make the assistant promise the wrong refund treatment. Retaining the heading and retrieving adjacent context may help, but the result must be tested on both normal and exception cases.

A mistake to avoid

Increasing overlap can duplicate passages and consume the context budget without improving evidence. Also check whether the parser preserved the original document correctly; sophisticated chunking cannot repair missing table headers.

Try it yourself

Choose five questions about one document. Compare fixed-size chunks with section-based chunks and record whether each retrieved result contains the full condition needed for the answer.

Your implementation checklist

• Headings and source IDs survive parsing.
• Exceptions remain connected to their rules.
• Retrieval is inspected before generation.
• Overlap has a measured purpose.