Unstructured Documentation — Document Preprocessing and ETL for RAG
by Unstructured
The ingestion half of RAG, documented properly: partition strategies, typed elements, element metadata and document-aware chunking.
Overview
Unstructured is an open-source ETL layer for documents, and its documentation is organized around the five operations the platform names: partition, extract, enrich, chunk and embed. The open-source library — Apache-2.0, roughly 15.4k GitHub stars, and released on a near-weekly cadence with v0.27.5 shipping in late August 2026 — is documented separately from the hosted API and platform, which is the split that matters when deciding what you can run yourself. The core-functionality guides carry the substance. Partitioning documents a universal partition() that auto-detects file type, plus per-format functions for PDF, HTML, email, Word, PowerPoint, Excel, EPUB, Markdown, images and more, and for PDFs and images four explicit strategies: fast (pdfminer text extraction), hi_res (a layout model), ocr_only (Tesseract) and vlm. Output is typed elements — Title, NarrativeText, ListItem, Table, Header and Footer, plus email-specific Subject, Sender and Recipient — each carrying metadata such as page number, header/footer type and text_as_html for tables. The chunking guide then explains document-aware chunking: because the chunkers consume partition metadata rather than raw text, the basic and by_title strategies group semantic units and only fall back to text splitting when a single element exceeds max_characters, with overlap, combine_text_under_n_chars and multipage_sections as controls. Connector docs cover 20+ sources and 20+ destinations; staging is deprecated in favour of destination connectors.
At a Glance
- Topic
- RAG
- Level
- Intermediate
- Format
- Documentation
- Cost
- Freemium
- Duration
- ~3-4 hours to work through the open-source core-functionality guides; reference docs for ongoing use
- Provider
- Unstructured
- Hands-on
- Yes — code/exercises
- Certificate
- None
What You’ll Learn
- ✓Choose between the fast, hi_res, ocr_only and vlm PDF partitioning strategies
- ✓Turn raw documents into typed elements such as Title, NarrativeText and Table
- ✓Use element metadata like page number and text_as_html in downstream retrieval
- ✓Apply document-aware chunking with basic and by_title instead of naive text splitting
- ✓Tune max_characters, overlap and combine_text_under_n_chars for retrieval quality
- ✓Wire ingest source and destination connectors into a repeatable document pipeline
- ✓Decide where the open-source library suffices and where the hosted API earns its cost
Highlights
- •Genuinely current — v0.27.5 shipped in late August 2026, on a roughly weekly release cadence
- •Apache-2.0 with about 15.4k stars, so the library is properly self-hostable rather than a demo funnel for a SaaS
- •Documents chunking as a consumer of layout metadata, which is exactly the step most RAG tutorials skip
- •Names the real backend behind each PDF strategy — pdfminer, a layout model, Tesseract, a VLM — so you can predict cost and accuracy before running it
- •Publishes an llms.txt documentation index and an agent guide, so a coding agent can navigate the docs directly
Who It’s For
Best For
- ✓Engineers building RAG over real enterprise PDFs, email and Office files
- ✓Teams whose retrieval quality is capped by ingestion, not by the model
- ✓Data engineers assembling a document ETL pipeline with source and destination connectors
- ✓Anyone comparing Docling, MinerU and Unstructured for document parsing
Prerequisites
- •Working Python, plus comfort installing OCR and layout-model system dependencies
- •Basic understanding of embeddings, chunking and vector search
- •A vector store or other destination in mind if you follow the connector guides
FAQ
What is Unstructured Documentation — Document Preprocessing and ETL for RAG?
The official documentation for Unstructured, the Apache-2.0 Python library and hosted platform that turns PDFs, email, scanned images and Office files into structured JSON elements ready for a vector store. It is for engineers building RAG or fine-tuning pipelines whose retrieval quality is limited by ingestion rather than by the model: layout detection, OCR strategy, element metadata and chunking are all documented function by function.
Is Unstructured Documentation — Document Preprocessing and ETL for RAG free?
Unstructured Documentation — Document Preprocessing and ETL for RAG offers free content, with paid options for certificates or premium features.
What level is Unstructured Documentation — Document Preprocessing and ETL for RAG for?
Unstructured Documentation — Document Preprocessing and ETL for RAG is aimed at a intermediate audience. Recommended background: Working Python, plus comfort installing OCR and layout-model system dependencies, Basic understanding of embeddings, chunking and vector search, A vector store or other destination in mind if you follow the connector guides.
How long does Unstructured Documentation — Document Preprocessing and ETL for RAG take?
Expect roughly ~3-4 hours to work through the open-source core-functionality guides; reference docs for ongoing use. Most learners work through it at their own pace.
What will I learn from Unstructured Documentation — Document Preprocessing and ETL for RAG?
You'll learn: Choose between the fast, hi_res, ocr_only and vlm PDF partitioning strategies; Turn raw documents into typed elements such as Title, NarrativeText and Table; Use element metadata like page number and text_as_html in downstream retrieval; Apply document-aware chunking with basic and by_title instead of naive text splitting; Tune max_characters, overlap and combine_text_under_n_chars for retrieval quality; Wire ingest source and destination connectors into a repeatable document pipeline; Decide where the open-source library suffices and where the hosted API earns its cost.
Topics
Sources
This page was written from 4 sources, 1 on domains other than docs.unstructured.io.