Building Multimodal Data Pipelines
by DeepLearning.AI
Start your RAG pipeline one step earlier — at the images, audio and video, using OCR, ASR and VLMs to make them retrievable.
Overview
Building Multimodal Data Pipelines is a one-hour DeepLearning.AI short course built with Snowflake and taught by Gilberto Hernandez, a Lead Developer Advocate there. It sits upstream of the usual RAG tutorial: instead of starting from a folder of clean text, it starts from the images, audio and video that most enterprise data actually is, and turns them into LLM-ready text. Seven lessons move in order. The first frames what multimodal data is and where extraction breaks down. The second combines automatic speech recognition for audio transcripts with OCR for slides and scanned documents, then embeds the results so they are semantically searchable. The third and fourth build a vision-language-model pipeline over video, sampling frames, generating timestamped scene descriptions, and tracking events across a recording so a video becomes queryable by what happened and when. The fifth assembles those pieces into a multimodal RAG application over a real meeting-recording dataset — slides, audio and video together — returning grounded answers with citations back to the source modality and timestamp. The stack is Python and SQL over Snowflake's ASR, OCR, VLM and vector-search primitives, and the pipeline patterns transfer to any equivalent stack. Coursera lists the course at Intermediate level and estimates about two hours with the labs. It is short enough for a single sitting, and is aimed squarely at engineers whose retrieval systems currently ignore every non-text asset their organization owns.
At a Glance
- Topic
- RAG
- Level
- Intermediate
- Format
- Course
- Cost
- Freemium
- Duration
- ~1 hour (7 lessons, 1h1m of video), self-paced
- Provider
- DeepLearning.AI
- Hands-on
- Yes — code/exercises
- Certificate
- Available
What You’ll Learn
- ✓Transcribe audio with automatic speech recognition and turn images into LLM-ready text descriptions
- ✓Extract structured, queryable text from scanned documents and presentation slides using OCR
- ✓Generate timestamped scene descriptions by reasoning across video frames with a vision-language model
- ✓Build a VLM-backed pipeline that tracks events over time across an entire recording
- ✓Embed mixed-modality content and run semantic search spanning slides, audio and video together
- ✓Assemble a multimodal RAG application that returns grounded answers with citations to the source
- ✓Design ingestion and feature-extraction stages that replace manual tagging of unstructured assets
Highlights
- •Goes upstream of the standard RAG tutorial — this is ingestion and extraction, not another chunk-and-embed walkthrough
- •Taught by Gilberto Hernandez, Lead Developer Advocate at Snowflake, using production data-platform primitives rather than notebook toys
- •A single pipeline spans three modalities end to end and finishes as a queryable, cited interface over real meeting recordings
- •Citations resolve back to modality and timestamp, so a video answer points at the moment it came from
- •About an hour of video, which makes it a realistic single-sitting fix for the most common blind spot in an existing RAG system
Who It’s For
Best For
- ✓Data engineers building ingestion for unstructured enterprise assets
- ✓RAG builders whose retrieval quality is capped because non-text sources are simply skipped
- ✓ML engineers implementing search over video, meeting recordings or scanned document archives
Prerequisites
- •Working Python and the ability to read and write SQL queries
- •Basic data-engineering concepts — pipelines, batch processing, embeddings and vector search
- •Familiarity with how a text RAG system works, since this course extends one rather than introducing it
FAQ
What is Building Multimodal Data Pipelines?
A one-hour course on the ingestion stage that most RAG tutorials assume away. It covers extracting queryable text from images and scanned slides with OCR, transcribing audio with ASR, and reasoning across video frames with a vision-language model to produce timestamped scene descriptions — then assembles all three into a multimodal RAG app that answers with grounded citations.
Is Building Multimodal Data Pipelines free?
Building Multimodal Data Pipelines offers free content, with paid options for certificates or premium features.
What level is Building Multimodal Data Pipelines for?
Building Multimodal Data Pipelines is aimed at a intermediate audience. Recommended background: Working Python and the ability to read and write SQL queries, Basic data-engineering concepts — pipelines, batch processing, embeddings and vector search, Familiarity with how a text RAG system works, since this course extends one rather than introducing it.
How long does Building Multimodal Data Pipelines take?
Expect roughly ~1 hour (7 lessons, 1h1m of video), self-paced. Most learners work through it at their own pace.
What will I learn from Building Multimodal Data Pipelines?
You'll learn: Transcribe audio with automatic speech recognition and turn images into LLM-ready text descriptions; Extract structured, queryable text from scanned documents and presentation slides using OCR; Generate timestamped scene descriptions by reasoning across video frames with a vision-language model; Build a VLM-backed pipeline that tracks events over time across an entire recording; Embed mixed-modality content and run semantic search spanning slides, audio and video together; Assemble a multimodal RAG application that returns grounded answers with citations to the source; Design ingestion and feature-extraction stages that replace manual tagging of unstructured assets.
Topics
Sources
This page was written from 2 sources, 1 on domains other than deeplearning.ai.