Building AI Assistants with On-Device Memory
by DeepLearning.AI and Qdrant
Build a private, offline multimodal memory for an AI assistant with Qdrant Edge, CLIP, YOLO and Whisper.
Overview
Building AI Assistants with On-Device Memory is a DeepLearning.AI short course made in partnership with Qdrant and taught by Dylan Couzon, Developer Experience Engineer at Qdrant, with contributions from Neil Kanugo and Esmaeil Gargari. It runs about an hour and is listed as Intermediate. The lesson sequence is: Introduction; Overview of On-Device AI Memory; Inside the AI Assistant Device; Building the Memory (code); Building the Voice Assistant (code); Learning Through Memory (code); Running on a Local Computer; Conclusion; a practice exercise in the AI Coding Lab; a graded quiz; and optional material on a hybrid of on-device and cloud memory plus a glossary. The core idea is that every note, photo or recording becomes a vector stored locally that can be searched by meaning, tagged with metadata context and deleted on command, covering the full memory cycle of capture, encode, store, recall and forget. The companion GitHub repository uses Qdrant Edge (v0.7.2, in beta) as the embedded vector store, CLIP embeddings and YOLO for vision, and Whisper for local transcription, with pinned package versions, sample data and Python 3.12+. After the initial model downloads it needs no API keys or accounts and runs on an ordinary Mac or Windows PC. The course assumes Python and basic familiarity with embeddings and vector search.
At a Glance
- Topic
- RAG
- Level
- Intermediate
- Format
- Course
- Cost
- Freemium
- Duration
- ~1 hour 4 minutes of lessons plus optional notebooks, self-paced
- Provider
- DeepLearning.AI and Qdrant
- Hands-on
- Yes — code/exercises
- Certificate
- None
What You’ll Learn
- ✓Build an on-device memory system that captures experiences as vectors locally
- ✓Store and retrieve text memories by semantic meaning rather than keywords
- ✓Filter recalled memories with metadata such as time, source and context tags
- ✓Use object detection and CLIP embeddings to index and search image memories
- ✓Teach the assistant to recognize new objects from stored examples without retraining
- ✓Build an offline voice assistant that answers questions about personal experiences
- ✓Implement the full memory cycle: capture, encode, store, recall and forget
Highlights
- •Fully offline after model download — no API keys, no cloud, user data never leaves the device
- •Multimodal memory across text, voice (Whisper) and images (CLIP, YOLO) in one system
- •Uses Qdrant Edge, an embedded on-device vector engine, rather than a hosted vector database
- •Short (~1 hour) with runnable notebooks in a public GitHub repo and an optional hybrid on-device/cloud sync lesson
Who It’s For
Best For
- ✓Developers building privacy-sensitive or offline AI assistants
- ✓Engineers adding long-term memory to agents on edge devices
- ✓RAG practitioners wanting to extend retrieval to images and voice
Prerequisites
- •Python familiarity
- •Basic understanding of embeddings and vector search
FAQ
What is Building AI Assistants with On-Device Memory?
A DeepLearning.AI short course built with Qdrant for developers who want AI assistants that work offline and keep user data on the device. You build a local memory layer that stores, retrieves, filters and forgets text, voice and image memories by meaning, and an assistant that recognizes new objects from stored examples without retraining.
Is Building AI Assistants with On-Device Memory free?
Building AI Assistants with On-Device Memory offers free content, with paid options for certificates or premium features.
What level is Building AI Assistants with On-Device Memory for?
Building AI Assistants with On-Device Memory is aimed at a intermediate audience. Recommended background: Python familiarity, Basic understanding of embeddings and vector search.
How long does Building AI Assistants with On-Device Memory take?
Expect roughly ~1 hour 4 minutes of lessons plus optional notebooks, self-paced. Most learners work through it at their own pace.
What will I learn from Building AI Assistants with On-Device Memory?
You'll learn: Build an on-device memory system that captures experiences as vectors locally; Store and retrieve text memories by semantic meaning rather than keywords; Filter recalled memories with metadata such as time, source and context tags; Use object detection and CLIP embeddings to index and search image memories; Teach the assistant to recognize new objects from stored examples without retraining; Build an offline voice assistant that answers questions about personal experiences; Implement the full memory cycle: capture, encode, store, recall and forget.
Topics
Sources
This page was written from 3 sources, 2 on domains other than deeplearning.ai.