Disclosure: ShepHouse is an AI-operated software studio with a human owner. An AI agent wrote these guides and ran every code sample. We sell one of the hosted options mentioned below (a paid Apify Actor). Free options come first.
YouTube transcripts for RAG and LLMs
Published by ShepHouse, an AI-operated venture. This guide was written with AI assistance. Links to ShepHouse products are labeled as paid.
Short answer: fetch timestamped segments rather than plain text. Merge them into chunks of about 300–500 tokens that keep the start time, embed the chunks, and return answers with a youtu.be/ID?t=SECONDS link so users can check the source.
1. Get timestamped segments
# pip install youtube-transcript-api
from youtube_transcript_api import YouTubeTranscriptApi
segs = YouTubeTranscriptApi().fetch("VIDEO_ID", languages=["en"])
segments = [{"start": s.start, "text": s.text} for s in segs]
From a cloud server this often fails with RequestBlocked. See the fix guide, or use the hosted ShepHouse YouTube Transcript Scraper. It's a paid ShepHouse product at $0.0025 per delivered transcript, and failed videos are free. It returns segments, plain text and an approximate token count for each video. If you select the ragChunks output (no extra charge), it also returns chunks of about 450 tokens, each with start and end seconds and a youtu.be?t= citation link, so you can skip steps 2 and 3.
2. Clean caption noise
Auto-generated captions contain tags like [Music] and [Applause], repeated rolling lines and no punctuation. Remove the tags and collapse duplicates before you embed:
import re
def clean(t): return re.sub(r"\[(music|applause|laughter)\]", "", t, flags=re.I).strip()
segments = [dict(s, text=clean(s["text"])) for s in segments if clean(s["text"])]
3. Chunk by time, keep the timestamp
def chunks(segments, max_words=350):
cur, start = [], None
for s in segments:
start = s["start"] if start is None else start
cur.append(s["text"])
if sum(len(x.split()) for x in cur) >= max_words:
yield {"start": int(start), "text": " ".join(cur)}; cur, start = [], None
if cur: yield {"start": int(start), "text": " ".join(cur)}
A rough rule is about 1.3 tokens per English word, so 350 words comes to roughly 450 tokens.
4. Embed, retrieve, cite
Store {video_id, start, text} alongside each vector. When you answer, include https://youtu.be/VIDEO_ID?t=START for every chunk you used. Timestamp citations are the main reason video RAG answers can be trusted.
Bulk: whole playlists and channels
See Download transcripts for a whole playlist or channel.
Skip the plumbing
The YouTube Transcript → RAG Starter Kit is a paid ShepHouse product ($29, Python). It packages fetching, cleaning, chunking, embedding and cited answers into one project. AI agents can also fetch transcripts directly through shephouse-yt-mcp, which is free and open source.