AI / Data Engineer2025

PodcastIQ

An AI platform that makes podcast audio searchable, analyzable, and verifiable — semantic search across 290+ episodes powered by a 9-agent LangGraph system, a Neo4j knowledge graph, and Snowflake Cortex.

GitHub Slide Deck

25
Channels: 286
Episodes: 44
Months covered: 13,807
Chunks: 768
Vector dimensions: 100%
Embedded

See it run

Recorded trace, then the video

Eight queries, one per intent. Each replays the real execution trace: guardrails, routing, agent steps and measured timings.

PodcastIQ

Agent console

recorded runs, not live

Pick a query

> What did experts say about AGI?

Full walkthrough

The problem

Audio is not queryable

Cannot Search
100 episodes = 100 hours of listening
Cannot Track change
No record of a revised prediction
Cannot Attribute
No link from a claim to a speaker

Three phases

11 stages

01
Ingest
Data engineering
In
YouTube captions
Out
Queryable chunk table
5 stages
02
Enrich
AI engineering
In
Raw chunks
Out
Attributed, linked claims
4 stages
03
Serve
Agent runtime
In
A user question
Out
Guarded, cited answer
2 stages

Stage by stage

Decisions, not descriptions

Phase 1Ingest

Data engineering

YouTube captionsQueryable chunk table

Phase 2Enrich

AI engineering

Raw chunksAttributed, linked claims

Phase 3Serve

Agent runtime

A user questionGuarded, cited answer

Arrow keys to step

Ingest01 of 11

Extraction

yt-dlp + YouTube Data API v3

25 channels, 6 genres
Filter: 30 min or longer, post-2022
WebVTT captions plus metadata
Resumable via checkpoint JSON

Decision

Chose: Captions
Over: Audio download plus Whisper

Because

No GPU
No transcription cost
Timestamps come free

Cost

Inherits YouTube caption errors. Rules out acoustic diarization.

286
episodes: 146 MB
JSON: 0
failures

Agent runtime

1 router, 8 specialists

Routerllama3.1-8b87.5% on 48 labelled queries (8b, shipped)

Classifies into one of eight intents, then dispatches. Unrecognised output falls back to SEARCH.

SEARCH1.5s
Search
no LLM
Vector retrieval over Cortex Search.
“What did Sam Altman say about GPT-5?”
chained pair
SUMMARIZE3.5s
Summarization
llama3.1-70b
Synthesises retrieved chunks, cited back to timestamps.
“Best strategies for building a startup?”
chained pair
GRAPH4.5s
Knowledge graph
70b + Cypher
Generates Cypher, traverses Neo4j, retries on malformed output.
“Who has Sam Altman appeared with?”
TEMPORAL3.8s
Temporal
SQL + 70b
Reads pre-computed drift pairs.
“How has AGI opinion changed over time?”
FACTCHECK2.5-5.5s
Fact-check
Cortex + Brave
Cortex pre-filter first; only uncertain claims reach the web.
“Fact check: GPT-5 released in 2024”
COMPARE4.0s
Comparison
2 SQL + 70b
Pulls each side independently, then contrasts.
“Compare Lex Fridman vs Joe Rogan on AI safety”
RECOMMEND2.0s
Recommendation
SQL + 70b
Falls back through looser filters so narrow asks still return.
“Suggest episodes about AI”
INSIGHT4.2s
Insight
5 SQL + 70b
Aggregates across the corpus to answer questions about the corpus.
“Which channel has the most contradicted claims?”

Architecture

Full system

Scroll to follow the flow

Cost per query

$1.19 per 1,000

Estimated cost by agent, USD

Summarize

0.002183,000 in / 600 out tokens

Compare

0.002123,000 in / 500 out

Recommend

0.001462,000 in / 400 out

Fact-check

0.001402,000 in / 300 out

Temporal

0.0011670b plus 8b

Insight

0.001151,500 in / 400 out

0.00002embedding only, no generation

Graph

0.00002embedding only, no generation

Search and Graph are effectively free: both retrieve without a generation step. Summarize costs the most because it passes 5 to 8 retrieved chunks into the prompt.

Stack

Everything used

Warehouse and modelling

Snowflake
dbt
SQL
Snowflake Cortex

Graph

Neo4j
Cypher
Docker

Models

llama3.1-70b
llama3.1-8b
arctic-embed-m
GPT-4o

AI orchestration

LangGraph
Cortex Search
GraphRAG
Brave Search API

Ingestion

Python
yt-dlp
YouTube Data API v3
ydata-profiling

Interface

Streamlit

See these across every project