No Docker rebuild: add a decorator to a Python function and throw the code onto a GPU cloud with hot reload, right from your local IDE
GPU Cloud Deployment Without Leaving Your IDE — Audry Hsu, RunPod · Audry Hsu
20 min total·Actually worth watching closely: ~7 min·3 must-watch clips
- 0:00 – 5:31Listen
From spare mining GPUs to $120M ARR
RunPod's origin story (spare GPUs from a failed 2022 crypto mining venture, plus a Reddit post trading free GPUs for feedback) and a tour of the four product lines — Pods, Serverless, Clusters, Hub — now spanning 30-plus data centers across 10 countries.
Four product lines for four usage shapes: reserved GPUs, elastic functions, multi-node training, one-click deploys of pre-vetted models
Pure spoken narration over standard intro slides; nothing critical on screen, fine to listen to like a podcast▶ Jump to 0:00Speaker · Audry Hsu - 5:43 – 9:01Listen
Flash's split philosophy
Names the real bottleneck in GPU development as infrastructure rather than models (CUDA/PyTorch version alignment, a full image cycle for every change), then introduces Flash: add the endpoint decorator to an async function and the function body runs on the cloud while everything around it stays local.
The local/cloud split happens at the granularity of a single function, with no cloud infrastructure to set up first
Mostly conceptual, low information density on screen; at 502s you can glance at the terminal as flash run spins up the local FastAPI server▶ Jump to 5:43Speaker · Audry Hsu - 9:06 – 12:17Watch
Live demo: decorator config and the first request
Walks through the declarative parameters of the endpoint decorator (endpoint name, GPU family, max workers, always-on active workers) and sends a request to the local endpoint, watching the cloud job get seen, started, and queued in the terminal in real time.
All deployment config converges into the decorator's parameters, and scaling behavior follows from that declaration
Dense with visual moments (626s live terminal, 668s decorator parameters, 719s the prompt-argument stumble); the code and terminal are what make the mechanism click▶ Jump to 9:06Speaker · Audry Hsu - 12:21 – 16:18Watch
Hot-swapping the model and the three-model pipeline
Demonstrates hot file reload — SDXL Turbo swapped for Dreamshaper with the inference steps bumped, then the same request re-sent — and then strings together a three-stage pipeline: Qwen3 writes the prompts, Dreamshaper generates, Nano Banana 2 composes.
The complexity of a real AI app is multi-model orchestration; open-source, self-hosted, and commercial models can be mixed in one pipeline
The highest-value stretch of the talk; the 776s hot-swap and the 903s pipeline both need you to see the code change and compare the generated results▶ Jump to 12:21Speaker · Audry Hsu - 16:24 – 19:43Skim
Pricing numbers and choosing between Pods and Serverless
Gives the number for Serverless billing by actual run time (an H100 at about $0.00116 per second) and the premium logic versus Pods, and recommends Pods while you are experimenting, with Serverless reserved for variable loads running hundreds of workers across data centers. Interleaved with the before/after of Qwen3's prompt engineering.
The Serverless premium is what buys you auto-scaling; while experimenting, by-the-second Pods are the better deal
Mostly slide numbers and spoken advice — fine to skim past the pricing page; the 1110s before/after image comparison is worth a pause▶ Jump to 16:24Speaker · Audry Hsu - 19:46 – 20:01Listen
Wrap-up: you can start locally
Sums up where Flash sits: start right in your local development environment, and deploy open-source or your own private models.
The on-ramp is starting in your local development environment and getting one function working, then thinking about production shape
A 15-second spoken close; nothing on screen to depend on▶ Jump to 19:46Speaker · Audry Hsu