My portfolio deploys on Vercel. Vercel doesn't give you a GPU. PersonaPlex — the full-duplex voice model behind this site's real-time mode — needs one. That mismatch produced the most operationally interesting part of this project: an on-demand GPU pod lifecycle managed entirely from serverless functions and Redis.
The shape of the problem
A GPU pod costs money for every second it exists. A portfolio has long stretches of zero traffic punctuated by one curious visitor. Keeping a pod warm 24/7 would be absurd; starting one on demand means cold starts measured in tens of seconds to minutes. Everything in the design follows from those two facts.
Starting exactly one pod
Warmup is a POST to /api/personaplex/warmup, fired in the background as soon as a normal voice call starts. Two tabs, or two serverless function instances, could easily fire it at once — so the route takes a distributed lock in Upstash Redis (SET NX, 180-second TTL). Whoever holds the lock checks RunPod for an existing pod and starts one only if none exists.
The subtle part: the lock is deliberately not released on success. RunPod's API takes a while to list a freshly started pod, so releasing early would let a second caller double-start. The lock simply expires after covering the worst-case provisioning window. It's released on error, so a failure doesn't block retries for three minutes.
The client then polls a status route every couple of seconds until the pod's WebSocket endpoint is live.
Keeping it alive — and making sure it dies
While a real-time session is active, the browser refreshes a heartbeat key in Redis every 30 seconds with a 90-second TTL. That one key feeds two independent kill mechanisms:
- The watchdog. Inside the Docker image, a small Python sidecar polls the heartbeat key every 30 seconds. After a 120-second startup grace period, two consecutive misses mean the visitor is gone — crashed tab, dead network, phone swiped away — and the watchdog calls RunPod's API to terminate its own pod.
- The reaper. A Vercel cron hits
/api/personaplex/reaper, which lists every pod under the template, skips anything still in its boot grace period, and terminates any pod with no heartbeat. This catches orphans the watchdog missed, and it fails closed: if Redis is unreachable, it assumes the worst and terminates, because leaking GPU credits is the expensive error.
The container entrypoint ties the pod's lifecycle to the inference server itself: if the moshi server process exits, the container exits, and RunPod decommissions the pod.
Cold starts and codecs
Cold start is the tax you pay for on-demand. I hide it rather than remove it — warmup begins in the background during the managed VAPI conversation, so by the time someone wants real-time mode, the pod is usually ready. When it isn't, the UI shows a warming state and nothing breaks.
On the wire, everything is Opus in an OGG container at 24kHz mono, streamed as binary WebSocket frames with a one-byte type prefix. Opus at that rate is light enough for hotel Wi-Fi while staying intelligible, and the opus-recorder encoder in VOIP mode runs fine in a web worker.
What this actually costs me
Money-wise: a handful of pod-minutes per real-time session, nothing when idle. Complexity-wise: a warmup lock, a heartbeat, two termination paths, a custom Docker image, and a patch to make the upstream server bind 0.0.0.0 so RunPod's proxy can reach it.
Is that proportionate for a portfolio? No. But it's an honest answer to a real question — "how do you run GPU inference on a serverless budget?" — and I'd rather demonstrate that answer than describe it.