Streaming AI clinical notes on AWS Lambda
· Ciaran McCaughey
A vet finishes a consult, taps “stop recording” on their phone, and watches the SOAP note appear word by word in the browser before they’ve put the dog back in the car. That’s the experience we’re chasing. Here’s what’s behind it.
The pipeline
audio.m4a (S3)
→ transcribe Lambda (AssemblyAI primary, Deepgram fallback)
→ classifier (Gemini 3 Flash)
→ fan-out: key-information, patient/client, notes, client summary, billable items
→ WorkflowEvents (DynamoDB for replay + WebSocket for live tail)Every stage is an Effect workflow running inside a single Lambda. The WorkflowEvent is the unit of progress — each one is appended to DynamoDB (so we can replay the entire run from the beginning) and pushed onto a WebSocket channel (so the browser can render as the tokens arrive).
Why one Lambda, not a step function
We started with Step Functions. It was the wrong shape. Each step incurred 150–400ms of orchestration latency, billing got murky, and the local development story was painful — you can’t easily replay a Step Function execution against a code change.
Collapsing into one long-running Lambda gave us:
- One process to reason about. A workflow is just an Effect program; tests run it in-memory.
- Sub-second orchestration. Stages hand off via Effect, not via DynamoDB writes and EventBridge hops.
- Cheaper. No state-transition pricing, no orchestration GB-seconds.
The trade-off is the 15-minute Lambda timeout. We’ve never hit it; transcription dominates wall time and AssemblyAI streams partials, so we know within seconds if it’ll fit.
What’s next
We’re working on durable streams next — letting a vet close their laptop mid-consult and have the note finish generating server-side, ready to read when they reopen. More on that soon.