Voice infrastructure · Applied AI · September 2026
I built a voice receptionist that can check appointment availability and reserve a slot. LiveKit carries the conversation; Python tools turn the caller’s request into a change in Redis.
“Do you have anything on Tuesday?”
check_appointments("tuesday")“Available on Tuesday: 10:00, 14:30.”
Next: portfolio Q&A
A portfolio Q&A agent is coming soon. Below, explore the original scheduling demo’s architecture and interactive walkthrough.
01 / The problem
A receptionist needs more than a convincing voice. If someone asks for an appointment, the answer has to come from the available slots—not from what a model thinks the schedule might be.
I used a fictional clinic as a small, concrete workflow: hear a request, ask the backend, and return the result to the caller. The speech model handles the conversation. The application owns availability and the reservation decision.
02 / System design
One real-time conversation, with a separate path for application actions.
Speak a request.
Hear the answer.
Room + agent session.
Connect the audio.
Understand the request.
Choose a registered tool.
check_appointments(day)
book_appointment(day, time)Pass an explicit action and structured state into the graph.
Conditional routing. Each branch returns a result, then ends.
appointments:tuesdayLRANGE reads the slots.
LREM removes a matching slot.
The LiveKit room is the transport boundary. The graph receives tool arguments, not raw audio. A successful reservation means a matching slot was removed from the demo’s Redis list. The hosted worker runs separately from LiveKit Cloud, with a private Redis socket and an isolated, expiring schedule for each call.
03 / Inside a call
Step through the same check-and-book logic. This is a scripted illustration in your browser, not a live call or a real appointment.
“What appointments are available on Tuesday?”
The speech model hears a request to check Tuesday. The application has not read or changed any slots yet.
"What appointments are available on Tuesday?"appointments:tuesdayTwo example slots. Nothing has changed.
Step 1 of 4
04 / Implementation decisions
A LiveKit AgentServer registers the receptionist; each room starts an AgentSession using OpenAI’s Realtime model. Instructions keep the interaction brief and fictional, and require an availability tool call instead of an improvised answer.
Both Python functions invoke a compiled LangGraph workflow with action, requested_day, time, and result. The action chooses the check or book node. It is a small conditional workflow, not a multi-agent system.
Checking reads a day’s Redis list. Booking uses LREM to remove one matching time and tests the removal count before reporting success. If no slot matched, the tool returns “not available.” The model receives that result for its reply.
The worker runs as a non-root service. Blocking graph calls move to a worker thread; Redis has short timeouts and no public port. Each call gets fresh weekday slots with a ten-minute expiry. Calls have a three-minute limit, idle shutdown, and a daily usage cap. Session recording is disabled in the agent.
Tests cover input validation, isolated call schedules, temporary-data cleanup, Redis failures, and simultaneous reservations where only one request succeeds. This remains fictional availability, not a patient booking system: durable records, idempotency, and external calendar integration are future work.
05 / The follow-on build
I then carried LiveKit and OpenAI Realtime into a separate Wedge interview build: a browser-based assistant that looks up the selected patient’s demographic record through an API.
A custom browser interface requests a room-scoped token from FastAPI. The server dispatches the agent with the selected patient ID in session metadata.
I built a Python SDK covering 18 read/search API actions. The voice agent exposes one demographics tool, with the blocking API call moved to a worker thread.
A patient-keyed, 300-second in-process cache avoids repeating the lookup. Five mocked tests cover the SDK and cache behavior; an end-to-end browser conversation was recorded.
These are two separate builds. The scheduling prototype uses LangGraph and Redis; the Wedge assistant uses its API client and a Python cache. They are not presented as one combined deployment.
The through-line
For me, the interesting work is the connection: a human request, a well-defined tool, an actual backend result, and a response the caller can understand.
Explore my other projects