Your AI Voice Agent Struggling With Complex Calls?
A client calls your physiotherapy clinic and wants to know if their Sun Life insurance covers both an assessment and a treatment on the same day — and whether a pre-approval form can be emailed before their Thursday appointment. Your AI voice agent stumbles, gives a partial answer, then redirects to voicemail. Client lost. Appointment missed.
On May 7, 2026, OpenAI launched GPT-Realtime-2 — the first voice model with GPT-5-class reasoning baked directly into the audio loop. This is not a cosmetic upgrade: the context window jumped from 32K to 128K tokens, parallel tool calls are now native, and reasoning effort is configurable per use case. In plain terms: your agent can now think while it talks, not after.
ElevenLabs — the platform we use to build AI voice agents for Quebec SMBs — lets you choose which LLM powers your agent. This guide walks you through exactly 5 steps to switch to GPT-Realtime-2, configure reasoning effort correctly, and avoid the two most expensive mistakes we see during deployment.
What GPT-Realtime-2 Actually Changes for a Quebec SMB
Before touching anything in your ElevenLabs dashboard, let us understand what changed — and why it matters to you specifically.
Previous voice models, including GPT-Realtime-1.5, chained steps: transcribe speech, run a text model, generate a response, convert to speech. Each handoff added latency and lost conversational nuance. GPT-Realtime-2 handles all of this inside a single audio-to-audio model. Reasoning happens during the conversation, not between turns.
For your Quebec SMB, this unlocks three call types that were unreliable before:
1. Multi-part questions: Do you have availability Friday morning for two people, and does the meeting room have a projector? The agent handles both sub-questions simultaneously, consults your tools in parallel, and replies in one coherent answer.
2. Complex information checks: A customer asking whether a specific service is included in their package, with particular conditions. With a 128K-token context window, the agent holds the full conversation history without losing the thread.
3. Bilingual calls with register shifts: A caller who switches from French to English mid-call, using industry-specific terms. GPT-Realtime-2 handles these transitions without mentally restarting the conversation — which was the primary weakness of its predecessor.
What You Need Before Starting
Before following the 5 steps, make sure you have:
An active ElevenLabs account with a configured agent — this guide assumes your agent is already running. If you are starting from scratch, our team at Agent IA Vocal handles the full initial setup.
Access to your agent dashboard — specifically the LLM or Model section in agent settings. The exact location varies slightly by your ElevenLabs version.
Your current system prompt — you will need it at Step 4. Pull it up now.
15 minutes for testing — Step 5 is not optional. Errors that surface after an LLM change almost always happen because someone skipped the test phase.
Step 1 — Access Your ElevenLabs Agent LLM Settings
Log into your ElevenLabs dashboard and navigate to the agent you want to upgrade. In the left sidebar, find Agent Settings and then the LLM tab. This is where your agent text-processing brain lives.
You will see a dropdown labeled Model or LLM Provider. This dropdown may currently show options like gpt-4o, claude-3-5-sonnet, gpt-realtime-1.5, or other models depending on your ElevenLabs subscription tier. This is where you will make your change.
One important note before proceeding: do not touch any other settings yet — not the voice, not the prompt, not the tools. Change only the model for now. That way, if something behaves unexpectedly, you will know exactly what caused it.
Step 2 — Select GPT-Realtime-2 as Your Active Model
In the model dropdown, select gpt-realtime-2. If you do not see it listed, there are two likely reasons: your ElevenLabs plan does not include it yet, or ElevenLabs is rolling out access gradually across accounts.
Once selected, a new section typically appears below: Reasoning Effort. This is the most important setting — and the one most people configure wrong. We cover it in Step 3.
Save this first change and immediately test with a simple call: Hi, what are your hours? If the agent responds correctly, you are ready to continue. If it fails to respond, verify that your OpenAI API key in ElevenLabs is tied to an account with access to Realtime models. Managing API keys securely in ElevenLabs is worth reviewing if you have not checked those settings since the v2.47 update.
Step 3 — Configure Reasoning Effort
This is the step nobody explains properly. GPT-Realtime-2 offers multiple reasoning levels: low, normal, high, and xhigh. The natural instinct is to set it to xhigh for the best performance. That is a mistake.
Here is why: reasoning effort determines how long the model takes before responding. In voice, every millisecond matters. The xhigh level creates noticeable pauses of 1.5 to 3 seconds before each response — which your callers will interpret as a broken or slow agent.
For the vast majority of Quebec SMBs, the recommended setting is low or normal depending on the complexity of your typical calls. Quick guide:
— Low: ideal for salons, restaurants, retail. Questions are direct — hours, availability, prices — and speed matters more than reasoning depth.
— Normal: the right balance for clinics, professional offices, and customer service with two or three-part questions. This is the default setting recommended in OpenAI official production voice agent documentation.
— High or xhigh: reserve this for specific use cases where precision absolutely trumps latency — live insurance coverage verification, loan qualification, or tier-2 technical support.
Our recommendation at Agent IA Vocal: always start at low, test your most complex call scenarios, and move up one level only if you spot errors on multi-step questions.
Step 4 — Update Your System Prompt for Reasoning-Aware Calls
GPT-Realtime-2 is not GPT-4o in a different costume. It reasons differently. That means your current system prompt might work fine — but it might also produce unexpected behavior if it was written to read scripts rather than solve problems.
Three specific adjustments to make in your system prompt:
1. Remove rigid scripting instructions. Old-style instructions like If the question is about X, respond exactly with [script] worked with less intelligent models. With GPT-Realtime-2, these instructions create conflicts between the model reasoning and forced scripts, producing incoherent responses. Replace with objectives: Your goal is to help the customer book an appointment by collecting these details in order: [list].
2. Add silence-handling instructions. When your agent consults a tool — checking a calendar, querying a database — GPT-Realtime-2 can now verbalize that action: Let me check your availability or One moment while I look that up. Enable this by adding to your prompt: When executing a tool, always verbally announce what you are doing in a brief phrase.
3. Define human escalation thresholds clearly. A smarter model can sometimes try to resolve situations that should be escalated to your team. Be explicit: If the request involves a complaint, a medical emergency, or expressed dissatisfaction, immediately transfer the call. As we detailed in our guide on RAG integration for AI voice agents, the quality of escalation instructions is often what separates a high-performing agent from a problematic one.
Step 5 — Test With Your Most Complex Call Scenarios
Do not go live before testing these four scenario categories. They reveal 90% of post-migration issues.
Test 1 — The double sub-question: Do you accept new patients, and if so, do you require a doctor referral? The agent must address both parts in a single fluent reply, without asking you to repeat.
Test 2 — Mid-call language switch: Start in English, ask a sub-question in French (Est-ce que vous acceptez la RAMQ?), then return to English. GPT-Realtime-2 handles this natively — but verify your system prompt does not block responses in the secondary language.
Test 3 — Conditional request: I would like an appointment Friday, but only if Dr. Martin is available — otherwise forget it. The agent must check specific availability before proposing a time, not automatically accept.
Test 4 — Interruption during a tool call: Ask something that triggers a tool query — calendar, database — then interrupt with Actually, Thursday would be better. The agent should cancel the first search and restart with the new criteria, without losing information already collected.
If all 4 tests pass, you are production-ready. If not, the adjustments are almost always in the system prompt, not the model choice itself.
What It Actually Costs (In Canadian Dollars, Per Month)
Here is the cost reality that most English-language articles skim over.
GPT-Realtime-2 is billed at $32 USD per million audio input tokens and $64 USD per million output tokens. At the current exchange rate, that is roughly $44 CAD and $88 CAD per million tokens respectively. That sounds steep until you look at what it means per call.
A typical 3-minute conversation with an AI voice agent consumes approximately 4,000 to 6,000 tokens (context plus responses). At that rate, one million tokens covers roughly 200 complete calls. For a Quebec SMB receiving 400 calls per month, you are looking at $90 to $130 CAD per month in model costs — on top of your ElevenLabs subscription.
The good news: the caching mechanism drops cached input tokens to $0.40 USD per million, about $0.55 CAD. For repetitive system prompts and constant FAQ context, aggressive caching can cut your costs by 40 to 60 percent. Our team at Agent IA Vocal configures this automatically for our clients.
Worth noting: GPT-Realtime-2 is not always the right choice. If your call volume is under 150 per month and your questions are straightforward, the previous model may be more cost-effective. The real value of GPT-Realtime-2 shows when complex calls exceed 20% of your total volume. And if your SMB serves both French and English-speaking callers, keep in mind that GPT-Realtime-Translate is a separate model built for live speech translation — the two can coexist in your architecture depending on your needs.
The 3 Most Common Mistakes After Migration
We see these come up consistently.
Mistake 1: Leaving reasoning effort at high by default. If nobody explicitly set the level, ElevenLabs may initialize at high depending on your dashboard version. Check manually. A 2-second pause before every response shows up immediately in customer feedback.
Mistake 2: Keeping a system prompt written for a simpler model. Rigid scripting instructions, explicit if-then blocks, and markdown formatting (headers, bullets) are leftovers from an era when the model needed heavy structural guidance. GPT-Realtime-2 prefers objectives and constraints over scripts — rewrite accordingly.
Mistake 3: Not testing behavior with slow tools. If your agent calls an external API — CRM, calendar — that responds in more than 3 seconds, the silence gap bothers callers. Configure verbal action announcements as in Step 4, and if possible, target tool response times under 1.5 seconds. Integration performance optimization is part of what the TECHMA team configures when setting up your agent.
FAQ — Common Questions About GPT-Realtime-2 in ElevenLabs
Is GPT-Realtime-2 available on all ElevenLabs plans? Not necessarily. As of this writing, access is available on Starter and higher plans, but rollout is gradual. Check the ElevenLabs changelog or contact their support to confirm your access.
My agent is already live. Can I change the model without a service interruption? Yes. Changing the LLM in ElevenLabs takes effect on subsequent calls, not ones in progress. You can migrate during a low-traffic window — evening or weekend — with no visible downtime for your customers.
Will GPT-Realtime-2 change my agent voice? No. The voice is generated by ElevenLabs TTS engine, which is independent from the LLM. Your selected voice will remain identical after migration.
My calls are mostly in English. Does GPT-Realtime-2 perform better in one language? No, performance is comparable in both French and English. GPT-Realtime-2 handles both languages natively, including mid-call language switches. For bilingual scenarios common in Quebec SMBs, this is actually one of its specific strengths.
Ready to Move to GPT-Realtime-2?
Migrating to GPT-Realtime-2 is not an urgent requirement if your current agent handles your everyday calls well. But if you find that 15 to 20 percent of your calls require a transfer to a human for questions that should resolve automatically — that is the signal the time has come.
At Agent IA Vocal, our TECHMA teams handle this migration end-to-end for our clients: auditing the current system prompt, selecting the right reasoning level based on your call volume and use cases, testing against your real-world scenarios, and deploying without interruption. Schedule a consultation to see whether GPT-Realtime-2 is the right next step for your SMB.