Real-time voice AI assistant with 56 tools
A browser-based voice assistant with a live transcript, two voice engines behind one interface and metered usage billing, covered by 1,500+ automated tests.
- voice-callable tools
- 56
- automated tests
- 1,500+
- voice engines, one interface
- 2
- unbounded spend paths
- 0
The challenge
- Client
- Options analytics company (name withheld under NDA)
- Industry
- FinTech · Voice AI
Voice assistants feel broken when they pause to think. The product needed fast spoken answers that pull live market data, with costs that could never run away.
What we built
Gemini Live native audio and ElevenLabs engines behind one interface, so the WebSocket layer is provider-independent.
56 tools exposed to the model, with a turn's tool calls run concurrently via asyncio to keep spoken replies fast.
Usage metering with weekly and monthly spend caps and prepaid credits.
Docker Compose deployment behind Nginx with rate limiting, JWT authentication and MFA, covered by more than 1,500 automated tests.
How the system fits together
The path a request or message takes through the production system, end to end.
- Microphone
- Django Channels
- Gemini Live / ElevenLabs
- 56 tools via asyncio
- Metering & spend caps
- Spoken reply