Voice-First Accessible Food Ordering Platform

Voice2Bite

An accessibility-first food ordering platform for visually impaired users, driven end-to-end by voice — Whisper STT, Qdrant RAG menu lookup, a resilient Node API gateway, and an async order pipeline with real-time audio announcements, all wired across three Dockerized services.

The problem

Food-ordering UIs assume you can see them — nested categories, image menus, micro-interactions. Voice2Bite inverts that: the entire flow is operable by voice, with earcons and text-to-speech guiding the user through every order state. Under the hood it's a real distributed system — a Node gateway, a Python AI service, and async workers — that had to stay available and fast even when the expensive AI/LLM path degrades.

Architecture & approach

1. Next.js/React + Redux frontend captures speech via the Web Audio API with Voice Activity Detection, plays earcons for state changes, and reads results back through the browser's speechSynthesis TTS.

2. A Node.js/Express API gateway orchestrates everything: Prisma over PostgreSQL for the relational core, Redis for hot state, plus the BullMQ pipeline and live Socket.IO connections.

3. A Python/Flask AI service runs OpenAI Whisper for transcription, queries a Qdrant vector DB for semantic menu retrieval ('a cold drink' → 'Iced Lemon Tea'), and parses intent via Gemini through OpenRouter.

4. Resilience is built in: pybreaker circuit breakers fail open around the AI/OpenRouter path so requests never hang, and an atomic Redis + Lua token-bucket rate limiter throttles expensive transcription/LLM calls, returning 429 on excess without race conditions.

5. Orders run asynchronously — placeOrder returns 202 Accepted with a trackingId, a BullMQ worker processes the transaction, and Socket.IO streams RECEIVED → PREPARING → READY updates that the frontend announces aloud via TTS.

6. Full-stack observability with OpenTelemetry (OTLP) distributed tracing across the gateway and AI backend, aggregated in Jaeger to profile STT, LLM, and routing latency; the whole stack boots from a single docker-compose command.

Stack

• Next.js
• Redux
• Node
• Express
• PostgreSQL
• Prisma
• Redis
• BullMQ
• Socket.IO
• Flask
• Whisper
• Qdrant
• OpenRouter
• OpenTelemetry
• Jaeger
• Docker