I am trying to develop an application where the users can call and interact with the OpenAI agent. I am trying to stream audio response which is getting generated by Google TTS. But I am not able to figure out why the audio is not getting played in the response. The users audio is captured and even the OpenAI answers the users input but while sending there is no audio at the receivers end. Below is my code. Any help is appreciated. Thanks in advance.
const express = require('express');
const { OpenAI } = require('openai');
const textToSpeech = require('@google-cloud/text-to-speech');
const speech = require('@google-cloud/speech');
const WebSocket = require('ws');
const twilio = require('twilio');
require('dotenv').config();
const app = express();
const port = process.env.PORT || 3000;
// Init clients
const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });
const ttsClient = new textToSpeech.TextToSpeechClient();
const sttClient = new speech.SpeechClient();
const twilioClient = twilio(process.env.TWILIO_ACCOUNT_SID, process.env.TWILIO_AUTH_TOKEN);
// === Twilio Initial Voice Webhook ===
app.use(express.urlencoded({ extended: false }));
app.post('/voice', (req, res) => {
const twiml = new twilio.twiml.VoiceResponse();
// Replace with your Ngrok WebSocket URL
const wsUrl = `wss://a567-103-205-124-130.ngrok-free.app/media`;
console.log('Connecting Media Stream to:', wsUrl);
const connect = twiml.connect();
connect.stream({ url: wsUrl });
res.type('text/xml');
res.send(twiml.toString());
console.log('