I want to display the real-time speech to text data in the browser. By real-time what I mean is, "while I am speaking I am getting the text output simultaneously". I have implemented the speech-to-text part in Python using the Google cloud service API. Then I used "child process" to run my python program in the node.js environment. Till now everything is fine. Next, I want to display the real-time text in the browser. In another word, I want to send the real-time text output from the python (which is now running in node.js using the child process) to the web browser. I was trying to do that with socket.io. Here is my server side (node.js) code where socket.io is also applied:
const express = require('express');
//const router = express.Router();
const {spawn} = require('child_process');
const path = require('path');
const app = express();
const http = require('http');
const server = http.createServer(app);
//const server = http.createServer(app);
const { Server } = require("socket.io");
const io = new Server(server);
function runScript(){
return spawn('python3', [
"-u",
path.join(__dirname, 'script.py')
]);
}
const subprocess = runScript()
// print output of the script
app.get('/', (req,res) => {
res.sendFile(__dirname + '/index.html');
});
io.on('connection', (socket) => {
subprocess.stdout.on('data', (data) => {
//console.log(`data:${data}`);
socket.on('message', (data) => {
socket.broadcast.emit('message', data);
});
});
});
server.listen(3000, () => {
console.log('listening on *:3000');
});
Above, I am first using the child process to call the python program in node.js and then I am using socket.broadcast.emit to send the text output of the python program to my client side. The client-side code looks like this:
<!DOCTYPE html>
<html>
<head>
<script src="/socket.io/socket.io.js"></script>
<script>
var socket = io();
var messages = document.getElementById('messages');
//const EventEmitter = require('events');
//const emitter = new EventEmitter()
//emitter.setMaxListeners(50)
socket.on('messages', function(data) {
document.querySelector("#style1".innerHTML = `<p>${data1}</p>`
});
</script>
</head>
<body id="messages">
<h1> This is crazy </h1>
<div id="style1">
</div>
</body>
</html>
Above, I want to display the real-time text output from the python program inside the <p> tag. The problem is, I am not able to get anything in the web browser. My objective is, I want to display whatever I am speaking as text in the web browser in real-time.
I don't know much about socket.io. In fact, this is the first time I am using this technology.
Your Node.js server will act as the socket server. As your code shows, it listens on a port for a socket connection, and on connection, creates a socket, which you then send messages too. From a simple cursory review, the server code looks okay.
On your webpage, you are creating the socket, and listening for messages.
However the socket running on the webpage hasn't yet connected to the server, which is why nothing is working yet.
Assuming you're doing this on localhost, just add the socket server address to it's constructor, and then listen for connect.
const socket = io('ws://localhost:3000');
socket.on('connect', () => {
// do any authentication or handshaking here.
console.log('socket connected');
});
More advanced implementations should gracefully handle closing sockets.
Per the following comment:
I added the lines you suggested above. Even now nothing is visible on the webpage but I am getting this warning: (node:14016) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 message listeners added. Use emitter.setMaxListeners() to increase limit
Looking more closely at your server code, I believe this is the root issue
subprocess.stdout.on('data', (data) => {
//console.log(`data:${data}`);
socket.on('message', (data) => {
socket.broadcast.emit('message', data);
});
});
});
Each time you receive data from subprocess.stdout, you are adding a new onmessage event handler to your socket, so after a period of time, you have added too many event handlers.
Re-write your logic so that you only add socket.on('message') once (usually after your create the socket).
It is also worth noting that in the above code, data from stdout is not being used, as that data variable is being redefined in a lower scope by your onmessage function. Since data is being redefined, the output of your Python program is being ignored.
I think this is what you want:
//echo any message you receive from the socket back to the socket
socket.on('message', (data) => {
socket.broadcast.emit('message', data);
});
//send data from std out to the socket.
subprocess.stdout.on('data', (data) => {
//console.log(`data:${data}`);
socket.broadcast.emit('message', data);
});