Calling python script in node js => ERR_HTTP_HEADERS_SENT

Question

i node js project in which i call python script. The problem is that it displays only part of the output and the server drops with error ERR_HTTP_HEADERS_SENT.

hello.py

import sys
for i in range(3000):
 print(i)
sys.stdout.flush()

index.js

const express = require('express')
const app = express()
const bodyParser = require('body-parser')
app.set('view engine', 'ejs')
app.use(bodyParser.urlencoded({ limit: '10mb', extended: false }))
app.get("/", (req, res) => {
 const spawn = require("child_process").spawn
 const pythonProcess = spawn('python', ["./hello.py"])
 pythonProcess.stdout.on('data', (data) => {
 results = data.toString().split("\r\n")
 res.render("../index.ejs", {results})
 })
})
app.listen(process.env.PORT || 3000)

index.ejs

<h2>Results:</h2>
<% results.forEach(result => { %>
 <div><%= result %> </div>
<% }) %>

The result is page with numbers from 0 to 1550, each on separate line, but it should be 0 to 3000, what's wrong?

Answer 1

This happens due to the fact that data is actually a stream, its output is read as chunks of data piece by piece, which means the listener on('data',.. is being triggered more than once; every time a chunk is received.

To get the entire result we have to wait until all pieces are received, here is how:

router.get('/', (req, res) => {
 const pythonProcess = spawn('python', ['./hello.py']);
 let result = [];
 // this listener will keep triggering until all chunks are received
 pythonProcess.stdout.on('data', data => {
 result = result.concat(data.toString().split('\r\n'));
 });
 // on.exit() will indicate that process has finished
 pythonProcess.on('exit', code => {
 console.log(`process has finished and exited with code: ${code}`);
 res.render('../index.ejs', { result });
 });
});

Hope I've helped