How to process requests in Python through an Express server - python

My approach has been to write a Flask server in Python and forward the request to that.
Demo.py
from flask import Flask
app= Flask(__name__)
#app.route("/")
def index():
return "Hello World"
if __name__ == '__main__':
app.run(debug=True)
Alternately, I also tried directly invoking a python script and then collecting the result from stdout.
Trial.js
const express = require('express')
const app = express()
app.get('/', (req, res) => {
const { spawn } = require('child_process');
const pyProg = spawn('python', ['C:/Users/Selectigence/PycharmProjects/jobapplicationbot/Sample3.py']);
pyProg.stdout.on('data', function(data) {
console.log(data.toString());
res.write(data);
res.end('end');
});
})
app.listen(4000, () => console.log('Application listening on port 4000!'))

I had a similar problem recently. I was running a Node JS server but the client request had to be processed in Python. So I went for an architecture which went like this:
The Web process was the Node JS server and the Background service was written in Python.
To implement this architecture, I had to answer the following questions:
How to implement the Background service.
How to get the Background service to communicate with the Web process.
I used celery to implement my Background service. For example:
tasks.py
import os
from celery import Celery
REDIS_URL='redis://127.0.0.1:6379'
app = Celery('tasks', backend=REDIS_URL, broker='REDIS_URL')
#app.task
def doSomething(arguments):
# Do whatever you want with the arguments
# and return the result ...
return result
My Web process was defined as follows:
app.js
const celery = require("node-celery");
const express = require("express");
const app = express();
const REDIS_URL = "redis://127.0.0.1:6379";
app.get("/", (req, res, next) => {
const client = celery.createClient({
CELERY_BROKER_URL: REDIS_URL,
CELERY_RESULT_BACKEND: REDIS_URL
});
client.on("error", function(err) {
console.log(err);
});
client.on("connect", function() {
const { arguments } = req.body;
// Here send task to Python process and
// process the result.
client.call("tasks.doSomething", [arguments], result => {
// The result object contains the result
// of the doSomething function in the "result"
// field along with other information regarding
// whether the call was successful.
res.send({ result: result.result });
client.end();
});
});
});
app.listen(4000, () => console.log('Application listening on port 4000!'))
Now I needed a communication channel between these two processes. I used redis as the message broker.
In the command line, I ran:
$ redis-server &
$ npm start &
$ celery -A tasks worker --loglevel=INFO &
Now, whenever I send a GET request at the server through the browser, the server forwards the arguments in the request to the Python service. This service processes the arguments, and returns the result to the server. The server finally fulfills the request with the response which is sent back to the browser.

Related

How to fetch data analyzed in python to node.js and pass it to angular?

I am new to angular and i want to display JSON data from python to angular with the help of node.js and I used child process to connect python and node.js but I dont know how to pass it to angular service
node.js file
const express = require('express')
const { spawn } = require('child_process')
const app = express()
const port = 8000
app.get('/', (req, res) => {
let dataToSend
let largeDataSet = []
// spawn new child process to call the python script
const python = spawn('python', ['test.py'])
// collect data from script
python.stdout.on('data', function (data) {
console.log('Pipe data from python script ...')
//dataToSend = data;
largeDataSet.push(data)
})
// in close event we are sure that stream is from child process is closed
python.on('close', (code) => {
console.log(`child process close all stdio with code ${code}`)
// send data to browser
res.send(largeDataSet.join(''))
})
})
app.listen(port, () => {
console.log(`App listening on port ${port}!`)
})
Technically you just have to send a Http GET request from your service.
I suggest that you should read and follow this offical http client guide to set it up correctly.
Here is a simple service snippet. This should be enough.
#Injectable({
providedIn: 'root',
})
export class MyService {
constructor(private http: HttpClient) {}
getData(): Observable<any> {
const url = '';
return this.http.get(url);
}
}

Connection is not getting established from react socket-io client to flask-socket-io server for real time update

Just for learning purpose, I am trying to establish a connection between react and flask with SOCKET.io to get real time updates.
But, sadly I am not able to establish a connection between client socket.io to sever socket.io till now.
I followed several blogs and explored GitHub issues, but none of them had helped me.
Codes
1. React codes
import React, { useState, useEffect } from "react";
import socketIOClient from "socket.io-client";
import io from "socket.io-client/dist/socket.io";
import "./App.css";
const ENDPOINT = "http://127.0.0.1:5000";
function App() {
const [data, setData] = useState("");
const setRecords = (response) => {
console.log("response", response);
setData(response.records);
};
useEffect(() => {
// const socket = socketIOClient(ENDPOINT); // >>>>> Not Working
const socket = io.connect(ENDPOINT, { rejectUnauthorized: false }); //{ transports:["websocket"]}
console.log("connected", socket);
socket.on("success", (data) => console.log(data));
// socket.on("FetchRecords");
// socket.emit("FetchRecords");
// socket.on("SendingRecords", setRecords);
// socket.on("FromAPI", data => {
// setResponse(data);
// });
}, []);
return (
<div className="App">WEBSOCKET with FLASK and REACT...{data.length}</div>
);
}
export default App;
Now if I see the logs in the console, it is being disconnected.
2. Backend Code
from flask import Flask, jsonify
from mongoConnect import todo
from flask_socketio import SocketIO, send, emit
app = Flask(__name__)
app.config['SECRET_KEY'] = 'secret!'
socketio = SocketIO(app, cors_allowed_origins='*', engineio_logger=True)
#socketio.on('connect')
def test_connect():
print('CONNECT EVENT happened...')
emit('success', {'data': 'Connected'})
#socketio.on('FetchRecords')
def fetch_all_records():
records = todo.find({})
print('fetch_all_records', len(records))
response = jsonify({'code': 200, 'records': records})
print(response)
emit('sendingRecords', response)
if __name__ == '__main__':
socketio.run(app, port=5000)
Logs of backend (using geevent-websocket)
Can anyone tell me where am I going wrong? Weirdly none of them had the problem like me...
Here are the blogs and issues I checked...
1. React Native: unable to establish the socket connection
2. Simple Chat-App with React & Flask
3. Socket.IO, React and Node.js: Going Real-Time
I had the same problem too.
Try uninstall socket.io-client and install version 2.3.1 npm i socket.io-client#2.3.1
The old version works for me.
Another thing is to make sure there isn't another server running on the same port you're trying to connect to.
Simple mistake but it could happen

Is there a way to increase the time out on a callback function in nodejs?

I am running a simple nodejs web application that executes a Python 2.7 script using the module python-shell. However, since this script takes too long to execute, around 3 minutes, the function fails with an ERR_EMPTY_RESPONSE error.
The function executes at the end, since Python still runs it in the background but the web application crashes.
generatePPage: (req, res) => {
const ps = require('python-shell');
let nombre = req.params.id;
var PythonShell = require('python-shell');
var options = {
pythonPath: '/usr/bin/python2.7',
args: [nombre],
};
var users = ps.PythonShell.run('./generateDetail.py', options, function (err, results) {
if (err){throw err;}
console.log('The results: %j', results);
res.render('success.ejs', {
title: 'Edit Project'
,project: ''
,message: 'Success'
});
});
},
How could I force it to wait longer?
Yes, you have to use setTimeout.
Check this link for more information: http://www.java2s.com/Tutorials/Javascript/Node.js_Tutorial/0270__Node.js_setTimeout_setInterval.htm
And also, check out this link: https://nodejs.org/ar/docs/guides/timers-in-node/
Solved it. The way to do it is to assign the return value of calling the listen function to a const and then assign a new timeout value in milliseconds.
const server = app.listen(port, () => {
console.log(`Server running on port: ${port}`);
});
server.timeout = 300000;

Executing Python from NodeJS (hosted on an external domain)

So I have the following node.js express server running:
const express = require('express')
const app = express()
const PythonShell = require('python-shell');
app.get('/', (req, res) => {
res.send('Please append a search');
});
app.get('/:card', (req, res) => {
card = [req.params.card]
var pyshell = new PythonShell('search_card.py', {args: card});
pyshell.on('message', (message) => {
res.send(message);
});
pyshell.end((err) => {
if (err) throw err;
});
});
app.listen(9000, () => console.log('App listening on port 9000!'));
When I use the app using the local IP of the server (which is hosted within the intranet), everything works fine.
But if I use the external IP, the root of the app works ( / ) so clearly the connection is possible and the port is open, etc, but if I use the search function ( /search_query ), it takes a few seconds (as if the python script is actually running) and then chrome says ERR_CONNECTION_REFUSED (This site can't be reached).
Can anyone explain this behavior?
EDIT:
I put a console.log inside the pyshell.on('message') and the data is being received from the python program. Its as soon as it writes ends that it fails

Emitting messages from python-shell to a browser in a Node Express App

I have an express app and I would like to be able to trigger python scripts via routes and emit the log into the browser.
I have created a route which triggers the python scripts correctly and outputs the log into the node console using python-shell module. How can I push this log to the browser in real time.
I have a button on an index.html page which triggers an Ajax post request to a /python route below. I have tried to implement this with socket.io but haven't managed to get my head around it. Is socket.io the way forward, please point me in the right direction or can someone recommend an alternative?
My file layout is:
server.js
var express = require('express');
var path = require('path');
var app = express();
var port = process.env.PORT || 3000;
var http = require('http');
var server = http.createServer(app);
var io = require('socket.io').listen(server);
/* A bit inbetween */
server.listen(port)
app/routes.js
var PythonShell = require('python-shell');
var config = require('../config/config');
module.exports = function(app) {
app.get('/', function(req, res) {
res.render('pages/index.ejs', {
pageTitle: 'Index'
}); // load the index.ejs file
});
app.post('/test', function(req, res) {
var options = {
mode: 'text',
pythonPath: config.pythonPath,
pythonOptions: ['-u'],
scriptPath: config.pythonScriptsDirectory
};
var pyshell = new PythonShell('printRange.py', options);
pyshell.on('message', function (message) {
console.log(message);
});
res.sendStatus(200)
});
}
Thanks in advance
socket.io is a good solution. This will take care of passing the messages from the server to the client. You just post the message to on the server side, and have a callback react to it on the client side
On the server side you'll have something like this:
var app = require('express')();
var server = require('http').createServer(app);
var io = require('socket.io')(server);
io.on('connection', function(){
console.log('connected');
});
server.listen(3000);
...
var pyshell = new PythonShell('printRange.py', options);
pyshell.on('message', function (message) {
if (connected)
io.emit('logentry',message);
});
On the client side, you'll have something like this:
<script src='/socket.io/socket.io.js'></script>
<script>
var socket = io();
socket.on('logentry',function(log_entry){
add_to_html(log_entry); // You need to implement this function.
});
</script>

Categories

Resources