I have two servers; receiving server(1) runs on NodeJS, and the sender server(2) is on Python.
Server(1) code :
import {Storage} from "#google-cloud/storage";
this.post("/webhook",
async (req, res) => {
const storage = new Storage({/** google credential filename */ });
const clientBucket = storage.bucket(/** bucket name */);
// Create a reference to a file object
const fileName = `${moment().format("YYYYMMDDHHmmss")}.tar.gz`;
const file = clientBucket.file(fileName);
req.pipe(file.createWriteStream());
req.on("end", () => {
// have to update in db that file has been uploaded
});
})
Headers from the server(2)
Problem: When running both servers on HTTP file uploading works fine and I received whole file. But if any one server is on HTTPs it stops working and only gets a few chunks of data or not.
Related
I am new to angular and i want to display JSON data from python to angular with the help of node.js and I used child process to connect python and node.js but I dont know how to pass it to angular service
node.js file
const express = require('express')
const { spawn } = require('child_process')
const app = express()
const port = 8000
app.get('/', (req, res) => {
let dataToSend
let largeDataSet = []
// spawn new child process to call the python script
const python = spawn('python', ['test.py'])
// collect data from script
python.stdout.on('data', function (data) {
console.log('Pipe data from python script ...')
//dataToSend = data;
largeDataSet.push(data)
})
// in close event we are sure that stream is from child process is closed
python.on('close', (code) => {
console.log(`child process close all stdio with code ${code}`)
// send data to browser
res.send(largeDataSet.join(''))
})
})
app.listen(port, () => {
console.log(`App listening on port ${port}!`)
})
Technically you just have to send a Http GET request from your service.
I suggest that you should read and follow this offical http client guide to set it up correctly.
Here is a simple service snippet. This should be enough.
#Injectable({
providedIn: 'root',
})
export class MyService {
constructor(private http: HttpClient) {}
getData(): Observable<any> {
const url = '';
return this.http.get(url);
}
}
I am trying to fetch python output from Node.js script using postman but unable to get the required output. I am explaining my code below.
app.js:
router.post('/usecase-workflow', async (req, res) => {
try{
let responseUsecase = await usecaseWorkflow.fetchUsecaseWorkflow(req);
res.send(responseUsecase);
}catch(error) {
responseObj = {
status: 'error',
msg: 'Error occurred while downloading ubot file',
body: error
}
res.send(responseObj);
}
})
usecaseWorkflow.js:
const mongoose = require('mongoose');
const axios = require('axios');
const request = require('request');
class DefineUseCase {
fetchUsecaseWorkflow = async(req) => {
try{
const response = await axios.post('http://127.0.0.1:5005/usecase-workflow',req);
//console.log(response);
return response;
}catch(error) {
console.log(error);
}
}
}
module.exports = new DefineUseCase();
When I am doing REST API call from postman the above code is executing. I am giving the screen shot of postman below.
Here my need is i will upload one zip file and one node.js REST API will call. Inside the node script I am calling one python file to get the final output. But as per my code its not giving any result. If I am calling the Python file directly from postman Its giving some result. I am also giving python call postman screen shot below.
So here I need to fetch the same above output via node.js REST API.
My approach has been to write a Flask server in Python and forward the request to that.
Demo.py
from flask import Flask
app= Flask(__name__)
#app.route("/")
def index():
return "Hello World"
if __name__ == '__main__':
app.run(debug=True)
Alternately, I also tried directly invoking a python script and then collecting the result from stdout.
Trial.js
const express = require('express')
const app = express()
app.get('/', (req, res) => {
const { spawn } = require('child_process');
const pyProg = spawn('python', ['C:/Users/Selectigence/PycharmProjects/jobapplicationbot/Sample3.py']);
pyProg.stdout.on('data', function(data) {
console.log(data.toString());
res.write(data);
res.end('end');
});
})
app.listen(4000, () => console.log('Application listening on port 4000!'))
I had a similar problem recently. I was running a Node JS server but the client request had to be processed in Python. So I went for an architecture which went like this:
The Web process was the Node JS server and the Background service was written in Python.
To implement this architecture, I had to answer the following questions:
How to implement the Background service.
How to get the Background service to communicate with the Web process.
I used celery to implement my Background service. For example:
tasks.py
import os
from celery import Celery
REDIS_URL='redis://127.0.0.1:6379'
app = Celery('tasks', backend=REDIS_URL, broker='REDIS_URL')
#app.task
def doSomething(arguments):
# Do whatever you want with the arguments
# and return the result ...
return result
My Web process was defined as follows:
app.js
const celery = require("node-celery");
const express = require("express");
const app = express();
const REDIS_URL = "redis://127.0.0.1:6379";
app.get("/", (req, res, next) => {
const client = celery.createClient({
CELERY_BROKER_URL: REDIS_URL,
CELERY_RESULT_BACKEND: REDIS_URL
});
client.on("error", function(err) {
console.log(err);
});
client.on("connect", function() {
const { arguments } = req.body;
// Here send task to Python process and
// process the result.
client.call("tasks.doSomething", [arguments], result => {
// The result object contains the result
// of the doSomething function in the "result"
// field along with other information regarding
// whether the call was successful.
res.send({ result: result.result });
client.end();
});
});
});
app.listen(4000, () => console.log('Application listening on port 4000!'))
Now I needed a communication channel between these two processes. I used redis as the message broker.
In the command line, I ran:
$ redis-server &
$ npm start &
$ celery -A tasks worker --loglevel=INFO &
Now, whenever I send a GET request at the server through the browser, the server forwards the arguments in the request to the Python service. This service processes the arguments, and returns the result to the server. The server finally fulfills the request with the response which is sent back to the browser.
I have managed to setup my webcam to point to a specific location on the Firebase Database and broadcast a video using WebRTC.
I do this as follows in Javascript (and display in my HTML):
<video id="yourVideo" autoplay muted playsinline></video>
...
var database = firebase.database().ref('node_on_firebase');
var yourVideo = document.getElementById("yourVideo");
var friendsVideo = document.getElementById("friendsVideo");
var yourId = Math.floor(Math.random()*1000000000);
var servers = {'iceServers': [{'urls': 'stun:stun.services.mozilla.com'}, {'urls': 'stun:stun.l.google.com:19302'}, {'urls': 'turn:numb.viagenie.ca','credential': 'webrtc','username': 'websitebeaver#mail.com'}]};
var pc = new RTCPeerConnection(servers);
pc.onicecandidate = (event => event.candidate?sendMessage(yourId, JSON.stringify({'ice': event.candidate})):console.log("Sent All Ice") );
pc.onaddstream = (event => friendsVideo.srcObject = event.stream);
function sendMessage(senderId, data) {
var msg = database.push({ sender: senderId, message: data });
msg.remove();
}
function readMessage(data) {
// works
var msg = JSON.parse(data.val().message);
var sender = data.val().sender;
if (sender != yourId) {
if (msg.ice != undefined)
pc.addIceCandidate(new RTCIceCandidate(msg.ice));
else if (msg.sdp.type == "offer")
pc.setRemoteDescription(new RTCSessionDescription(msg.sdp))
.then(() => pc.createAnswer())
.then(answer => pc.setLocalDescription(answer))
.then(() => sendMessage(yourId, JSON.stringify({'sdp': pc.localDescription})));
else if (msg.sdp.type == "answer")
pc.setRemoteDescription(new RTCSessionDescription(msg.sdp));
}
};
database.on('child_added', readMessage);
function closeMyFace() {
yourVideo.srcObject.getTracks().forEach(track => track.stop());
}
function showMyFace() {
navigator.mediaDevices.getUserMedia({audio:false, video:true}).
then(function(stream){
pc.addStream(stream)
yourVideo.srcObject = stream
})
.catch(function(error){
console.log(error)
})
}
function showFriendsFace() {
pc.createOffer()
.then(offer => pc.setLocalDescription(offer) )
.then(() => sendMessage(yourId, JSON.stringify({'sdp': pc.localDescription})) );
}
However, how do I download/stream this video and process the video in chunks, ideally in a Python script?
If you intend to download/process the video while it is streaming, then your (python) client will need to create its own RTCPeerConnection so that it can also receive the video stream. I believe that would not be trivial in python, though probably easier on other platforms. More info: WebRTC Python implementation
If your use case allows you to process the video after the recording is complete (or at least, your use case is okay with significant latency), then could have the javascript client upload the data as it received or later in batch (from friendsVideo stream in the example above), possibly in chunks, to a location where your custom (python) client could then download and process.
Although not related to RTCPeerConnection, you can search here on SO for other users that have used firebase for streaming video (with mixed results). Again though, that is somewhat different from what you are trying to do with RTCPeerConnection. Example: Firebase Storage Video Streaming
So I have the following node.js express server running:
const express = require('express')
const app = express()
const PythonShell = require('python-shell');
app.get('/', (req, res) => {
res.send('Please append a search');
});
app.get('/:card', (req, res) => {
card = [req.params.card]
var pyshell = new PythonShell('search_card.py', {args: card});
pyshell.on('message', (message) => {
res.send(message);
});
pyshell.end((err) => {
if (err) throw err;
});
});
app.listen(9000, () => console.log('App listening on port 9000!'));
When I use the app using the local IP of the server (which is hosted within the intranet), everything works fine.
But if I use the external IP, the root of the app works ( / ) so clearly the connection is possible and the port is open, etc, but if I use the search function ( /search_query ), it takes a few seconds (as if the python script is actually running) and then chrome says ERR_CONNECTION_REFUSED (This site can't be reached).
Can anyone explain this behavior?
EDIT:
I put a console.log inside the pyshell.on('message') and the data is being received from the python program. Its as soon as it writes ends that it fails