Develop Reference

ExpressJS closes connection on bigger data requests

I am trying to create an ExpressJS API that interpolates some data. I use Python requests to test my API. Everything works just fine if I send smaller datasets. However, when I send a bigger dataset Python (on Windows) returns a requests.exceptions.ChunkedEncodingError: "Connection broken: ConnectionResetError 10054 Exception.
The dataset size I am trying to send is 732808 bytes. I tried increasing the timeout limit and the datalimit, that did not help me:
app.use('/tests', test_router)
app.use(express.json({limit: '10mb'}))
app.listen(5000, () => console.log('Server running')).setTimeout(120000)
I tried to debug and found that none of my middleware gets invoked at all (router is "test_router" in code above).
router.get('/test_interpolation', (req, res) => {
console.log('Hello') //Never gets called
res = test_controller.do_something(req, res)
})
Why does ExpressJS not accept the request? Thank you for helping!

My first mistake was using router.get(). Apparently ExpressJS only accepts bigger datasets through a post request. Also, my middleware was not configured right either. This is what it looks like now:
app.use(express.json({ limit: '100mb' }));
app.use(express.urlencoded({ limit: '100mb' }));
app.use(bodyParser.json({ limit: '100mb' }));
app.use(
bodyParser.urlencoded({
limit: '100mb',
extended: true,
}),
);
I used npm body-parser for "bodyParser". It does give me a message that urlencode is deprecated, but this does not bother me right now.

useing flask-socketIO as a Notification Handler (server-client ) connection

I'm new to using socketIO and I'm trying to build a notification system using it.
It says on flask-socketIO Official website link That it's possible and ready for it. but for some reason I kept getting error
AttributeError: 'Request' object has no attribute 'namespace'
pythonApp.py
#app.route("/ajaxHandler", methods=['post'])
def ajaxHandler(userSessionId):
if request.method == 'POST':
data = request.get_json()
print(f'\n\n >> data: {data}\n\n')
notify_User({'test':'test1'},userSessionId) # >>>> Notice 1
return jsonify({'Ajax': 'completed'})
#socketio.on('notify_User' , namespace='/notifications/')
def notify_User(data,sid):
msg= {"hi": hello}
emit('notify_User',msg, room=sid) # >>> Notice 2
Notice 1: I could pull userSessionId from database but for the sake of this example i'm pullingit from the user
Notice 2 : the error is generated from this line
socket_Io.js
document.addEventListener('DOMContentLoaded', () => {
// Connect to websocket
var socket = io.connect(location.protocol + '//' + document.domain + ':' + location.port);
var socket = io();
socket.on("notify_User",(data) => {
console.log(`i got this : ${data}`)
});
I kept searching the error everywhere, but I couldn't find any answer that could help in my situation.

The notify_User() is defined as a Socket.IO event handler. This function will automatically run when the client emits the notify_User event.
You are calling this function directly in your ajaxHandler() view function. This is not the correct usage, a Socket.IO event handler is not supposed to be called from the server, the client triggers this function when it emits the Socket.IO event.
If you want to emit an event to the client from the ajaxHandler() route, then use the emit() function to do so. Because this is an HTTP route and not a Socket.IO event, you will need to provide some information that is not available outside of the Socket.IO connection. For that reason the namespace and sid arguments are required when using emit() outside of a Socket.IO handler.

Alamofire 5 (Beta 6): Parameters of PUT Request do not arrive in Flask-Restful

UPDATE
For me the Problem got fixed as soon as I was putting "encoding: URLEncoding(destination: .queryString)" in my request. Maybe this helps somebody else. link
I struggled the whole day to find the problem in my Alamofire PUT Request or the Flask Restful API. Request like GET, DELETE and POST are working fine with Alamofire, except the PUT Request.
When I'm using PUT Requests in combination with Postman and Flask-Restful everything is also working fine. But as soon as I'm trying to achieve the same Result with Alamofire, I'm not getting any parameters in Flask. I tried to illustrate this in the code examples.
So in short my example illustrates the following:
DELETE Request(Same with GET and POST)
Postman: success
Alamofire: success
PUT Request
Postman: success
Alamofire: failure (parameter dictionary empty in Flask-Restful)
Here is my Python Code [API Server]:
from flask import Flask, request, jsonify
from flask_restful import Resource, Api, reqparse
app = Flask(__name__)
api = Api(app)
class Stackoverflow(Resource):
def delete(self):
print(request.args)
if request.args.get('test-key') is None:
return jsonify({"message": "failure"})
else:
return jsonify({"message": "success"})
def put(self):
print(request.args)
if request.args.get('test-key') is None:
return jsonify({"message": "failure"})
else:
return jsonify({"message": "success"})
api.add_resource(Stackoverflow, '/stackoverflow')
if __name__ == '__main__':
app.run(debug=True, host='0.0.0.0')
If I'm using Postman, I get this result (like expected):
Result in Postman
But now I'm trying to do the same with Alamofire in Swift. Same Server, nothing changed.
SWIFT demo Code [IOS APP]:
import UIKit
import Alamofire
import SwiftyJSON
class ViewController: UIViewController {
override func viewDidLoad() {
super.viewDidLoad()
// Do any additional setup after loading the view
simplePUTRequest()
simpleDELETERequest()
}
func simplePUTRequest(){
AF.request("http://localhost:5000/stackoverflow", method: .put, parameters: ["test-key":"testvalue"])
.validate(statusCode: 200..<300)
.responseJSON { response in
if let data = response.data {
print("Result PUT Request:")
print(String(data: data, encoding: .utf8)!)
//print(utf8Text)
}else{
}
}
}
func simpleDELETERequest(){
AF.request("http://localhost:5000/stackoverflow", method: .delete, parameters: ["test-key":"testvalue"])
.validate(statusCode: 200..<300)
.responseJSON { response in
if let data = response.data {
print("Result DELETE Request:")
print(String(data: data, encoding: .utf8)!)
//print(utf8Text)
}else{
}
}
}
Xcode Console:
Result PUT Request:
{
"message": "failure"
}
Result DELETE Request:
{
"message": "success"
}
python Console (both Alamofire Requests):
ImmutableMultiDict([])
127.0.0.1 - - [15/Jun/2019 21:17:31] "PUT /stackoverflow HTTP/1.1" 200 -
ImmutableMultiDict([('test-key', 'testvalue')])
127.0.0.1 - - [15/Jun/2019 21:17:31] "DELETE /stackoverflow?test-key=testvalue HTTP/1.1" 200 -
As you can see, I'm getting the success message only while using the DELETE method.
Till now I tried using different encodings like URLEncoding.httpbody and URLEncoding.default, but nothing really helped.
For me it seems like it's a Alamofire/Swift Problem, because in Postman the same request method is working fine.
I would really appreciate your help, because I'm stuck and don't know anything further to do. I hope I didn't misunderstood something essential.
Thank you in advance!

I am currently using the same version AlamoFire, and when I use the PUT method, I use it as follows:
let request = AF.request(url, method: .put, parameters: ["uid": uid],
encoding: JSONEncoding.default, headers: headers)
request.responseJSON(completionHandler: { response in
guard response.error == nil else {
//Handle error
}
if let json = response.value as? [String: Any]
// Handle result.
}
The only difference to your post is that I used the encoding option. You can try to put the option and see what happens.

It looks like your server is expecting your PUT parameters to be URL form encoded into the URL. You may be hitting the version of the request method that uses JSON encoding by default, so adding encoder: URLEncodedFormParameterEncoder.default at the end of your request call should fix that. A future release will make that the default, as it's safe across all request types.
If that's not the issue, I suggest you investigate more closely to see what the differences between the requests may be. Since you control the server you should have easy access to the traffic.

Long Polling in Python with Flask

I'm trying to do long polling with JQuery and Python under the Flask Framework.
Having done long polling before in PHP, I've tried to go about it in the same way:
A script/function that has a while(true) loop, checking for changes periodically eg.every 0,5 seconds in the database, and returns some data when a change occurs.
So in my ini.py I've created an app.route to /poll for JQuery to call. JQuery gives it some information about the client's current state, and the poll() function compares this with what's currently in the database. The loop is ended and returns information when a change is observed.
Here's the python code:
#app.route('/poll')
def poll():
client_state = request.args.get("state")
#remove html encoding + whitesapce from client state
html_parser = HTMLParser.HTMLParser()
client_state = html_parser.unescape(client_state)
client_state = "".join(client_state.split())
#poll the database
while True:
time.sleep(0.5)
data = get_data()
json_state = to_json(data)
json_state = "".join(data) #remove whitespace
if json_state != client_state:
return "CHANGE"
The problem is that, when the code above starts polling, the server appears to be overloaded and other Ajax calls, and other requests like loading a "loading" image to the html using JQuery are unresponsive and timeout.
For completion's sake I've included the JQuery here:
function poll() {
queryString = "state="+JSON.stringify(currentState);
$.ajax({
url:"/poll",
data: queryString,
timeout: 60000,
success: function(data) {
console.log(data);
if(currentState == null) {
currentState = JSON.parse(data);
}
else {
console.log("A change has occurred");
}
poll();
},
error: function(jqXHR, textStatus, errorThrown) {
console.log(jqXHR.status + "," + textStatus + ", " + errorThrown);
poll();
}
});
}
Does this need to multi-threaded or something? Or does anyone have any idea why I'm experiencing this behavior?
Thanks in advance!! :)

Just as the link #Robᵩ mentioned, you flask app is just overload. That's because a flask app is in single threading mode by default when running with app.run(), so it can only serve one request per time.
You can start multi threading with:
if __name__ == '__main__':
app.run(threaded=True)
Or using a WSGI server like gunicorn or uwsgi to serve flask with multi processing:
gunicorn -w 4 myapp:app
Hopes you're enjoying with Python and Flask!

How to limit Autobahn python subscriptions on a per session basis

I am using autobahnpython with twisted (wamp) on server side and autobahnjs in browser. Is there a straight-forward way to allow/restrict subscriptions on a per session basis? For example, a client should not be able to subscribe to topics relavant to other users.
While I am NOT using crossbar.io, I tried using the Python code shown in the 'Example' section at the end of this page http://crossbar.io/docs/Authorization/ where a RPC call is first used to give authorization to a client. Of course, I am using my own authorization logic. Once this authorization is successful, I'd like to give the client privileges to subscribe to topics related only to this client, like 'com.example.user_id'. My issue is that even if auth passes, however, I have not found a way to limit subscription requests in the ApplicationSession class which is where the authorization takes place. How can I prevent a client who authorizes with user_id=user_a from subscribing to 'com.example.user_b'?

You can authorize by creating your own router. To do that, subclass Router() and override (at a minumum) the authorize() method:
def authorize(self, session, uri, action):
return True
This method is pretty simple, if you return a True then the session is authorized to do whatever it is attempting. You could make a rule that all subscriptions must start with 'com.example.USER_ID', so, your python code would split the uri, take the third field, and compare it to the current session id, returning True if they match, false otherwise. This is where things get a little weird though. I have code that does a similar thing, here is my authorize() method:
#inlineCallbacks
def authorize(self, session, uri, action):
authid = session._authid
if authid is None:
authid = 1
log.msg("AuthorizeRouter.authorize: {} {} {} {} {}".format(authid,
session._session_id, uri, IRouter.ACTION_TO_STRING[action], action))
if authid != 1:
rv = yield self.check_permission(authid, uri, IRouter.ACTION_TO_STRING[action])
else:
rv = yield True
log.msg("AuthorizeRouter.authorize: rv is {}".format(rv))
if not uri.startswith(self.svar['topic_base']):
self.sessiondb.activity(session._session_id, uri, IRouter.ACTION_TO_STRING[action], rv)
returnValue(rv)
return
Note that I dive into the session to get the _authid, which is bad karma (I think) because I should not be looking at these private variables. I don't know where else to get it, though.
Also, of note, this goes hand in hand with Authentication. In my implementation, the _authid is the authenticated user id, which is similar to a unix user id (positive unique integer). I am pretty sure this can be anything, like a string, so you should be ok with your 'user_b' as the _auth_id if you wish.
-g

I found a relatively simple solution using a Node guest. Here's the code:
// crossbar setup
var autobahn = require('autobahn');
var connection = new autobahn.Connection({
url: 'ws://127.0.0.1:8080/ws',
realm: 'realm1'
}
);
// Websocket to Scratch setup
// pull in the required node packages and assign variables for the entities
var WebSocketServer = require('websocket').server;
var http = require('http');
var ipPort = 1234; // ip port number for Scratch to use
// this connection is a crossbar connection
connection.onopen = function (session) {
// create an http server that will be used to contain a WebSocket server
var server = http.createServer(function (request, response) {
// We are not processing any HTTP, so this is an empty function. 'server' is a wrapper for the
// WebSocketServer we are going to create below.
});
// Create an IP listener using the http server
server.listen(ipPort, function () {
console.log('Webserver created and listening on port ' + ipPort);
});
// create the WebSocket Server and associate it with the httpServer
var wsServer = new WebSocketServer({
httpServer: server
});
// WebSocket server has been activated and a 'request' message has been received from client websocket
wsServer.on('request', function (request) {
// accept a connection request from Xi4S
//myconnection is the WS connection to Scratch
myconnection = request.accept(null, request.origin); // The server is now 'online'
// Process Xi4S messages
myconnection.on('message', function (message) {
console.log('message received: ' + message.utf8Data);
session.publish('com.serial.data', [message.utf8Data]);
// Process each message type received
myconnection.on('close', function (myconnection) {
console.log('Client closed connection');
boardReset();
});
});
});
};
connection.open();