I am trying to create an ExpressJS API that interpolates some data. I use Python requests to test my API. Everything works just fine if I send smaller datasets. However, when I send a bigger dataset Python (on Windows) returns a requests.exceptions.ChunkedEncodingError: "Connection broken: ConnectionResetError 10054 Exception.
The dataset size I am trying to send is 732808 bytes. I tried increasing the timeout limit and the datalimit, that did not help me:
app.use('/tests', test_router)
app.use(express.json({limit: '10mb'}))
app.listen(5000, () => console.log('Server running')).setTimeout(120000)
I tried to debug and found that none of my middleware gets invoked at all (router is "test_router" in code above).
router.get('/test_interpolation', (req, res) => {
console.log('Hello') //Never gets called
res = test_controller.do_something(req, res)
})
Why does ExpressJS not accept the request? Thank you for helping!
My first mistake was using router.get(). Apparently ExpressJS only accepts bigger datasets through a post request. Also, my middleware was not configured right either. This is what it looks like now:
app.use(express.json({ limit: '100mb' }));
app.use(express.urlencoded({ limit: '100mb' }));
app.use(bodyParser.json({ limit: '100mb' }));
app.use(
bodyParser.urlencoded({
limit: '100mb',
extended: true,
}),
);
I used npm body-parser for "bodyParser". It does give me a message that urlencode is deprecated, but this does not bother me right now.
Related
I have a plain simple Python function which should dead-letter a message if it does not match few constraint. Actually I'm raising an exception and everything works fine (I mean the message is being dead-lettered), but I would like to understand if there is a "clean" way to dead-letter the message without raising an exception.
async def function_handler(message: func.ServiceBusMessage, starter: str):
for msg in [message]:
client = df.DurableOrchestrationClient(starter)
message_body = msg.get_body().decode("utf-8")
msg = json.loads(message_body)
if 'valid' in msg:
instance_id = await client.start_new('orchestrator', None, json.dumps(message_body))
else:
raise Exception(f'not found valid {msg["id"]}')
This is part of host.json, this should indicate I'm working with version 2.0 of Azure Functions
"extensionBundle": {
"id": "Microsoft.Azure.Functions.ExtensionBundle",
"version": "[2.*, 3.0.0)"
},
Suggestions are welcome
At time of writing, in Python it is not possible interactively send a message in dead-letter.
I found out that autocomplete=false is only supported for C#.
This basically means that the only way to dead letter a message is raise an exception, just like I was doing in my code.
Thanks to #GauravMantri to pointing me the right way (i.e. have a look at how to use the autocomplete configuration parameter).
Azure Service Bus Queue has this Max Delivery Count property that you can make use of. Considering you only want to process a message exactly once and then deadletter the message in case Function is unable to process, what you can do is set the max delivery count to 1. That way the message will be automatically deadlettered after 1st delivery.
By default, Function runtime tries to auto-complete the message if there is no exception in processing the message. You do not want Function runtime to do that. For that what you would need to do is set auto complete setting to false. However if the message is processed successfully, you would want to delete that message thus you will need to call auto complete manually if the message processing is successful.
Something like:
if 'valid' in msg:
instance_id = await client.start_new('orchestrator', None, json.dumps(message_body))
//auto complete the message here...
else:
//do nothing and the message will be dead-lettered
UPDATE
For me the Problem got fixed as soon as I was putting "encoding: URLEncoding(destination: .queryString)" in my request. Maybe this helps somebody else. link
I struggled the whole day to find the problem in my Alamofire PUT Request or the Flask Restful API. Request like GET, DELETE and POST are working fine with Alamofire, except the PUT Request.
When I'm using PUT Requests in combination with Postman and Flask-Restful everything is also working fine. But as soon as I'm trying to achieve the same Result with Alamofire, I'm not getting any parameters in Flask. I tried to illustrate this in the code examples.
So in short my example illustrates the following:
DELETE Request(Same with GET and POST)
Postman: success
Alamofire: success
PUT Request
Postman: success
Alamofire: failure (parameter dictionary empty in Flask-Restful)
Here is my Python Code [API Server]:
from flask import Flask, request, jsonify
from flask_restful import Resource, Api, reqparse
app = Flask(__name__)
api = Api(app)
class Stackoverflow(Resource):
def delete(self):
print(request.args)
if request.args.get('test-key') is None:
return jsonify({"message": "failure"})
else:
return jsonify({"message": "success"})
def put(self):
print(request.args)
if request.args.get('test-key') is None:
return jsonify({"message": "failure"})
else:
return jsonify({"message": "success"})
api.add_resource(Stackoverflow, '/stackoverflow')
if __name__ == '__main__':
app.run(debug=True, host='0.0.0.0')
If I'm using Postman, I get this result (like expected):
Result in Postman
But now I'm trying to do the same with Alamofire in Swift. Same Server, nothing changed.
SWIFT demo Code [IOS APP]:
import UIKit
import Alamofire
import SwiftyJSON
class ViewController: UIViewController {
override func viewDidLoad() {
super.viewDidLoad()
// Do any additional setup after loading the view
simplePUTRequest()
simpleDELETERequest()
}
func simplePUTRequest(){
AF.request("http://localhost:5000/stackoverflow", method: .put, parameters: ["test-key":"testvalue"])
.validate(statusCode: 200..<300)
.responseJSON { response in
if let data = response.data {
print("Result PUT Request:")
print(String(data: data, encoding: .utf8)!)
//print(utf8Text)
}else{
}
}
}
func simpleDELETERequest(){
AF.request("http://localhost:5000/stackoverflow", method: .delete, parameters: ["test-key":"testvalue"])
.validate(statusCode: 200..<300)
.responseJSON { response in
if let data = response.data {
print("Result DELETE Request:")
print(String(data: data, encoding: .utf8)!)
//print(utf8Text)
}else{
}
}
}
Xcode Console:
Result PUT Request:
{
"message": "failure"
}
Result DELETE Request:
{
"message": "success"
}
python Console (both Alamofire Requests):
ImmutableMultiDict([])
127.0.0.1 - - [15/Jun/2019 21:17:31] "PUT /stackoverflow HTTP/1.1" 200 -
ImmutableMultiDict([('test-key', 'testvalue')])
127.0.0.1 - - [15/Jun/2019 21:17:31] "DELETE /stackoverflow?test-key=testvalue HTTP/1.1" 200 -
As you can see, I'm getting the success message only while using the DELETE method.
Till now I tried using different encodings like URLEncoding.httpbody and URLEncoding.default, but nothing really helped.
For me it seems like it's a Alamofire/Swift Problem, because in Postman the same request method is working fine.
I would really appreciate your help, because I'm stuck and don't know anything further to do. I hope I didn't misunderstood something essential.
Thank you in advance!
I am currently using the same version AlamoFire, and when I use the PUT method, I use it as follows:
let request = AF.request(url, method: .put, parameters: ["uid": uid],
encoding: JSONEncoding.default, headers: headers)
request.responseJSON(completionHandler: { response in
guard response.error == nil else {
//Handle error
}
if let json = response.value as? [String: Any]
// Handle result.
}
The only difference to your post is that I used the encoding option. You can try to put the option and see what happens.
It looks like your server is expecting your PUT parameters to be URL form encoded into the URL. You may be hitting the version of the request method that uses JSON encoding by default, so adding encoder: URLEncodedFormParameterEncoder.default at the end of your request call should fix that. A future release will make that the default, as it's safe across all request types.
If that's not the issue, I suggest you investigate more closely to see what the differences between the requests may be. Since you control the server you should have easy access to the traffic.
i am trying to create bill for payment and send to my customer via telegram bot:
i am using blockchain API V2-https://blockchain.info/api/api receive .my code is:
xpub='***'
keyk='02e57f1***'
url='https://api.blockchain.info/v2/receive?xpub='+str(xpub)+'&callback=https%3A%2F%2Fdoors03.ru&key='+keyk
x=requests.get(url)
r=x.json()
r=r['address']
r -is an adress wich was made.
i am sending it to my costumer(by the way is there any way to send adress with exact sum for pay ) . After i want to check is payment was recieved:
data={ "Content-Type": "text/plain","key":keyk,"addr":r,"callback":"https%3A%2F%2Fdoors03.ru","onNotification":"KEEP", "op":"RECEIVE"}
r = requests.post(url, data=data)
and this is the response - u'{\n "message" : "Internal handlers error"\n}'
what i am doing wrong ? how to check payments ? how to send address with exact sum of btc or ethereum ?
Sorry, i don't have enough reputation to post a comment, so this is
the only option i have. #egorkh have you solved this problem? Maybe
you have received explanation from blockchain.info support? I have
sent them a question about that, but they are answering for too long.
UPDATE: Finally, i have found solution.
In my case, reason of "Internal handlers error" message is in a wrong interpretation of their API.
As they haven't implemented balance_update request in their java-api, i did it on my own and i did it in wrong way.
I have put this parameters:
{"key":keyk,"addr":r,"callback":"https%3A%2F%2Fdoors03.ru","onNotification":"KEEP", "op":"RECEIVE"}
as post parameters, like in other methods they have provided in api. In those methods parameters are URLEncoded like you did with callback link. But...
In this HTML request they must be sent as plain text in json format without any special encoding, like that:
Map<String, String> params = new HashMap<String, String>();
params.put("addr", address);
params.put("callback", callbackUrl);
params.put("key", apiCode);
params.put("onNotification", keepOnNotification? "KEEP" : "DELETE");
params.put("confs", Integer.toString(confirmationCount));
params.put("op", StringUtils.isBlank(operationType) ? "ALL" : operationType);
//parse parameters map to json string(that's optional: you can write it directly as string)
String body = new Gson().toJson(params);
if (requestMethod.equals("POST")) {
byte[] postBytes = body.getBytes("UTF-8");
conn.setDoOutput(true);
conn.setRequestProperty("Content-Type", "text/plain");
conn.setRequestProperty("Content-Length", String.valueOf(postBytes.length));
conn.getOutputStream().write(postBytes);
conn.getOutputStream().close();
}
The main reason of your error may be that you put "Content-Type": "text/plain" in data object (, and maybe encoded callback url) .
I am trying to upload a file using python requests to my java/scala spring rest server. I am getting the following response:
{"timestamp":1454331913056,"status":400,"error":"Bad Request","exception":"org.springframework.web.bind.MissingServletRequestParameterException","message":"Required MultipartFile parameter 'image' is not present","path":"/parking_api/images"}
my server code:
#RequestMapping(value = Array("/parking_api/images"), method = Array(RequestMethod.POST)) def images(#RequestParam("image") image: MultipartFile) = {
if (image.isEmpty){
// handle empty
new ResponseEntity(response, HttpStatus.BAD_REQUEST
} else {
// process
new ResponseEntity(response, HttpStatus.OK)
}
}
My client code:
requests.post(parking_webservice_address, files={"image": ("image", open(event.pathname, "rb"), "image/jpeg")})
I have tried:
setting files parameter in python code to just {"image":open(...)} instead of tuple
passing image data loaded to memory using open(...).read() to files parameter in python code
setting CommonsMultipartResolver in server like this
#Bean(name="multipartResolver")
public CommonsMultipartResolver multipartResolver(){
return new CommonsMultipartResolver();
}
setting multipart headers manually, but then it server expects boundary which is missing as things get from bad to worse
None of these options have worked for me. What am I missing?
I am developing a RESTFUL webservice using Django. On some occassion, we need to push the server object to the connected client without client polling.
We decided to use django-websocket 0.3.0.
I am writing the test cases and tried to connect to the server using nodejs ws client module
My View Function in Django is following:
from django.http import HttpResponse
from django_websocket import accept_websocket, require_websocket
from django.views.decorators.csrf import csrf_exempt
import json, sys, os, time, datetime
#csrf_exempt
#accept_websocket
def home(request) :
if not request.is_websocket():
return HttpResponse('new message')
else:
for message in request.websocket:
message = modify_message(message)
request.websocket.send(message)
request.websocket.close()
My Client Side code in js is like this:-
//Normal Get
var request = require('request');
request('http://127.0.0.1:8000',function(err,resp,flag){
console.log(resp.body);
});
//Opening a websocket
var WebSocket = require('ws');
var ws = new WebSocket('ws://127.0.0.1:8000/', {origin: 'http://127.0.0.1:8000'});
ws.on('open', function() {
console.log('connected');
ws.send(Date.now().toString(), {mask: true});
});
ws.on('close', function() {
console.log('disconnected');
});
ws.on('message', function(data, flags) {
console.log('Roundtrip time: ' + (Date.now() - parseInt(data)) + 'ms', flags);
setTimeout(function() {
ws.send(Date.now().toString(), {mask: true});
}, 500);
});
The first option gets the message as 'new message'
On the other side the second call throws the following error on the client side. On the server side, both commands pass through a 200OK
events.js:72
throw er; // Unhandled 'error' event
^
Error: unexpected server response (200)
at ClientRequest.<anonymous> (../ws/lib/WebSocket.js:603:17)
at ClientRequest.g (events.js:175:14)
at ClientRequest.EventEmitter.emit (events.js:95:17)
at HTTPParser.parserOnIncomingClient [as onIncoming] (http.js:1689:21)
at HTTPParser.parserOnHeadersComplete [as onHeadersComplete] (http.js:120:23)
at Socket.socketOnData [as ondata] (http.js:1584:20)
at TCP.onread (net.js:525:27)
On a side note if I log the request.is_websocket() on both calls it returns false meaning on the server side it never goes into the else part.
Please help me understand what mistake I am doing here
Thank you
Well,
I downloaded their entire code (not pip install) and run the supplied example chat program. Same error. The system sends a 400 response code for any ws:// call.
The git hub project page linked on the pypi site returns a 404 error. No way I can file a bug report. Emailed the developer and didn't get any response.
Probably something should have been broken on the new Django 1.5.2 version.
I consider that this is a dead project and hence moving to a complex but working solution like gevent or twisted.
thanks for your support!!!