In order to test a Flask application, I got a flask test client POSTing request with files as attachment
def make_tst_client_service_call1(service_path, method, **kwargs):
_content_type = kwargs.get('content-type','multipart/form-data')
with app.test_client() as client:
return client.open(service_path, method=method,
content_type=_content_type, buffered=True,
follow_redirects=True,**kwargs)
def _publish_a_model(model_name, pom_env):
service_url = u'/publish/'
scc.data['modelname'] = model_name
scc.data['username'] = "BDD Script"
scc.data['instance'] = "BDD Stub Simulation"
scc.data['timestamp'] = datetime.now().strftime('%d-%m-%YT%H:%M')
scc.data['file'] = (open(file_path, 'rb'),file_name)
scc.response = make_tst_client_service_call1(service_url, method, data=scc.data)
Flask Server end point code which handles the above POST request is something like this
#app.route("/publish/", methods=['GET', 'POST'])
def publish():
if request.method == 'POST':
LOG.debug("Publish POST Service is called...")
upload_files = request.files.getlist("file[]")
print "Files :\n",request.files
print "Upload Files:\n",upload_files
return render_response_template()
I get this Output
Files:
ImmutableMultiDict([('file', <FileStorage: u'Single_XML.xml' ('application/xml')>)])
Upload Files:
[]
If I change
scc.data['file'] = (open(file_path, 'rb'),file_name)
into (thinking that it would handle multiple files)
scc.data['file'] = [(open(file_path, 'rb'),file_name),(open(file_path, 'rb'),file_name1)]
I still get similar Output:
Files:
ImmutableMultiDict([('file', <FileStorage: u'Single_XML.xml' ('application/xml')>), ('file', <FileStorage: u'Second_XML.xml' ('application/xml')>)])
Upload Files:
[]
Question:
Why request.files.getlist("file[]") is returning an empty list?
How can I post multiple files using flask test client, so that it can be retrieved using request.files.getlist("file[]") at flask server side ?
Note:
I would like to have flask client I dont want curl or any other client based solutions.
I dont want to post single file in multiple requests
Thanks
Referred these links already:
Flask and Werkzeug: Testing a post request with custom headers
Python - What type is flask.request.files.stream supposed to be?
You send the files as the parameter named file, so you can't look them up with the name file[]. If you want to get all the files named file as a list, you should use this:
upload_files = request.files.getlist("file")
On the other hand, if you really want to read them from file[], then you need to send them like that:
scc.data['file[]'] = # ...
(The file[] syntax is from PHP and it's used only on the client side. When you send the parameters named like that to the server, you still access them using $_FILES['file'].)
Lukas already addressed this,just providing these info as it may help someone
Werkzeug client is doing some clever stuff by storing requests data in MultiDict
#native_itermethods(['keys', 'values', 'items', 'lists', 'listvalues'])
class MultiDict(TypeConversionDict):
"""A :class:`MultiDict` is a dictionary subclass customized to deal with
multiple values for the same key which is for example used by the parsing
functions in the wrappers. This is necessary because some HTML form
elements pass multiple values for the same key.
:class:`MultiDict` implements all standard dictionary methods.
Internally, it saves all values for a key as a list, but the standard dict
access methods will only return the first value for a key. If you want to
gain access to the other values, too, you have to use the `list` methods as
explained below.
getList call looks for a given key in the "requests" dictionary. If the key doesn't exist, it returns empty list.
def getlist(self, key, type=None):
"""Return the list of items for a given key. If that key is not in the
`MultiDict`, the return value will be an empty list. Just as `get`
`getlist` accepts a `type` parameter. All items will be converted
with the callable defined there.
:param key: The key to be looked up.
:param type: A callable that is used to cast the value in the
:class:`MultiDict`. If a :exc:`ValueError` is raised
by this callable the value will be removed from the list.
:return: a :class:`list` of all the values for the key.
"""
try:
rv = dict.__getitem__(self, key)
except KeyError:
return []
if type is None:
return list(rv)
result = []
for item in rv:
try:
result.append(type(item))
except ValueError:
pass
return result
Related
I have a list named linebline that contains around 99 lists within it (sublists). Within these sublists are dictionaries. I am interested in these dictionaries, as each sublist has a key named 'title' that I want a client/user to be able to query using a URL.
type(linebline) outputs list
type(linebline[0]) outputs dict
So far I have the following:
from flask import Flask, request, Response
app3 = Flask(__name__)
app3.route('/', methods = ['GET'])
def grab_title():
given_title = requests.args.get('title')
book_info = [obj for obj in linebline if obj['title'] == given_title] #.to_html(header="true", table_id="table")
return(book_info)
#return Response(city_inCom, mimetype = 'application/json')
app3.run(host = 'localhost', port = 5003)
An example of the query from a client would be:"http://localhost:5003/?title=New%20iPad%20Air%20may%20come%20with%20USB-C%20not%20Lightning%20Port%27"
I am currently getting:
Not Found
The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again.
I am not sure if I am addressing the 'title' key correctly OR am I not returning the right thing.
I am launching REST API build in python.
I am inputting a list as input and getting the required data.
example:
your.api.com/birth?name=James&date=2015-02-01&name=Robert&date=2020-01-01
from flask import request
#app.route('/birth')
def birth():
names = request.form.getlist('name')
dates = request.form.getlist('date')
As the number of inputs I have is huge, the end point URL is becoming huge. Is there any way to do the same using PUT or POST where I dump a doc in some format (say json) as my input?
Sure, put the request fields in a JSON and go through the dictionary
#app.route('/birth', methods=['POST'])
def birth():
data = request.get_json()
names = data["names"]
# etc.
When the input grows it is suggested to use a POST call rather than GET call.
from flask import request
#app.route('/birth')
def birth():
names = request.form.getlist('name')
dates = request.form.getlist('date')
Convert /birth to the following
from flask import request
#app.route('/birth', methods=['POST'])
def birth():
input = request.get_json()
# <Do the processing>
And in the POST call from client-side use a JSON like the following
{
name: [<array of values>],
date: [<array of values>]
}
I want to call a generate() function and send a user a message, but then continue executing a function.
#application.route("/api/v1.0/gen", methods=['POST'])
def generate():
return "Your id for getting the generated data is 'hgF8_dh4kdsRjdr'"
main() #generate a data
return "Successfully generated something. Use your id to get the data"
I understand that this is not a correct way of returning, but I hope you get the idea of what I am trying to accomplish. Maybe Flask has some build-in method to return multiple times from one api call?
Basically, what are you describing is called Server-Sent Events (aka SSE)
The difference of this format, that they returned an 'eventstream' Response type instead of usual JSON/plaintext
And if you want to use it with python/flask, you need generators.
Small code example (with GET request):
#application.route("/api/v1.0/gen", methods=['GET'])
def stream():
def eventStream():
text = "Your id for getting the generated data is 'hgF8_dh4kdsRjdr'"
yield str(Message(data = text, type="message"))
main()
text = "Successfully generated something. Use your id to get the data"
yield str(Message(data = text, type="message"))
resp.headers['Content-Type'] = 'text/event-stream'
resp.headers['Cache-Control'] = 'no-cache'
resp.headers['Connection'] = 'keep-alive'
return resp
Message class you can find here: https://gist.github.com/Alveona/b79c6583561a1d8c260de7ba944757a7
And of course, you need specific client that can properly read such responses.
postwoman.io supports SSE at Real-Time tab
I am trying to pass comma separated query parameters to a Flask endpoint.
An example URI would be:
localhost:3031/someresource#?status=1001,1002,1003
Looking at the return of request.args or request.args.getlist('status') I see that I only get a string.
ipdb> pp request.args
ImmutableMultiDict([('status', '1001,1002,1003')])
ipdb> request.args.getlist('status')
['1001,1002,1003']
I know I can split the string by comma but that feels hacky. Is there a more idiomatic way to handle this in Flask? Or are my query params wrong format?
Solution
Since Flask does not directly support comma separated query params, I put this in in my base controller to support comma-separated or duplicate query params on all endpoints.
request_data = {}
params = request.args.getlist('status') or request.form.getlist('status')
if len(params) == 1 and ',' in params[0]:
request_data['status'] = comma_separated_params_to_list(params[0])})
else:
request_data['status'] = params
def comma_separated_params_to_list(param):
result = []
for val in param.split(','):
if val:
result.append(val)
return result
The flask variant getlist expects multiple keys:
from flask import Flask, request
app = Flask(__name__)
#app.route('/')
def status():
first_status = request.args.get("status")
statuses = request.args.getlist("status")
return "First Status: '{}'\nAll Statuses: '{}'".format(first_status, statuses)
❯ curl "http://localhost:5000?status=5&status=7"
First Status: '5'
All Statuses: '['5', '7']'
There's no standard for this, how multiple GET args are parsed/passed depends on which language/framework you're using; flask is built on werkzeug so it allows this style, but you'll have to look it up if you switch away from flask.
As an aside, Its not uncommon REST API design to have commas to pass multiple values for the same key - makes it easier for the user. You're parsing GET args anyway, parsing a resulting string is not that much more hacky. You can choose to raise a 400 HTTP error if their string with commas isn't well formatted.
Some other languages (notably PHP) support 'array' syntax, so that is used sometimes:
/request?status[]=1000&status[]=1001&status[]=1002
This is what you might want here:
request.args.to_dict(flat=False)
flat is True by default, so by setting it to False, you allow it to return a dict with values inside a list when there's more than one.
According to to_dict documentation:
to_dict(flat=True)
Return the contents as regular dict. If flat is True
the returned dict will only have the first item present, if flat is False
all values will be returned as lists.
How to decode messages.Message to JSON in python 2.7 for Google Cloud Endpoints Frameworks ? Especially when we have some nested messages.
Endpoints version :
google-endpoints==2.4.5 and google-endpoints-api-management==1.3.0
from protorpc import messages
# messsage definition
class GPSCoord(messages.Message):
"""
GPS data obj
"""
latitude = messages.FloatField(1)
longitude = messages.FloatField(2)
class Address(messages.Message):
"""
Address objectt
"""
type = messages.StringField(1)
name = messages.StringField(2)
number = messages.StringField(3)
city = messages.StringField(4)
zip_code = messages.IntegerField(5)
gps_coord = messages.MessageField(GPSCoord, 6)
I tried to add a method "to_json" to messages definition but I had "MessageDefinitionError: May only use fields in message definitions. " exception.
It looks like a rudimentary operation but it's not that easy. Python SDK needs a huge improvement for this part.
You should make use of the built-in Endpoints JSON code. This is not exact, but something like this:
from endpoints import protojson
p = protojson.EndpointsProtoJson()
p.decode_message(Address, '{...}')
I developed my own solution finally, here the code :
def request_to_json(request):
"""
Take a coming request (POST) and
get JSON.
"""
json_dict = {}
for field in request.all_fields():
if field.__class__.__name__ == 'MessageField':
data = getattr(request, field.name)
if data:
if data.__class__.__name__ == 'FieldList':
json_dict.update({
field.name: [request_to_json(data[i]) for i in range(len(data))]
})
else:
json_dict.update({
field.name: request_to_json(data)
})
else:
json_dict.update({
field.name: getattr(request, field.name)
})
return json_dict
It considers the nested messages fields, the list fields and primitive fields.
I tested it on POST requests and it works well.