I'm trying to send a couple of files to my backend:
class AccountsImporterTestCase(APITestCase):
def test(self):
data = [open('accounts/importer/accounts.csv'), open('accounts/importer/apartments.csv')]
response = self.client.post('/api/v1/accounts/import/', data, format='multipart')
self.assertEqual(response.status_code, status.HTTP_202_ACCEPTED)
But I'm getting an error:
Error
Traceback (most recent call last):
File "/vagrant/conjuntos/accounts/tests/cases.py", line 128, in test
response = self.client.post('/api/v1/accounts/import/', data, format='multipart')
File "/vagrant/venv/lib/python3.4/site-packages/rest_framework/test.py", line 168, in post
path, data=data, format=format, content_type=content_type, **extra)
File "/vagrant/venv/lib/python3.4/site-packages/rest_framework/test.py", line 89, in post
data, content_type = self._encode_data(data, format, content_type)
File "/vagrant/venv/lib/python3.4/site-packages/rest_framework/test.py", line 64, in _encode_data
ret = renderer.render(data)
File "/vagrant/venv/lib/python3.4/site-packages/rest_framework/renderers.py", line 757, in render
return encode_multipart(self.BOUNDARY, data)
File "/vagrant/venv/lib/python3.4/site-packages/django/test/client.py", line 156, in encode_multipart
for (key, value) in data.items():
AttributeError: 'list' object has no attribute 'items'
I know I'm not preparing the data correctly but is it possible to do it?, how?. Thanks!
Update: Trying #Kevin Brown solution
def test(self):
data = QueryDict('', mutable=True)
data.setlist('files', [open('accounts/importer/accounts.csv'), open('accounts/importer/apartments.csv')])
response = self.client.post('/api/v1/accounts/import/', data, format='multipart')
self.assertEqual(response.status_code, status.HTTP_202_ACCEPTED)
Got the following:
Error
Traceback (most recent call last):
File "/vagrant/conjuntos/accounts/tests/cases.py", line 130, in test
response = self.client.post('/api/v1/accounts/import/', data, format='multipart')
File "/vagrant/venv/lib/python3.4/site-packages/rest_framework/test.py", line 168, in post
path, data=data, format=format, content_type=content_type, **extra)
File "/vagrant/venv/lib/python3.4/site-packages/rest_framework/test.py", line 89, in post
data, content_type = self._encode_data(data, format, content_type)
File "/vagrant/venv/lib/python3.4/site-packages/rest_framework/test.py", line 64, in _encode_data
ret = renderer.render(data)
File "/vagrant/venv/lib/python3.4/site-packages/rest_framework/renderers.py", line 757, in render
return encode_multipart(self.BOUNDARY, data)
File "/vagrant/venv/lib/python3.4/site-packages/django/test/client.py", line 182, in encode_multipart
return b'\r\n'.join(lines)
TypeError: sequence item 4: expected bytes, bytearray, or an object with the buffer interface, str found
You are sending a list of files to the view, but you aren't sending them correctly. When you send data to a view, whether it is a Django view or DRF view, you are supposed to send it as a list of key-value pairs.
{
"key": "value",
"file": open("/path/to/file", "rb"),
}
To answer your question...
is it possible to do it?
It does not appear to be possible to upload multiple files using the same key (in tests), but it is possible to spread them out across multiple keys to achieve the same goal. Alternatively, you could set up your views to only handle a single file, and have multiple tests covering different test cases (apartments.csv, accounts.csv, etc.).
Your exception is being triggered because you are passing a single list instead of a dictionary, and Django cannot parse them correctly.
You might have some luck by directly forming the request dictionary using a QueryDict which is the internal representation of form data used by Django.
data = QueryDict(mutable=True)
data.setlist("files", [
open('accounts/importer/accounts.csv', 'rb'),
open('accounts/importer/apartments.csv', 'rb')
])
As this would more closely represent the data sent in through the browser. This has not been tested, but it's the way to send multiple non-file-values in one key.
Check if this solves your problem as from the errors its clearly says that the data should not be a list
data = {"account_csv": open('accounts/importer/accounts.csv'), "apartments_csv": open('accounts/importer/apartments.csv')}
This link you might find useful Uploading multiple files in a single request using python requests module
Related
ok so im using python 3
i was able to get the data of the api using print(endpoint.json())
but i want to make it readable with pandas, so i can iterate through it easier.
this is the code (keep in mind i discarded my own api key and im using rapid api as a resource (specificly the movie database)
import requests
import json
import pandas
url = "https://movies-tvshows-data-imdb.p.rapidapi.com/"
querystring = {"type":"get-popular-movies","page":"1","year":"2020"}
headers = {
'x-rapidapi-host': "movies-tvshows-data-imdb.p.rapidapi.com",
'x-rapidapi-key': my key
}
response = requests.request("GET", url, headers=headers, params=querystring)
data=response.json()
df=pandas.read_json(data)
print(df)
i get this error
Traceback (most recent call last):
File "c:\Users\Home\Documents\studying\newproject\newproject.py", line 15, in <module>
df=pandas.read_json(data)
File "C:\Users\Home\AppData\Local\Programs\Python\Python38\lib\site-packages\pandas\util\_decorators.py", line 199, in wrapper
return func(*args, **kwargs)
File "C:\Users\Home\AppData\Local\Programs\Python\Python38\lib\site-packages\pandas\util\_decorators.py", line 296, in wrapper
return func(*args, **kwargs)
File "C:\Users\Home\AppData\Local\Programs\Python\Python38\lib\site-packages\pandas\io\json\_json.py", line 593, in read_json
filepath_or_buffer, _, compression, should_close = get_filepath_or_buffer(
File "C:\Users\Home\AppData\Local\Programs\Python\Python38\lib\site-packages\pandas\io\common.py", line 243, in get_filepath_or_buffer
raise ValueError(msg)
ValueError: Invalid file path or buffer object type: <class 'dict'>
In your case data is a dict.
So, try with:
pandas.DataFrame.from_dict(data)
I'm pulling data from my database and trying to pass it my web app via the API using python requests. I've done this several times, but I'm having a hard time on this script.
It gives me "ValueError: not enough values to unpack (expected 2, got 1)"
rpt_cursor = rpt_conn.cursor()
sql = """SELECT `field` FROM `db`.`table`;"""
rpt_cursor.execute(sql)
#Creating list of field names.
num_fields = len(rpt_cursor.description)
field_names = [i[0] for i in rpt_cursor.description]
# getting results and defining dict to load them into.
results = rpt_cursor.fetchall()
dictofemails = []
print('These are the SQL results....')
print(results) # These look fine.
# Appending data to the dict.
for row in results:
dictofdata.append(dict(zip(field_names,row)))
print('These are the dict results...')
print(dictofdata) # Once again this looks like a fine dict with a format of [{'field_name' : 'xyz'}, {'field_name' : 'abc'}].
api_request_url = 'https://api.domainname.com/api/list/' + str(target) +'/Api_Key/' + my_apikey
print('api_request_url') # This looks fine.
response = requests.put(api_request_url, headers=headers, data=dictofdata)
print(response)
print(response.content)
Any clues you guys could give me would be appreciated.
Edit:
Traceback as requested....
Traceback (most recent call last):
File "NGM_ListMaker_WeeklyBounceLoad.py", line 306, in <module>
response = requests.put(api_request_url, headers=headers, data=dictofemails)
File "C:\Program Files (x86)\Python36-32\lib\site-packages\requests\api.py", line 126, in put
return request('put', url, data=data, **kwargs)
File "C:\Program Files (x86)\Python36-32\lib\site-packages\requests\api.py", line 58, in request
return session.request(method=method, url=url, **kwargs)
File "C:\Program Files (x86)\Python36-32\lib\site-packages\requests\sessions.py", line 494, in request
prep = self.prepare_request(req)
File "C:\Program Files (x86)\Python36-32\lib\site-packages\requests\sessions.py", line 437, in prepare_request
hooks=merge_hooks(request.hooks, self.hooks),
File "C:\Program Files (x86)\Python36-32\lib\site-packages\requests\models.py", line 308, in prepare
self.prepare_body(data, files, json)
File "C:\Program Files (x86)\Python36-32\lib\site-packages\requests\models.py", line 499, in prepare_body
body = self._encode_params(data)
File "C:\Program Files (x86)\Python36-32\lib\site-packages\requests\models.py", line 97, in _encode_params
for k, vs in to_key_val_list(data):
ValueError: not enough values to unpack (expected 2, got 1)
Edit with the Answer:
The API was expecting a string instead of the list that I was passing it. Changing the API call to data=str(dictofdata) was all that was needed. Thanks for the help.
The API was expecting a string instead of the list I was trying to pass.
Change this...
response = requests.put(api_request_url, headers=headers, data=dictofdata)
to this...
response = requests.put(api_request_url, headers=headers, data=str(dictofdata))
Works fine now. Thanks everyone.
I am trying to use Face API Python 2.7. I wanted to make gender recognition by photo but always getting the error. This is my code:
from six.moves import urllib
import httplib
import json
params = urllib.urlencode({
'returnFaceId': 'true',
'returnFaceAttributes': 'true',
'returnFaceAttributes': '{string}',
})
headers = {
'ocp-apim-subscription-key': "ee6b8785e7504dfe91efb96d37fc7f51",
'content-type': "application/octet-stream"
}
img = open("d:/Taylor.jpg", "rb")
conn = httplib.HTTPSConnection("api.projectoxford.ai")
conn.request("POST", "/vision/v1.0/tag?%s" % params, img, headers)
res = conn.getresponse()
data = res.read()
conn.close()
I've got this error :
Traceback (most recent call last):
File "<ipython-input-314-df31294bc16f>", line 3, in <module>
res = conn.getresponse()
File "d:\Anaconda2\lib\httplib.py", line 1136, in getresponse
response.begin()
File "d:\Anaconda2\lib\httplib.py", line 453, in begin
version, status, reason = self._read_status()
File "d:\Anaconda2\lib\httplib.py", line 409, in _read_status
line = self.fp.readline(_MAXLINE + 1)
File "d:\Anaconda2\lib\socket.py", line 480, in readline
data = self._sock.recv(self._rbufsize)
File "d:\Anaconda2\lib\ssl.py", line 756, in recv
return self.read(buflen)
File "d:\Anaconda2\lib\ssl.py", line 643, in read
v = self._sslobj.read(len)
error: [Errno 10054]
If I use link instead photo:
img_url='https://raw.githubusercontent.com/Microsoft/Cognitive-Face-Windows/master/Data/detection1.jpg'
conn = httplib.HTTPSConnection("api.projectoxford.ai")
conn.request("POST", "/vision/v1.0/tag?%s" % params, img_url, headers)
res = conn.getresponse()
data = res.read()
conn.close()
I get:
data
Out[330]: '{ "statusCode": 401, "message": "Access denied due to invalid subscription key. Make sure to provide a valid key for an active subscription." }'
If I use:
KEY = 'ee6b8785e7504dfe91efb96d37fc7f51'
CF.Key.set(KEY)
img_url = 'https://raw.githubusercontent.com/Microsoft/Cognitive-Face-Windows/master/Data/detection1.jpg'
result = CF.face.detect(img_url)
all works fine:
result
[{u'faceId': u'52c0d1ac-f041-49cd-a587-e81ef67be2fb',
u'faceRectangle': {u'height': 213,
u'left': 154,
u'top': 207,
u'width': 213}}]
But in this case I don't know how to use method returnFaceAttribute (for gender detection)
and also if I use img in result = CF.face.detect(img_url) instead img_url
I get an error: status_code: 400
response: {"error":{"code":"InvalidImageSize","message":"Image size is too small or too big."}}
Traceback (most recent call last):
File "<ipython-input-332-3fe2623ccadc>", line 1, in <module>
result = CF.face.detect(img)
File "d:\Anaconda2\lib\site-packages\cognitive_face\face.py", line 41, in detect
data=data)
File "d:\Anaconda2\lib\site-packages\cognitive_face\util.py", line 84, in request
error_msg.get('message'))
CognitiveFaceException: Image size is too small or too big.
This happends with all sorts of img sizes.
Could anyone explain how to solve these problems?
It seems like you need to use your API key when connecting to Face API. That's why your second example works and the others don't.
Code 401 means that you're unauthorised, i.e. at that point you didn't log in with your key.
Maybe it'd be easier for you to try and use requests instead of urllib.
The first and error is caused by sending file object instead of file content. You should send image.read() instead of image.
The second error is caused by lacking of subscription key in the request header.
For the last error, I suspect you may have read the image file object once and thus reading it again will return empty result. You can try to send file path instead and it should work with the helper in Face API Python SDK.
I'm trying to write a redditbot; I decided to start with a simple one, to make sure I was doing things properly, and I got a RequestException.
my code (bot.py):
import praw
for s in praw.Reddit('bot1').subreddit("learnpython").hot(limit=5):
print s.title
my praw.ini file:
# The URL prefix for OAuth-related requests.
oauth_url=https://oauth.reddit.com
# The URL prefix for regular requests.
reddit_url=https://www.reddit.com
# The URL prefix for short URLs.
short_url=https://redd.it
[bot1]
client_id=HIDDEN
client_secret=HIDDEN
password=HIDDEN
username=HIDDEN
user_agent=ILovePythonBot0.1
(where HIDDEN replaces the actual id, secret, password and username.)
my Traceback:
Traceback (most recent call last):
File "bot.py", line 3, in <module>
for s in praw.Reddit('bot1').subreddit("learnpython").hot(limit=5):
File "/usr/local/lib/python2.7/dist-packages/praw/models/listing/generator.py", line 79, in next
return self.__next__()
File "/usr/local/lib/python2.7/dist-packages/praw/models/listing/generator.py", line 52, in __next__
self._next_batch()
File "/usr/local/lib/python2.7/dist-packages/praw/models/listing/generator.py", line 62, in _next_batch
self._listing = self._reddit.get(self.url, params=self.params)
File "/usr/local/lib/python2.7/dist-packages/praw/reddit.py", line 322, in get
data = self.request('GET', path, params=params)
File "/usr/local/lib/python2.7/dist-packages/praw/reddit.py", line 406, in request
params=params)
File "/usr/local/lib/python2.7/dist-packages/prawcore/sessions.py", line 131, in request
params=params, url=url)
File "/usr/local/lib/python2.7/dist-packages/prawcore/sessions.py", line 70, in _request_with_retries
params=params)
File "/usr/local/lib/python2.7/dist-packages/prawcore/rate_limit.py", line 28, in call
response = request_function(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/prawcore/requestor.py", line 48, in request
raise RequestException(exc, args, kwargs)
prawcore.exceptions.RequestException: error with request request() got an unexpected keyword argument 'json'
Any help would be appreciated. PS, I am using Python 2.7., on Ubuntu 14.04. Please ask me for any other information you may need.
The way i see it, it seems you have a problem with your request to Reddit API. Maybe try changing the user-agent in your in-file configuration. According to PRAW basic configuration Options the user-agent should follow a format <platform>:<app ID>:<version string> (by /u/<reddit username>) . Try that see what happens.
I have written a unit test to test an api....Its a GET call....
When I run it , i get this error ...Heres the traceback....
Traceback (most recent call last):
File "/home/arindam31/XXX-Name/mapi/tests.py", line 141, in test_get_cities
response = self.cl.get('/mapi/latest/cities/')
File "/usr/local/lib/python2.7/dist-packages/django/test/client.py", line 445, in get
response = super(Client, self).get(path, data=data, **extra)
File "/usr/local/lib/python2.7/dist-packages/django/test/client.py", line 229, in get
return self.request(**r)
File "/usr/local/lib/python2.7/dist-packages/django/test/client.py", line 387, in request
response = self.handler(environ)
File "/usr/local/lib/python2.7/dist-packages/django/test/client.py", line 84, in __call__
response = self.get_response(request)
File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py", line 169, in get_response
response = self.handle_uncaught_exception(request, resolver, sys.exc_info())
File "/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py", line 218, in handle_uncaught_exception
return callback(request, **param_dict)
File "/home/arindam31/XXX-Name/saul/views.py", line 546, in do_handle_500
return render_to_response("500.html", context_instance=RequestContext(request))
File "/usr/local/lib/python2.7/dist-packages/django/template/context.py", line 177, in __init__
self.update(processor(request))
File "/home/arindam31/XXX-Name/saul/context_processors.py", line 46, in common_context
ret_dict['city'] = request.city
AttributeError: 'WSGIRequest' object has no attribute 'city'
Heres my unit test...
def test_get_cities(self):
request = HttpRequest()
request.city = self.c1
response = self.cl.get('/mapi/latest/cities/')
content = response.content
data = json.loads(content)
for city in City.objects.all():
assert city.name in data
assert data[city.name] == city.pk
Here , self.c1 is a city type object in the setUp part....HttpRequest is from django.http.
The view being tested is below:
def get_cities(request):
print "1"
if ENABLE_HTTPS_CHECK and not request.is_secure():
return HttpResponseForbidden()
if request.method != 'GET':
return HttpResponseNotAllowed('Not allowed')
city_map = _get_city_map()
response = HttpResponse(json.dumps(city_map)
content_type='application/json')
response['Cache-Control'] = 'no-cache'
return response
If you want to test your view with your own request object, you should call the view directly. You should use the RequestFactory to create your request.
As Daniel Roseman points out your request object having a city attribute remains broken, unless you have some middleware that would set it. You clearly have some middleware that require it to be set in saul/context_processors.py.
Using RequestFactory and calling the view directly circumvents the middleware entirely, so you can concentrate on testing your view (if the view supports the absent middleware).
If your view requires the middelware to be operational, you probably just need to be logged in using the test client. It has a session pr test method.
I have no idea what you are trying to do here. You instantiate a request object, assign a city attribute, then proceed to ignore that object and just use the standard test client. I don't know why you would think that that request would be used for the client get.
To be honest though I think that your entire design is broken. You don't show how you expect to get the parameter into the request in a normal non-test scenario, but usually you would pass it via the POST or GET parameters rather than annotating it into the request somehow. That would of course make it easier to test, since you could just pass the dictionary to the client call.