Develop Reference

using request api into a json file that i can iterate through

ok so im using python 3
i was able to get the data of the api using print(endpoint.json())
but i want to make it readable with pandas, so i can iterate through it easier.
this is the code (keep in mind i discarded my own api key and im using rapid api as a resource (specificly the movie database)
import requests
import json
import pandas
url = "https://movies-tvshows-data-imdb.p.rapidapi.com/"
querystring = {"type":"get-popular-movies","page":"1","year":"2020"}
headers = {
'x-rapidapi-host': "movies-tvshows-data-imdb.p.rapidapi.com",
'x-rapidapi-key': my key
}
response = requests.request("GET", url, headers=headers, params=querystring)
data=response.json()
df=pandas.read_json(data)
print(df)
i get this error
Traceback (most recent call last):
File "c:\Users\Home\Documents\studying\newproject\newproject.py", line 15, in <module>
df=pandas.read_json(data)
File "C:\Users\Home\AppData\Local\Programs\Python\Python38\lib\site-packages\pandas\util\_decorators.py", line 199, in wrapper
return func(*args, **kwargs)
File "C:\Users\Home\AppData\Local\Programs\Python\Python38\lib\site-packages\pandas\util\_decorators.py", line 296, in wrapper
return func(*args, **kwargs)
File "C:\Users\Home\AppData\Local\Programs\Python\Python38\lib\site-packages\pandas\io\json\_json.py", line 593, in read_json
filepath_or_buffer, _, compression, should_close = get_filepath_or_buffer(
File "C:\Users\Home\AppData\Local\Programs\Python\Python38\lib\site-packages\pandas\io\common.py", line 243, in get_filepath_or_buffer
raise ValueError(msg)
ValueError: Invalid file path or buffer object type: <class 'dict'>

In your case data is a dict.
So, try with:
pandas.DataFrame.from_dict(data)

python-requests: "NoneType" has no attribute read

I'm trying to create a Python Wrapper for an upcoming API, so far I've been getting along well, but I keep bumping my head into the same problem. When making a POST request to add a mod to a game through the API, I feed the following dict item into the POST request
{'name': "Necro's Test Mod", 'name_id': None, 'summary': "This is a test mod submitted through the API using python's request library.", 'description': "I'm only about 43% sure this will actually work, whats more likely is that it will fail and return some messed up error with some even weirder error message before i am contacted by the mod.io developper saying they have personally revoked my api key (This isn't actually true i just need words", 'homepage': 'www.edain.wikia.com', 'metadata_blob': 'None', 'stock': 1, 'tags': ['cool', 'python', 'api'], 'logo': <_io.BufferedReader name='C:\\Users\\Clement\\Pictures\\Background\\7z6cSaI-lord-of-the-rings-wallpaper-hd.jpg'>}
And that dictionnary is added as follows
BASE_PATH = "https://api.test.mod.io/v1"
headers = {
'Authorization': 'Bearer ' + client.access_token,
'Content-Type': 'multipart/form-data',
'Accept': 'application/json'
}
r = requests.post(BASE_PATH + '/games/181/mods'.format(self.id), files = dict, headers = headers)
However it returns the following traceback, leading me to believe that the API returns a blank response
Traceback (most recent call last):
File "C:\Users\Clement\Desktop\mod.io\test.py", line 40, in <module>
new_mod = game.add_mod(newmod)
File "C:\Users\Clement\Desktop\mod.io\modio\game.py", line 83, in add_mod
r = requests.post(BASE_PATH + '/games/{}/mods'.format(self.id), files = mod.__dict__, headers = headers)
File "C:\Users\Clement\AppData\Local\Programs\Python\Python36\lib\site-packages\requests\api.py", line 112, in post
return request('post', url, data=data, json=json, **kwargs)
File "C:\Users\Clement\AppData\Local\Programs\Python\Python36\lib\site-packages\requests\api.py", line 58, in request
return session.request(method=method, url=url, **kwargs)
File "C:\Users\Clement\AppData\Local\Programs\Python\Python36\lib\site-packages\requests\sessions.py", line 494, in request
prep = self.prepare_request(req)
File "C:\Users\Clement\AppData\Local\Programs\Python\Python36\lib\site-packages\requests\sessions.py", line 437, in prepare_request
hooks=merge_hooks(request.hooks, self.hooks),
File "C:\Users\Clement\AppData\Local\Programs\Python\Python36\lib\site-packages\requests\models.py", line 308, in prepare
self.prepare_body(data, files, json)
File "C:\Users\Clement\AppData\Local\Programs\Python\Python36\lib\site-packages\requests\models.py", line 496, in prepare_body
(body, content_type) = self._encode_files(files, data)
File "C:\Users\Clement\AppData\Local\Programs\Python\Python36\lib\site-packages\requests\models.py", line 159, in _encode_files
fdata = fp.read()
AttributeError: 'NoneType' object has no attribute 'read'
Here's the documentation for the particular POST request I am attempting.

The files argument is meant for file uploads. You probably want to use data instead here.
r = requests.post(
BASE_PATH + '/games/181/mods'.format(self.id),
data = dict,
headers = headers
)
You get this error because requests expects the value of name_id to be a file pointer with a read() method. But in your payload it's None.
And don't use dict as a variable name.

Ebay API / ebaysdk "GetSellerList"

I'm trying to retrieve the active and completed items under a list of specified sellers. I've read a few things in a few different places, but haven't really been able to find something that I understand or anything I could work with. I think it might have something to do with my headers or maybe authentication or something, but I'm also uncertain if its just the way that I'm setting up my request. I have only ever used the Finding API, and it looks like I need to use Trading instead, but I'm not sure what little details I might be missing when I tried to switch API's
from ebaysdk.trading import Connection as trading
api = trading(appid='API_KEY_HERE', config_file=None)
api_request = {
'keywords': 'new'}
'outputSelector': 'SellerInfo',
'itemFilter': [
{'name': 'UserId',
'value': 'SELLER_NAME_HERE'},
{'name': 'StartTimeFrom',
'value': QueryTime_Start},
{'name': 'StartTimeTo',
'value': QueryTime_Complete}]}
response = api.execute('GetSellerList', api_request).
and my complete error message:
Traceback (most recent call last): File "C:\Users\dc500\Downloads\All
3.0\All 3\Python\3Pies\Project_Ordered Tools\practice_getsellerlist.py", line 24, in response =
api.execute('GetSellerList', api_request) File
"C:\Users\dc500\AppData\Local\Programs\Python\Python35\lib\site-packages\ebaysdk\connection.py",
line 120, in execute self.execute_request() File
"C:\Users\dc500\AppData\Local\Programs\Python\Python35\lib\site-packages\ebaysdk\connection.py",
line 184, in execute_request allow_redirects=True File
"C:\Users\dc500\AppData\Local\Programs\Python\Python35\lib\site-packages\requests\sessions.py",
line 576, in send r = adapter.send(request, **kwargs) File
"C:\Users\dc500\AppData\Local\Programs\Python\Python35\lib\site-packages\requests\adapters.py",
line 376, in send timeout=timeout File
"C:\Users\dc500\AppData\Local\Programs\Python\Python35\lib\site-packages\requests\packages\urllib3\connectionpool.py",
line 559, in urlopen body=body, headers=headers) File
"C:\Users\dc500\AppData\Local\Programs\Python\Python35\lib\site-packages\requests\packages\urllib3\connectionpool.py",
line 353, in _make_request conn.request(method, url,
**httplib_request_kw) File "C:\Users\dc500\AppData\Local\Programs\Python\Python35\lib\http\client.py",
line 1084, in request self._send_request(method, url, body, headers)
File
"C:\Users\dc500\AppData\Local\Programs\Python\Python35\lib\http\client.py",
line 1124, in _send_request self.putheader(hdr, value) File
"C:\Users\dc500\AppData\Local\Programs\Python\Python35\lib\http\client.py",
line 1060, in putheader if _is_illegal_header_value(values[i]):
TypeError: expected string or bytes-like object

from ebaysdk.trading import Connection as Trading
try:
api = Trading(config_file='ebay.yaml')
except ConnectionError as e:
print(e)
print(e.response.dict())
log.error('Attempting to get an API object failed with %s', e)
def get_active_listings(page):
acitvelist = api.execute('GetMyeBaySelling', {'ActiveList': True,
'DetailLevel': 'ReturnAll',
'PageNumber': page})
return acitvelist.dict()
This works for me and returns all active listings - maybe GetMyeBaySellingis a better call to use ?
Other than that
https://developer.ebay.com/devzone/XML/docs/Reference/eBay/GetSellerList.html#Input
does not seem to give the inputFilter as an option for parameters? there is outputSelector
??

Having issues getting firebase data through Oauth pin-auth session for Nest

Base script:
from sanction import Client
# client_id & client_secret are omitted but are valid
client_pin = input('Enter PIN:')
access_token_url = 'https://api.home.nest.com/oauth2/access_token'
c = Client(
token_endpoint=access_token_url,
client_id=client_id,
client_secret=client_secret)
c.request_token(code = client_pin)
[See edits for history]
Running c.request('/devices') returned:
Traceback (most recent call last):
File "C:\py\nest_testing_sanction.py", line 36, in <module>
c.request("/devices")
File "C:\Python34\lib\site-packages\sanction-0.4.1-py3.4.egg\sanction\__init__.py", line 169, in request
File "C:\Python34\lib\site-packages\sanction-0.4.1-py3.4.egg\sanction\__init__.py", line 211, in transport_query
File "C:\Python34\lib\urllib\request.py", line 258, in __init__
self.full_url = url
File "C:\Python34\lib\urllib\request.py", line 284, in full_url
self._parse()
File "C:\Python34\lib\urllib\request.py", line 313, in _parse
raise ValueError("unknown url type: %r" % self.full_url)
ValueError: unknown url type: 'None/devices?access_token=c.[some long session token]'
Given the output it seems like I need to be putting in a generic URL so I tried c.request('wss://developer-api.nest.com'):
Traceback (most recent call last):
File "C:\py\nest_testing_sanction.py", line 36, in <module>
data = c.request(query_url)
File "C:\Python34\lib\site-packages\sanction-0.4.1-py3.4.egg\sanction\__init__.py", line 171, in request
File "C:\Python34\lib\urllib\request.py", line 153, in urlopen
return opener.open(url, data, timeout)
File "C:\Python34\lib\urllib\request.py", line 455, in open
response = self._open(req, data)
File "C:\Python34\lib\urllib\request.py", line 478, in _open
'unknown_open', req)
File "C:\Python34\lib\urllib\request.py", line 433, in _call_chain
result = func(*args)
File "C:\Python34\lib\urllib\request.py", line 1257, in unknown_open
raise URLError('unknown url type: %s' % type)
urllib.error.URLError: <urlopen error unknown url type: nonewss>
I also tried https as per:
- same result
By contrast, this works (for a firebase.io virtual device):
firebase = firebase.FirebaseApplication('https://nesttest.firebaseio.com', None)
thermostat_result = firebase.get('/devices', 'thermostats')

In Python I would use something like sanction to keep things simple. You should be able to get it to work with the Nest API using code like: (untested, using token flow rather than pin flow)
from sanction.client import Client
# instantiating a client to get the auth URI
c = Client(auth_endpoint="https://home.nest.com/login/oauth2",
client_id=config["nest.client_id"])
# instantiating a client to process OAuth2 response
c = Client(token_endpoint="https://api.home.nest.com/oauth2/access_token",
client_id=config["nest.client_id"],
client_secret=config["nest.client_secret"])
The library is well documented, so you should be able to figure it out from here if something is missing.

This is more of a comment, but the system does not let me comment just yet.
To your question about where to put the web pin simply add code = pin to the request_token call.
c.request_token(code = nest_client_pin)
This still does not fully solve the issue as I can only use a PIN once. After I have used it once, every subsequent call will fail again as you describe. Still researching that.

Freebase python

I tried this python example from freebase and run it in my windows and ubuntu machine.
http://mql.freebaseapps.com/ch04.html
import sys # Command-line arguments, etc.
import simplejson # JSON encoding.
import urllib # URI encoding.
import urllib2 # High-level URL content fetching.
# These are some constants we'll use.
SERVER = 'api.freebase.com' # Metaweb server
SERVICE = '/api/service/mqlread' # Metaweb service
# Compose our MQL query as a Python data structure.
# The query is an array in case multiple bands share the same name.
band = sys.argv[1] # The desired band, from command line.
query = [{'type': '/music/artist', # Our MQL query in Python.
'name': band, # Place the band in the query.
'album': [{ 'name': None, # None is Python's null.
'release_date': None,
'sort': 'release_date' }]}]
# Put the query in an envelope
envelope = {
'query': query, # The query property specifies the query.
'escape': False # Turns off HTML escaping.
}
# These five lines are the key code for using mqlread
encoded = simplejson.dumps(envelope) # JSON encode the envelope.
params = urllib.urlencode({'query':encoded}) # Escape request parameters.
url ='http://%s%s?%s' % (SERVER,SERVICE,params) # The URL to request.
f = urllib2.urlopen(url) # Open the URL as a file.
response = simplejson.load(f) # Read and JSON parse response.
# Check for errors and exit with a message if the query failed.
if response['code'] != '/api/status/ok': # If not okay...
error = response['messages'][0] # First msg object.
sys.exit('%s: %s' % (error['code'], error['message'])) # Display code,msg.
# No errors, so handle the result
result = response['result'] # Open the response envelope, get result.
# Check the number of matching bands
if len(result) == 0:
sys.exit('Unknown band')
elif len(result) > 1:
print "Warning: multiple bands named " + band + ". Listing first only."
result = result[0] # Get first band from array of matches.
if not result['album']: # Exit if band has no albums
sys.exit(band + ' has no known albums.')
for album in result['album']: # Loop through the result albums.
name = album['name'] # Album name.
date = album['release_date'] # Release date timestamp or null.
if not date: date = ''
else: date = ' [%s]' % date[0:4] # Just the 4-digit year in brackets.
print "%s%s" % (name, date) # Print name and date.
However, it did not work. Could someone explain it why i got this error?
Traceback (most recent call last):
File "mql.py", line 29, in <module>
f = urllib2.urlopen(url) # Open the URL as a file.
File "C:\Python27\lib\urllib2.py", line 126, in urlopen
return _opener.open(url, data, timeout)
File "C:\Python27\lib\urllib2.py", line 394, in open
response = self._open(req, data)
File "C:\Python27\lib\urllib2.py", line 412, in _open
'_open', req)
File "C:\Python27\lib\urllib2.py", line 372, in _call_chain
result = func(*args)
File "C:\Python27\lib\urllib2.py", line 1199, in http_open
return self.do_open(httplib.HTTPConnection, req)
File "C:\Python27\lib\urllib2.py", line 1168, in do_open
h.request(req.get_method(), req.get_selector(), req.data, headers)
File "C:\Python27\lib\httplib.py", line 955, in request
self._send_request(method, url, body, headers)
File "C:\Python27\lib\httplib.py", line 989, in _send_request
self.endheaders(body)
File "C:\Python27\lib\httplib.py", line 951, in endheaders
self._send_output(message_body)
File "C:\Python27\lib\httplib.py", line 811, in _send_output
self.send(msg)
File "C:\Python27\lib\httplib.py", line 773, in send
self.connect()
File "C:\Python27\lib\httplib.py", line 754, in connect
self.timeout, self.source_address)
File "C:\Python27\lib\socket.py", line 562, in create_connection
sock.connect(sa)
One more thing, I am using python 2.7 and i don't have any special proxy configuration for the internet setting.

api.freebase.com has been retired and the Python client library was never updated to work with the new endpoint.

It would be better if you can enclose your code within a try-except block as below:
try:
#all the logic to create a URL and store in a variable say url for MQLReader service
invoke urllib2.urlopen(url)
#perform other processing.
except urllib2.URLError, e:
print str(e)
This will give you an idea about what is going wrong.
I tried your code sample and I get an error saying that:
A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond
Hence catching the exception and examining it will give you the root cause of the problem.