Develop Reference

How to search for books that have spaces in their title using Google books API

When i search for books with a single name(e.g bluets) my code works fine, but when I search for books that have two names or spaces (e.g white whale) I got an error(jinja2 synatx) how do I solve this error?
#app.route("/book", methods["GET", "POST"])
def get_books():
api_key =
os.environ.get("API_KEY")
if request.method == "POST":
book = request.form.get("book")
url =f"https://www.googleapis.com/books/v1/volumes?q={book}:keyes&key={api_key}"
response =urllib.request.urlopen(url)
data = response.read()
jsondata = json.loads(data)
return render_template ("book.html", books=jsondata["items"]
I tried to search for similar cases, and just found one solution, but I didn't understand it
Here is my error message
http.client.InvalidURL
http.client.InvalidURL: URL can't contain control characters. '/books/v1/volumes?q=white whale:keyes&key=AIzaSyDtjvhKOniHFwkIcz7-720bgtnubagFxS8' (found at least ' ')

Some chars in url need to be encoded - in your situation you have to use + or %20 instead of space.
This url has %20 instead of space and it works for me. If I use + then it also works
import urllib.request
import json
url = 'https://www.googleapis.com/books/v1/volumes?q=white%20whale:keyes&key=AIzaSyDtjvhKOniHFwkIcz7-720bgtnubagFxS8'
#url = 'https://www.googleapis.com/books/v1/volumes?q=white+whale:keyes&key=AIzaSyDtjvhKOniHFwkIcz7-720bgtnubagFxS8'
response = urllib.request.urlopen(url)
text = response.read()
data = json.loads(text)
print(data)
With requests you don't even have to do it manually because it does it automatically
import requests
url = 'https://www.googleapis.com/books/v1/volumes?q=white whale:keyes&key=AIzaSyDtjvhKOniHFwkIcz7-720bgtnubagFxS8'
r = requests.get(url)
data = r.json()
print(data)
You may use urllib.parse.urlencode() to make sure all chars are correctly encoded.
import urllib.request
import json
payload = {
'q': 'white whale:keyes',
'key': 'AIzaSyDtjvhKOniHFwkIcz7-720bgtnubagFxS8',
}
query = urllib.parse.urlencode(payload)
url = 'https://www.googleapis.com/books/v1/volumes?' + query
response = urllib.request.urlopen(url)
text = response.read()
data = json.loads(text)
print(data)
and the same with requests - it also doesn't need encoding
import requests
payload = {
'q': 'white whale:keyes',
'key': 'AIzaSyDtjvhKOniHFwkIcz7-720bgtnubagFxS8',
}
url = 'https://www.googleapis.com/books/v1/volumes'
r = requests.get(url, params=payload)
data = r.json()
print(data)

Api call using python and token_auth

"""
#Collects basic metrics from Matomo installation and returns a pandas dataframe
"""
token = os.getenv("token")
# Build url string
base_url = 'https://matomo.___.com/index.php?module=API'
site_num = '&idSite=1'
return_format = '&format=json'
period = '&period=day'
date_range = '&date=last30'
method = '&method=VisitsSummary.get'
token_string = "&token_auth=" + token
my_url = base_url + site_num + return_format + period + date_range + method + token_string
# send request for report
r = requests.get(my_url)
# parse and tidy collected data
data = pd.DataFrame(r.json()).T
data = data.reset_index()
data.columns = [
"date",
"uniq_visitors",
"users",
"visits",
"actions",
"visits_converted",
"bounces",
"sum_visit_length",
"max_actions",
"bounce_rate",
"actions_per_visit",
"avg_time_on_site",
]
return data
I am trying to get data from the matomo API using an auth_token and parameters by using above code but i am not able to access it and my url is not taking token code any one has idea how i can solve this

Given that you are using the request library, passing parameters and headers can be done using the following params in your get call:
r = requests.get(my_url, params=payload)
In the same way, an auth token is usually passed within headers:
r = requests.get(my_url, params=payload, headers=headers)
Using this format you can simply create a headers object which contains your token_auth and directly pass your parameters in a payload object:
headers = {'token_auth': token}
payload = {'module':'API', 'idSite':1, 'format':'json', 'period':'day', 'date':'last30', 'method':'VisitsSummary.get'}
Since you are now passing your parameters in you get request, there is no need to add them to the end of your url. Thus, your url should stay as https://matomo.___.com/index.php. These can then be used within your params and headers respectively. Please note that this assumes that the matomo API places the token_auth in its headers such as most APIs do. If this is not the case you could pass it directly within the params payload.
Here is a global overview:
token = os.getenv("token")
# Get url, headers and params
my_url = 'https://matomo.___.com/index.php'
payload = {'module':'API', 'idSite':1, 'format':'json', 'period':'day', 'date':'last30', 'method':'VisitsSummary.get'}
headers = {'token_auth': token}
# send request for report
r = requests.get(my_url, params=payload, headers=headers)
Note this answers your question specifically regarding the API call and not the processing after.

Formated output

I'm making a request in restapi to show application version, but the output i got is not the expected, i want to format this data.
from requests.api import request
from requests.packages.urllib3.exceptions import InsecureRequestWarning
import re
import requests
requests.packages.urllib3.disable_warnings(InsecureRequestWarning)
bipTmb='https://api1-foo'
bipAws='https://api2-foo'
def requestGet(bipTmb, bipAws):
cont = []
for urls in [bipTmb, bipAws]:
url = urls + '/mgmt/tm/sys?$top=4'
headers = {
'accept': '*/*',
'Content-Type': 'application/json',
}
response = requests.get(url, headers=headers, verify=False, auth=('auth', 'pass'))
data = response.json()
data = data['items']
reference = data[0]
version = reference['reference']
find = re.search("ver=.*",format(version))
content = (urls, find.group())
cont.append(content)
return cont
cont = requestGet(bipTmb, bipAws)
for item in cont:
treated_data = (item)
print(treated_data)
output:
[('https://api1-foo', "ver=13.1.3.6'}"), ('https://api2-foo', "ver=13.1.3.6'}")]
Formated output expected:
https://api1-foo ver=13.1.3.6,
https://api2-foo ver=13.1.3.6
How can i transform this data?

Try this.
print "\n".join(treated_data)

I reach the expected output looping through data.
for i in range(len(cont)):
print(cont[i])

Url unshortner python and urilib Request

Im traying to unshort a few urls that have been shortened. For such thing I'm using https://pypi.org/project/fake-headers/ to create the headers
but i cant get it to work I get 400 and 406 status codes.
url = 'http:// bit.ly /377n4o6' #i had to add spaces end url is https://es.banggood.com/Xiaomi-Mi-A1-MiA1-Dual-Rear-Camera-5_5-inch-4GB-RAM-64GB-Snapdragon-625-Octa-core-4G-Smartphone-p-1196064.html
parsed = urllib.parse.urlparse(url)
if parsed.scheme == '':
url = 'http://' + url
parsed = urllib.parse.urlparse(url)
h = http.client.HTTPConnection(parsed.netloc, timeout=5)
path = quote(parsed.path, safe='')
header = Headers(
browser="chrome", # Generate only Chrome UA
os="win", # Generate ony Windows platform
headers=True # generate misc headers
)
#request is asking to have a parans, so I got one from another code.
params = urlencode({'#number': 12524, '#type': 'issue', '#action': 'show'})
h.request('HEAD', path,params, header.generate())
response = h.getresponse()
print(response.status)
what could be wrong with this that fails?
thanks

Trying to call the AlchemyLanguage API

I have written code for calling the AlchemyLanguage API of Bluemix in Python. I need the keywords and entities, but it is only showing the first keyword and first entity for the text file. Where am I going wrong?
import requests
import urllib
import urllib2
def call_alchemy_api(text, API_KEY):
payload = {'outputMode':'json','extract':'entities,keywords','sentiment':'1','maxRetrieve':'1', 'url':'https://www.ibm.com/us-en/'}
payload['apikey'] = API_KEY
encoded_text = urllib.quote_plus(text)
payload['text'] = text
data = urllib.urlencode(payload)
url = 'https://gateway-a.watsonplatform.net/calls/text/TextGetCombinedData'
req = urllib2.Request(url, data)
response = urllib2.urlopen(req)
return response
if __name__ == "__main__":
api_key = 'xxxxxxxxxxxxxxxxxxxxxmyapi'
f = open('in0.txt','r')
text = f.read()
print text
response = call_alchemy_api(text, api_key)
print response.read()

Change the maxRetrieve keyword's value.
Example:
payload = {'outputMode':'json','extract':'entities,keywords','sentiment':'1','maxRetrieve':'3', 'url':'https://www.ibm.com/us-en/'}
API Link:
http://www.ibm.com/watson/developercloud/alchemy-language/api/v1/