I am trying to fetch a JSON response of multiple issues from an API and I am able to get the response successfully. My next part which I want to perform is to fetch/print only those lines which have specific keywords as "moviepass" and "login" in JSON tag "body". Here is my code
import json
import requests
api_url = '***************************************'
headers = {'Content-Type': 'application/json',
'Authorization':'Basic **************************'}
response = requests.get(api_url, headers=headers)
#print(response.text)
words = ('moviepass', 'login')
def lookingfor(words):
data = response.text
for line in data:
for word in words:
match = re.findall(word, line['body'])
if match:
print((word, line[]))
lookingfor(words)
My JSON looks like:
[{"tags":["moviepass"],"assignee_name":null,"app_id":"*******","hs_user_id":"*******","title":"1234","redacted":false,"updated_at":1611753805497,"messages":[{"body":"moviepass - Not '
'sure if this is what you guys meant or not but here '
'haha.","created_at":********,"author":{"name":"abc","id":"*****","emails":["abc#qwerty.com"]},"origin":"end-user","id":"*********"}]
You dont need regular expression.You can use json_data['tags']
But if you want to use regular expression, you need to convert json to string by using
import json
json.dumps(json_obj) #returns same object but type of string.
Convert JSON response and parse it - it's a list of [nested] dicts. You can use Response.json() method, no need to import json.
import requests
api_url = '***************************************'
headers = {'Content-Type': 'application/json',
'Authorization':'Basic **************************'}
words = ('moviepass', 'login')
response = requests.get(api_url, headers=headers)
data = response.json()
for item in data:
if any(word in item.get('tags', []) for word in words):
print(item)
Related
When i search for books with a single name(e.g bluets) my code works fine, but when I search for books that have two names or spaces (e.g white whale) I got an error(jinja2 synatx) how do I solve this error?
#app.route("/book", methods["GET", "POST"])
def get_books():
api_key =
os.environ.get("API_KEY")
if request.method == "POST":
book = request.form.get("book")
url =f"https://www.googleapis.com/books/v1/volumes?q={book}:keyes&key={api_key}"
response =urllib.request.urlopen(url)
data = response.read()
jsondata = json.loads(data)
return render_template ("book.html", books=jsondata["items"]
I tried to search for similar cases, and just found one solution, but I didn't understand it
Here is my error message
http.client.InvalidURL
http.client.InvalidURL: URL can't contain control characters. '/books/v1/volumes?q=white whale:keyes&key=AIzaSyDtjvhKOniHFwkIcz7-720bgtnubagFxS8' (found at least ' ')
Some chars in url need to be encoded - in your situation you have to use + or %20 instead of space.
This url has %20 instead of space and it works for me. If I use + then it also works
import urllib.request
import json
url = 'https://www.googleapis.com/books/v1/volumes?q=white%20whale:keyes&key=AIzaSyDtjvhKOniHFwkIcz7-720bgtnubagFxS8'
#url = 'https://www.googleapis.com/books/v1/volumes?q=white+whale:keyes&key=AIzaSyDtjvhKOniHFwkIcz7-720bgtnubagFxS8'
response = urllib.request.urlopen(url)
text = response.read()
data = json.loads(text)
print(data)
With requests you don't even have to do it manually because it does it automatically
import requests
url = 'https://www.googleapis.com/books/v1/volumes?q=white whale:keyes&key=AIzaSyDtjvhKOniHFwkIcz7-720bgtnubagFxS8'
r = requests.get(url)
data = r.json()
print(data)
You may use urllib.parse.urlencode() to make sure all chars are correctly encoded.
import urllib.request
import json
payload = {
'q': 'white whale:keyes',
'key': 'AIzaSyDtjvhKOniHFwkIcz7-720bgtnubagFxS8',
}
query = urllib.parse.urlencode(payload)
url = 'https://www.googleapis.com/books/v1/volumes?' + query
response = urllib.request.urlopen(url)
text = response.read()
data = json.loads(text)
print(data)
and the same with requests - it also doesn't need encoding
import requests
payload = {
'q': 'white whale:keyes',
'key': 'AIzaSyDtjvhKOniHFwkIcz7-720bgtnubagFxS8',
}
url = 'https://www.googleapis.com/books/v1/volumes'
r = requests.get(url, params=payload)
data = r.json()
print(data)
I am trying to get a certain value in a string of json but I can't figure out how exactly to do it. I don't want to convert it into a string and strip / replace the unwanted pieces because then I won't be able to get the other values. My current code is:
username = "Dextication"
url = f"https://minecraft-statistic.net/api/player/info/{username}/"
response = requests.get(url)
json_data = json.loads(response.text)
print(json_data)
Edit:
when I run this, json.data = "{"status":"ok","data":{"online":0,"total_time_play":46990,"last_play":1513960562,"license":1,"name":"Dextication","uuid":"74d57a754855410c90b3d51bc99b8beb"}}"
I would like to only print the value: 46990
Try below code
import json, requests
username = "Dextication"
url = f"https://minecraft-statistic.net/api/player/info/{username}/"
response = requests.get(url)
json_data = json.loads(response.text)
result = json_data['data']['total_time_play']
print (result)
I'm a noob and I need to use the sessionid to post other commands like search.do, Im using Python 3.5 but Im not sure the best way to get and post it.
here is how I posted the request.
import urllib.parse
url = 'https://myapi.application.com/dmapi/login.do'
values = {'account' : 'MYACCOUNT', 'username': 'admin', 'password': 'pas1234', 'appid':'12346'}
data = urllib.parse.urlencode(values)
data = data.encode('utf-8') # data should be bytes
req = urllib.request.Request(url, data)
resp = urllib.request.urlopen(req)
respData = resp.read()
print(respData)
printing gets this result.
b'errorcode=0\r\nsessionid=ef9a9cbd-e063-4be2-9301-9de59891304c\r\n'
I need to use the sessionid in subsequent request. Whats the best way to go about this.
In fact the response in composed of lines (in bytes) one of which contains the session id. You could simply read and parse what you get:
resp = urllib.request.urlopen(req)
errorcode = None
sessionid = None
for line in resp.read():
line = line.strip() # remove end of line
if line.startswith(b'errorcode'):
errorcode = line.split(b'=')[1]
if line.startswith(b'sessionid'):
sessionid = line.split(b'=')[1]
One idea is to split by sessionid= and extract the last item:
>>> respData.split("sessionid=")[-1].strip()
'ef9a9cbd-e063-4be2-9301-9de59891304c'
Another, is to use a regular expression:
>>> import re
>>>
>>> re.search(r"sessionid=([A-Za-z0-9-]+)", respData).group(1)
'ef9a9cbd-e063-4be2-9301-9de59891304c'
How do I send the ASCII encoded text via POST request in Python? The length of true_input I received via the POST is always different from the length I sent.
def insert_true_input(level, iteration, true_input):
url = master_url + "/insert_true_input?"
data = {'level': level, 'iteration': iteration, 'true_input': true_input}
headers = {'Content-Type': 'text/plain'}
res = requests.post(url, params=data, headers=headers).text
return res
The sample true_input that I want to send is directly from numpy.ndarray.tostring() and looks like
'\x00\x00\x00\x00\x00\x00\xf0?\x00\x00\x00\x00\x00\x00\x00#\x00\x00\x00\x00\x00\x00\x08#\x00\x00\x00\x00\x00\x00\xf0?\x00\x00\x00\x00\x00\x007#\x00\x00\x00\x00\x00\xc0^#\x00\x00\x00\x00\x00\xc0^#\x00\x00\x00\x00\x00\xc0^#\x00\x00\x00\x00\x00\x00(#\x00\x00\x00\x00\x00\x00?#'
As explained in the comments, the null characters \x00 are not sendable in raw text. You have to encode them one way or another (URL encoded, Base64, json, etc.). But then the other side that will receive the request must be adapted to decode them accordingly.
Actually requests will use URL encoding automatically for the parameters passed in the query string, but I suspect that your java code is not able to decode them properly.
Please post your Java code for the receiving side to see what we can do.
Suggestions on python side, using base64:
import base64
def insert_true_input(level, iteration, true_input):
url = master_url + "/insert_true_input?"
data = {'level': level, 'iteration': iteration, 'true_input': base64.b64encode(true_input)}
res = requests.post(url, params=data, headers=headers).text
return res
Using json (requests will do the work for you if you use the json parameter to .post()):
def insert_true_input(level, iteration, true_input):
url = master_url + "/insert_true_input?"
data = {'level': level, 'iteration': iteration, 'true_input': true_input}
res = requests.post(url, json=data, headers=headers).text
return res
You have to encode your string using str.encode('ascii'):
def insert_true_input(level, iteration, true_input):
url = master_url + "/insert_true_input?"
data = {'level': level, 'iteration': iteration, 'true_input': true_input.encode('ascii')}
headers = {'Content-Type': 'text/plain'}
res = requests.post(url, params=data, headers=headers).text
return res
I'm trying to send json as a parameter thru a get method for an api, I found that the url to which it is hitting is little bit different from the original url. Some ":%20" text is inserted in between the url. Not sure why this difference is coming, Can someone help
Original URL: http://258.198.39.215:8280/areas/0.1/get/raj/name?jsonRequest=%7B%22rajNames%22%3A%5B%22WAR%22%5D%7D
My URL : http://258.198.39.215:8280/areas/0.1/get/raj/name?jsonRequest=&%7B%22rajNames%22:%20%22WAR%22%7D
Python code:
headers = {'Accept': 'application/json','Authorization': 'Bearer '+access_token}
json = {'rajNames':'WAR'}
url = 'http://258.198.39.215:8280/areas/0.1/get/raj/name?jsonRequest='
r = requests.get(url, params=json.dumps(json),headers=headers)
print _r.url
The spaces are not the problem; your method of generating the query string is, as is your actual JSON payload.
Note that your original URL has a different JSON structure:
>>> from urllib import unquote
>>> unquote('%7B%22rajNames%22%3A%5B%22WAR%22%5D%7D')
'{"rajNames":["WAR"]}'
The rajNames parameter is a list, not a single string.
Next, requests sees all data in params as a new parameter, so it used & to delimit from the previous parameter. Use a dictionary and leave the ?jsonRequest= part to requests to generate:
headers = {'Accept': 'application/json', 'Authorization': 'Bearer '+access_token}
json_data = {'rajNames': ['WAR']}
params = {'jsonRequest': json.dumps(json_data)}
url = 'http://258.198.39.215:8280/areas/0.1/get/raj/name'
r = requests.get(url, params=params, headers=headers)
print _r.url
Demo:
>>> import requests
>>> import json
>>> headers = {'Accept': 'application/json', 'Authorization': 'Bearer <access_token>'}
>>> json_data = {'rajNames': ['WAR']}
>>> params = {'jsonRequest': json.dumps(json_data)}
>>> url = 'http://258.198.39.215:8280/areas/0.1/get/raj/name'
>>> requests.Request('GET', url, params=params, headers=headers).prepare().url
'http://258.198.39.215:8280/areas/0.1/get/raj/name?jsonRequest=%7B%22rajNames%22%3A+%5B%22WAR%22%5D%7D'
You can still eliminate the spaces used in the JSON output from json.dumps() by setting the separators argument to (',', ':'):
>>> json.dumps(json_data)
'{"rajNames": ["WAR"]}'
>>> json.dumps(json_data, separators=(',', ':'))
'{"rajNames":["WAR"]}'
but I doubt that is really needed.