Develop Reference

How to search for books that have spaces in their title using Google books API

When i search for books with a single name(e.g bluets) my code works fine, but when I search for books that have two names or spaces (e.g white whale) I got an error(jinja2 synatx) how do I solve this error?
#app.route("/book", methods["GET", "POST"])
def get_books():
api_key =
os.environ.get("API_KEY")
if request.method == "POST":
book = request.form.get("book")
url =f"https://www.googleapis.com/books/v1/volumes?q={book}:keyes&key={api_key}"
response =urllib.request.urlopen(url)
data = response.read()
jsondata = json.loads(data)
return render_template ("book.html", books=jsondata["items"]
I tried to search for similar cases, and just found one solution, but I didn't understand it
Here is my error message
http.client.InvalidURL
http.client.InvalidURL: URL can't contain control characters. '/books/v1/volumes?q=white whale:keyes&key=AIzaSyDtjvhKOniHFwkIcz7-720bgtnubagFxS8' (found at least ' ')

Some chars in url need to be encoded - in your situation you have to use + or %20 instead of space.
This url has %20 instead of space and it works for me. If I use + then it also works
import urllib.request
import json
url = 'https://www.googleapis.com/books/v1/volumes?q=white%20whale:keyes&key=AIzaSyDtjvhKOniHFwkIcz7-720bgtnubagFxS8'
#url = 'https://www.googleapis.com/books/v1/volumes?q=white+whale:keyes&key=AIzaSyDtjvhKOniHFwkIcz7-720bgtnubagFxS8'
response = urllib.request.urlopen(url)
text = response.read()
data = json.loads(text)
print(data)
With requests you don't even have to do it manually because it does it automatically
import requests
url = 'https://www.googleapis.com/books/v1/volumes?q=white whale:keyes&key=AIzaSyDtjvhKOniHFwkIcz7-720bgtnubagFxS8'
r = requests.get(url)
data = r.json()
print(data)
You may use urllib.parse.urlencode() to make sure all chars are correctly encoded.
import urllib.request
import json
payload = {
'q': 'white whale:keyes',
'key': 'AIzaSyDtjvhKOniHFwkIcz7-720bgtnubagFxS8',
}
query = urllib.parse.urlencode(payload)
url = 'https://www.googleapis.com/books/v1/volumes?' + query
response = urllib.request.urlopen(url)
text = response.read()
data = json.loads(text)
print(data)
and the same with requests - it also doesn't need encoding
import requests
payload = {
'q': 'white whale:keyes',
'key': 'AIzaSyDtjvhKOniHFwkIcz7-720bgtnubagFxS8',
}
url = 'https://www.googleapis.com/books/v1/volumes'
r = requests.get(url, params=payload)
data = r.json()
print(data)

Roblox Purchasing an item from catalog

I have written a script that should purchase an asset from catalog.
import re
from requests import post, get
cookie = "blablabla"
ID = 1562150
# getting x-csrf-token
token = post("https://auth.roblox.com/v2/logout", cookies={".ROBLOSECURITY": cookie}).headers['X-CSRF-TOKEN']
print(token)
# getting item details
detail_res = get(f"https://www.roblox.com/library/{ID}")
text = detail_res.text
productId = int(get(f"https://api.roblox.com/marketplace/productinfo?assetId={ID}").json()["ProductId"])
expectedPrice = int(re.search("data-expected-price=\"(\d+)\"", text).group(1))
expectedSellerId = int(re.search("data-expected-seller-id=\"(\d+)\"", text).group(1))
headers = {
"x-csrf-token": token,
"content-type": "application/json; charset=UTF-8"
}
data = {
"expectedCurrency": 1,
"expectedPrice": expectedPrice,
"expectedSellerId": expectedSellerId
}
buyres = post(f"https://economy.roblox.com/v1/purchases/products/{productId}", headers=headers,
data=data,
cookies={".ROBLOSECURITY": cookie})
if buyres.status_code == 200:
print("Successfully bought item")
The problem is that it somehow doesn't purchase any item with error 500 (InternalServerError).
Someone told me that if I add json.dumps() to the script it might work.
How to add json.dumps() here (I don't understand it though I read docs) and how to fix this so the script purchases item?
Big thanks to anyone who can help me.

Import the json package.
json.dumps() converts a python dictionary to a json string.
I'm guessing this is what you want.
buyres =
post(f"https://economy.roblox.com/v1/purchases/products/{productId}",
headers=json.dumps(headers),
data=json.dumps(data),
cookies={".ROBLOSECURITY": cookie})

I found the answer finally, I had to do it like this:
dataLoad = json.dumps(data)
buyres = post(f"https://economy.roblox.com/v1/purchases/products/{productId}", headers=headers,
data=dataLoad,
cookies={".ROBLOSECURITY": cookie})

python api automation / getting status code = 500 and {Message ":"An error has occured."}

import requests
import json
import jsonpath
def test_add_new_data():
# url = "http://thetestingworldapi.com/api/studentsDetails"
# f = open('/Users/sunghunkwak/PycharmProjects/apiTesting/postStudent.json', 'r')
# requests_post_id = json.loads(f.read())
# result = requests.post(url, requests_post_id)
# assert result.status_code == 201
# id = jsonpath.jsonpath(result.json(), 'id')
# print(id[0])
url_tech = "http://thetestingworldapi.com/api/technicalskills"
f = open('/Users/sunghunkwak/PycharmProjects/apiTesting/postTechskills.json', 'r')
requests_post_tech = json.loads(f.read())
result = requests.post(url_tech, requests_post_tech)
assert result.status_code == 200
print(result.text)
url_addr = "http://thetestingworldapi.com/api/addresses"
f = open('/Users/sunghunkwak/PycharmProjects/apiTesting/postAddress.json', 'r')
requests_post_addr = json.loads(f.read())
result = requests.post(url_addr, requests_post_addr)
print(result.status_code)
# assert result.status_code == 200
url_final = "http://thetestingworldapi.com/api/FinalStudentDetails/189969"
requests_get = requests.get(url_final)
print(requests_get.text)
I tried to test, but it keeps getting errors like below.
test.py {"status":"true","msg":"Add data success"}
500
{"Message":"An error has occurred."}
post address and get final data are getting errors.
How can I solve it??
Thank you.

Are you sure your JSON files are correct that you're trying to post? Here is an example post request:
import requests
url = 'https://reqres.in/api/users'
payload = {
"name": "foo",
"job": "bar"
}
x = requests.post(url, data=payload)
print(x.text)
When I tried to change the "URL" the URL to "http://thetestingworldapi.com/api/addresses" also throws the same error {"Message":"An error has occurred."}
Here is an another example I read it from JSON file and requested:
import requests
import os
import json
url = 'https://reqres.in/api/users'
filename = "foo.json"
folder_ = os.path.dirname(os.path.abspath(__file__))
absolute_filename = os.path.join(folder_, filename)
f = open(absolute_filename, 'r')
payload = json.loads(f.read())
x = requests.post(url, payload)
print(x.text)
Looks like you have an error in server side.
Also I suggest you read HTTP response status codes:
https://developer.mozilla.org/en-US/docs/Web/HTTP/Status

How can I make this API work with a payload?

I am using this API to list users. One of the parameters I could specify is a team id which is placed in an array. When I try to specify a team id it doesn't work when I put it in the payload, but it works when I change the url to include the team id.
This is the API reference: https://api-reference.pagerduty.com/#!/Users/get_users
Here is what I am basing my code off of: https://github.com/PagerDuty/API_Python_Examples/blob/master/REST_API_v2/Users/list_users.py
This is my code when I try to specify team id in the payload. It doesn't work like this for some reason, but it works when I change the url to url = 'https://api.pagerduty.com/users?team_ids%5B%5D=TEAMID&team_ids%5B%5D=' where in TEAMID I have an actual team id.
with open('config/config.json') as f:
config = json.load(f)
API_KEY = config['API_KEY']
TEAM_IDS = ['TEAMID']
def list_users():
url = 'https://api.pagerduty.com/users'
headers = {
'Accept': 'application/vnd.pagerduty+json;version=2',
'Authorization': 'Token token={token}'.format(token=API_KEY)
}
payload = {
'team_ids[]': TEAM_IDS
}
r = requests.get(url, headers=headers)
result = []
if r.status_code == 200:
# loops for each user and retrieves their email
result = [user['email'] for user in r.json()['users']]
return result
else:
return None
I want to get this work by listing team id's in the array and sending it in the payload so that I can list more than one team id and not clutter them all in the url.

Looks like you just need something like this
payload = {
'team_ids[]': TEAM_IDS
}
r = requests.get(url, headers=headers, params=payload)

logging in to website using requests

I've tried two completely different methods. But still I can't get the data that is only present after loggin in.
I've tried doing one using requests but the xpath returns a null
import requests
from lxml import html
USERNAME = "xxx"
PASSWORD = "xxx"
LOGIN_URL = "http://www.reginaandrew.com/customer/account/loginPost/referer/aHR0cDovL3d3dy5yZWdpbmFhbmRyZXcuY29tLz9fX19TSUQ9VQ,,/"
URL = "http://www.reginaandrew.com/gold-leaf-glass-top-table"
def main():
FormKeyTxt = ""
session_requests = requests.session()
# Get login csrf token
result = session_requests.get(LOGIN_URL)
tree = html.fromstring(result.text)
# Create payload
formKey = str((tree.xpath("//*[ # id = 'login-form'] / input / # value")))
FormKeyTxt = "".join(formKey)
#print(FormKeyTxt.replace("['","").replace("']",""))
payload = {
"login[username]": USERNAME,
"login[password]": PASSWORD,
"form_key": FormKeyTxt,
"persistent_remember_me": "checked"
}
# Perform login
result = session_requests.post(LOGIN_URL, data=payload)
# Scrape url
result = session_requests.get(URL, data=payload)
tree = html.fromstring(result.content)
bucket_names = tree.xpath("//span[contains(#class, 'in-stock')]/text()")
print(bucket_names)
print(result)
print(result.status_code)
if __name__ == '__main__':
main()
ive tried another one using Mechanical soup but still it returns a null
import argparse
import mechanicalsoup
import urllib.request
from bs4 import BeautifulSoup
parser = argparse.ArgumentParser(description='Login to GitHub.')
parser.add_argument("username")
parser.add_argument("password")
args = parser.parse_args()
browser = mechanicalsoup.Browser()
login_page = browser.get("http://www.reginaandrew.com/gold-leaf-glass-top-table")
login_form = login_page.soup.select("#login-form")[0]
login_form.input({"login[username]": args.username, "login[password]": args.password})
page2 = browser.submit(login_form,login_page.url )
messages = page2.soup.find(class_='in-stock1')
if messages:
print(messages.text)
print(page2.soup.title.text)
I understand the top solution better so id like to do it using that but is there anything I'm missing? (I'm sure I'm missing a lot)

This should do it
import requests
import re
url = "http://www.reginaandrew.com/"
r = requests.session()
rs = r.get(url)
cut = re.search(r'<form.+?id="login-form".+?<\/form>', rs.text, re.S|re.I).group()
action = re.search(r'action="(.+?)"', cut).group(1)
form_key = re.search(r'name="form_key".+?value="(.+?)"', cut).group(1)
payload = {
"login[username]": "fugees",
"login[password]": "nugees",
"form_key": form_key,
"persistent_remember_me": "on"
}
rs = r.post(action, data=payload, headers={'Referer':url})