Develop Reference

Python - Typings for http request call results

I'm interested in writing clean, readable and maintainable code. Some time we have to interact with external API. For example httpbin.org. For me it would be a good idea to somehow fix request call result structure in code. For example httpbin.org/get returns:
{
"args": {},
"headers": {
"Accept": "application/json",
"Accept-Encoding": "gzip, deflate, br",
"Accept-Language": "ru,en;q=0.9",
"Host": "httpbin.org",
"Referer": "https://httpbin.org/",
"Sec-Ch-Ua": "\" Not A;Brand\";v=\"99\", \"Chromium\";v=\"102\", \"Yandex\";v=\"22\"",
"Sec-Ch-Ua-Mobile": "?0",
"Sec-Ch-Ua-Platform": "\"macOS\"",
"Sec-Fetch-Dest": "empty",
"Sec-Fetch-Mode": "cors",
"Sec-Fetch-Site": "same-origin",
"User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.5005.167 YaBrowser/22.7.5.1018 Yowser/2.5 Safari/537.36",
"X-Amzn-Trace-Id": "Root=1-456c9715-53b4jhdsjhbjdh3dskjea0"
},
"origin": "134.51.34.46",
"url": "https://httpbin.org/get"
}
Now I see the way to convert the json into NamedTuple/dataclass/Pydantic.
I wonder to know if there is some common approaches or maybe ready solutions/libs or so for this? Also I need this for accessing through dot.

Unable to send a file with a correct POST request to the website using Python

I'm trying to submit a file (test.exe) to a website using a POST request, but instead of a normal 302 response, it keeps responding with 500. I don't know what I could change in my request: maybe in the headers or in the files format, or maybe I need to somehow pass the data parameter?
I would appreciate any advice on this!
import requests
url = "https://cuckoo.cert.ee/submit/api/presubmit"
files = {"test.exe": open("test.exe", "rb")}
headers = {
"Accept": "*/*",
"Accept-Encoding": "gzip, deflate, br",
"Accept-Language": "en-US,en;q=0.9",
"Connection": "keep-alive",
"Content-Length": "199",
"Content-Type": "multipart/form-data; boundary=----WebKitFormBoundarymoUA16cLBrh9JNGC",
"Cookie": "csrftoken=O9tFpNhZuZrj7DsEnBAcj0wmV00z8qE3; theme=cyborg; csrftoken=O9tFpNhZuZrj7DsEnBAcj0wmV00z8qE3",
"Host": "cuckoo.cert.ee",
"Origin": "https://cuckoo.cert.ee",
"Referer": "https://cuckoo.cert.ee/submit/",
"sec-ch-ua-mobile": "?0",
"sec-ch-ua-platform": "Windows",
"Sec-Fetch-Dest": "empty",
"Sec-Fetch-Mode": "cors",
"Sec-Fetch-Site": "same-origin",
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.0.0 Safari/537.36",
"X-CSRFToken": "O9tFpNhZuZrj7DsEnBAcj0wmV00z8qE3"
}
response = requests.post(url, headers=headers, files=files, verify=False)
print(response)

Possibly try changing content type to application/octet-stream.
500 Error indicates that the website may not be able to handle the file that you are trying to upload. It could simply be that the website is malfunctioning or having a temporary failure.
If you have access to the back-end logs, I would recommend looking at that, or contacting the website to see if they have any suggestions.
EDIT:
Verify that your content matches the length that you are declaring as well, it looks like you have a content-length parameter declared in your request. Try taking that out to see if that helps.

Cannot login with requests post

import requests
session = requests.Session()
data = "memberNo=&url=&cyberId={}&cyberPass={}"
headers = {
"Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9",
"Accept-Encoding": "gzip, deflate, br",
"Accept-Language": "ko-KR,ko;q=0.9,en-US;q=0.8,en;q=0.7",
"Cache-Control": "max-age=0",
"Connection": "keep-alive",
"Content-Length": "57",
"Content-Type": "application/x-www-form-urlencoded",
"Host": "www.sonofelicecc.com",
"Origin": "https://www.sonofelicecc.com",
"Referer": "https://www.sonofelicecc.com/login.dp/dmparse.dm",
"sec-ch-ua": '" Not A;Brand";v="99", "Chromium";v="99", "Google Chrome";v="99"',
"sec-ch-ua-mobile": "?0",
"sec-ch-ua-platform": "Windows",
"Sec-Fetch-Dest": "document",
"Sec-Fetch-Mode": "navigate",
"Sec-Fetch-Site": "same-origin",
"Sec-Fetch-User": "?1",
"Upgrade-Insecure-Requests": "1",
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.4844.82 Safari/537.36",
}
main_headers = {
"Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9",
"Accept-Encoding": "gzip, deflate, br",
"Accept-Language": "ko-KR,ko;q=0.9,en-US;q=0.8,en;q=0.7",
"Cache-Control": "max-age=0",
"Connection": "keep-alive",
"Host": "www.sonofelicecc.com",
"Referer": "https://www.sonofelicecc.com/login.dp/dmparse.dm",
"sec-ch-ua": '" Not A;Brand";v="99", "Chromium";v="99", "Google Chrome";v="99"',
"sec-ch-ua-mobile": "?0",
"sec-ch-ua-platform": "Windows",
"Sec-Fetch-Dest": "document",
"Sec-Fetch-Mode": "navigate",
"Sec-Fetch-Site": "same-origin",
"Sec-Fetch-User": "?1",
"Upgrade-Insecure-Requests": "1",
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.4844.82 Safari/537.36",
}
login_req = session.post("https://www.sonofelicecc.com/doLogin.dp/dmparse.dm", headers=headers, data=data)
main_resp = session.get("https://www.sonofelicecc.com/main.dp/dmparse.dm", headers=main_headers)
if main_resp.text.find(username) != -1:
print("login success")
I am trying to login with requests without selenium. I've first tried login with selenium; however it was too slow; thus, I saved the selenium's cookie and return it to requests for faster work.
Then, I want to make all working with only requests. I've loaded all the headers correctly, but it doesn't work at all. How can I?

Sneaker bot checkout process

hi every one im newbie in programing.and i need help to work my code i need to post this in formation to this web site https://live.adyen.com/hpp/completeCard.shtml.but i get error pls help...
import requests
headers = {"Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image /apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9",
"Accept-Encoding": "gzip, deflate, br",
"Accept-Language": "en-US,en;q=0.9",
"Cache-Control": "max-age=0",
"Connection": "keep-alive",
"Content-Length": "5185",
"Content-Type": "application/x-www-form-urlencoded",
"Host": "live.adyen.com",
"Origin": "https://live.adyen.com",
"Referer": "https://live.adyen.com/hpp/pay.shtml",
"sec-ch-ua-mobile": "?0",
"Sec-Fetch-Dest": "document",
"Sec-Fetch-Mode": "navigate",
"Sec-Fetch-Site": "same-origin",
"Sec-Fetch-User": "?1",
"Upgrade-Insecure-Requests": "1",
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.114 Safari/537.36"}
payload = {"card.cardNumber": "4001 0200 0000 0009",
"card.cardHolderName": "Juan Tamad",
"card.expiryMonth": "03",
"card.expiryYear": "2030",
"card.cvcCode": "737"}
r = requests.post("https://live.adyen.com/hpp/completeCard.shtml/post", headers=headers, data=payload)
print(r.text)
i got this error
There was a problem displaying this page. Please try again later, or verify your input on the previous page

How can I get this POST request to execute in Python using Google Colab. I am currently getting a 400 response code

I am trying to extract information for all the jobs on this website:
https://www.americanmobile.com/travel-nursing-jobs/search/
Looking at the network activity tab, it looks like all the data I need comes from a POST request made here:
https://jobs.amnhealthcare.com/api/jobs//search. I have attached an image that may help confirm exactly what I am referencing.
example_1
I wrote the following code in Google Colab to try to at least get the first 10 results. Referencing python requests POST with header and parameters, I know a lot of the headers may not even be necessary. I have tried sending this request without any headers at all.
Is what I'm trying to do even possible? I have only gotten a 400 response code so far.
If it is possible to accomplish this, is it possible to extract this information for all 4k + jobs?
import requests
import re
payload = {
"PageNumber": "1",
"JobsPerPage": "10",
"Filters": {
"Locations": "[]",
"Skillset": "[]",
"StartDates": "[]",
"Shifts": "[]",
"Durations": "[]",
"IsCovid19Job": "false",
"Exclusive": "false",
"Skillsets": "[]",
"DivisionCompanyId": "2",
"LocationSearch": "",
"KeywordSearch": "",
"DaxtraJobIds": "[]"
},
"SortOrder": {
"Header": "MaxPayRate",
"SortDirection": "Descending"
}
}
headers = {
"Host": "jobs.amnhealthcare.com",
"Connection": "keep-alive",
"Content-Length": "315",
"Accept": "application/json, text/plain, */*",
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.90 Safari/537.36",
"Content-Type": "application/json;charset=UTF-8",
"Origin": "https://www.americanmobile.com",
"Sec-Fetch-Site": "cross-site",
"Sec-Fetch-Mode": "cors",
"Sec-Fetch-Dest": "empty",
"Referer": "https://www.americanmobile.com/",
"Accept-Encoding": "gzip, deflate, br",
"Accept-Language": "en-US,en;q=0.9"
}
response = requests.post('https://jobs.amnhealthcare.com/api/jobs//search' , data=p , headers=headers)
Thank you

The formatting of your data wasn't entirely correct. This should work:
import requests
headers = {
"Host": "jobs.amnhealthcare.com",
"Connection": "keep-alive",
"Content-Length": "315",
"Accept": "application/json, text/plain, */*",
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.90 Safari/537.36",
"Content-Type": "application/json;charset=UTF-8",
"Origin": "https://www.americanmobile.com",
"Sec-Fetch-Site": "cross-site",
"Sec-Fetch-Mode": "cors",
"Sec-Fetch-Dest": "empty",
"Referer": "https://www.americanmobile.com/",
"Accept-Encoding": "gzip, deflate, br",
"Accept-Language": "en-US,en;q=0.9"
}
data = '{"PageNumber":1,"JobsPerPage":10,"Filters":{"Locations":[],"Skillset":[],"StartDates":[],"Shifts":[],"Durations":[],"IsCovid19Job":false,"Exclusive":false,"Skillsets":[],"DivisionCompanyId":2,"LocationSearch":"","KeywordSearch":"","DaxtraJobIds":[]},"SortOrder":{"Header":"MaxPayRate","SortDirection":"Descending"}}'
response = requests.post('https://jobs.amnhealthcare.com/api/jobs//search', headers=headers, data=data)
You can now adapt JobsPerPage and PageNumber to retrieve all the posts you need in a for-loop.