Develop Reference

404 Not Found when use request.urlretrieve

I get some response from external service. Then need to get url from this response and by this url download a file.
When i pass url to urlretrieve from response.text, urlretrieve return an Error.
But when i manually copy the url. Then set variable in python. url = 'https://My_service_site.com/9a57v4db5_2023-02-14.csv.gz'.
urlretrieve works fine and download the file to computer by this link.
response = requests.post(url, json=payload, headers=headers)
#method 1 - get error
url = response.text[17:-2] #get the link like 'https://my_provide_name.com/csv_exports/5704d5.csv.gz'
urlrtv = urllib.request.urlretrieve(url=url, filename='C:\\Users\\UserName\\Downloads\\test4.csv.gz')
>>return error: HTTP Error 404 Not Found
#method 2 - works fine
url2 = 'https://my_provide_name.com/csv_exports/5704d5.csv.gz'
urlrtv=urllib.request.urlretrieve(url=url2, filename='C:\\Users\\UserName\\Downloads\\test4.csv.gz')
>>works fine
When i copy url from method 1 and put in browser. It works fine.
Edit:
To be more precise i have tried to get url not like that response.text[17:-2]. Insted use json.loads to parse url from response. But still got the error
a = json.loads(response.text)
>>{'csv_file_url': 'https://service_name.com/csv_exports/746d6.csv.gz'}
url = a['csv_file_url']
print(url)
>>https://service_name.com/csv_exports/746d6.csv.gz
Solved: Just add time.sleep(3) before downloading file.
url = response.json()['csv_file_url']
time.sleep(3)
urlrtv = urllib.request.urlretrieve(url=url, filename=f'{storage_path}{filename}')

Python request resulting in blank response

I'm relatively new to Python so would like some help, I've created a script which simply use the request library and basic auth to connect to an API and returns the xml or Json result.
# Imports
import requests
from requests.auth import HTTPBasicAuth
# Set variables
url = "api"
apiuser = 'test'
apipass = 'testpass'
# CALL API
r = requests.get(url, auth=HTTPBasicAuth(apiuser, apipass))
# Print Statuscode
print(r.status_code)
# Print XML
xmlString = str(r.text)
print(xmlString)
if but it returns a blank string.
If I was to use a browser to call the api and enter the cretentials I get the following response.
<Response>
<status>SUCCESS</status>
<callId>99999903219032190321</callId>
<result xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="Dummy">
<authorFullName>jack jones</authorFullName>
<authorOrderNumber>1</authorOrderNumber>
</result>
</Response>
Can anyone tell me where I'm going wrong.

What API are you connecting to?
Try adding a user-agent to the header:
r = requests.get(url, auth=HTTPBasicAuth(apiuser, apipass), headers={'User-Agent':'test'})

Although this is not an exact answer for the OP, it may solve the issue for someone having a blank response from python-requests.
I was getting a blank response because of the wrong content type. I was expecting an HTML rather than a JSON or a login success. The correct content-type for me was application/x-www-form-urlencoded.
Essentially I had to do the following to make my script work.
data = 'arcDate=2021/01/05'
headers = {
'Content-Type': 'application/x-www-form-urlencoded',
}
r = requests.post('https://www.deccanherald.com/getarchive', data=data, headers=headers)
print(r.status_code)
print(r.text)
Learn more about this in application/x-www-form-urlencoded or multipart/form-data?

Run this and see what responses you get.
import requests
url = "https://google.com"
r = requests.get(url)
print(r.status_code)
print(r.json)
print(r.text)
When you start having to pass things in your GET, PUT, DELETE, OR POST requests, you will add it in the request.
url = "https://google.com"
headers = {'api key': 'blah92382377432432')
r = requests.get(url, headers=headers)
Then you should see the same type of responses. Long story short,
Print(r.text) to see the response, then you once you see the format of the response you get, you can move it around however you want.

I have an empty response only when the authentication failed or is denied.
The HTTP status is still ≤ 400.
However, in the header you can find :
'X-Seraph-LoginReason': 'AUTHENTICATED_FAILED'
or
'X-Seraph-LoginReason': 'AUTHENTICATED_DENIED'

If the request is empty, not even a status code I could suggest waiting some time between printing. Maybe the server is taking time to return the response to you.
import time
time.sleep(5)
Not the nicest thing, but it's worth trying
How can I make a time delay in Python?
I guess there are no errors during execution
EDIT: nvm, you mentioned that you got a status code, I thought you were literally geting nothing.
On the side, if you are using python3 you have to use Print(), it replaced Print

API Post Python

I am trying to make a post request within the Matchbook API.
I have logged in and I got below "Session- Tocken":
{"session-token":"xxxx_b0b8a6f22a82396b6afcfa344f3022","user-id":xx685,"role":"USER"}
However, I am not sure how to make the post request. See below code used:
headers = {"session-token" : "xxxx_b0b8a6f22a82396b6afcfa344f3022"}
r = requests.post('https://api.matchbook.com/edge/rest/reports/v1/offers/current/?odds-type=DECIMAL&exchange-type=binary&currency=EUR, headers = headers')
print r.text
Below is the error message that I got. It does not make sense to me because I logged in successfully and got the above session-token in response.
{"errors":[{"messages":["You are not authorised to access this resource. Login to continue."]}]}
Am I properly indicating the session-token in the header information of the post request?

You need to pass headers argument in post function.
headers = {"session-token" : "xxxx_b0b8a6f22a82396b6afcfa344f3022"}
response = requests.post('https://api.matchbook.com/edge/rest/reports/v1/offers/current/?odds-type=DECIMAL&exchange-type=binary&currency=EUR', headers=headers)
also if you need to get an json response, just call json() function on response variable.
something like response.json()

Get webpage text from URL after authorization

I need to get the output printed on the screen on accessing a url with username and password. When I access the url through my browser, I get a popup where I enter the credentials and get the output in the browser. How do I do it using python script? I tried the following, but it only returns <Response [200]> which means that the request is successful. The output I want is a simple text message.
import requests
response = requests.get(url, auth=(username, password))
print response
I have tried requests.post also, with same results.

print response tries to print out a Response object. If you want the text of the response, use print response.text.
You may want to read the Quickstart documentation for the python-requests library here: http://docs.python-requests.org/en/latest/user/quickstart/.

Python - POST request repsonse and JSON parsing

I´m using Python 2.7.7 to send a post request to a website. Im using the requests module and my code looks like this: (NAME and PASS are substituted)
r = requests.post("http://play.pokemonshowdown.com/action.php", data="act=login&name=NAME&pass=PASS&challengekeyid="+challstrarr[2]+"&challenge="+challstrarr[3])
print(r.text)
print(r.json())
r.text returns just a blank line, r.Json returns this error: "ValueError: No JSON object could be decoded"
The website i´m requesting has the following tutorial:
you'll need to make an HTTP POST request to http://play.pokemonshowdown.com/action.php with the data act=login&name=USERNAME&pass=PASSWORD&challengekeyid=KEYID&challenge=CHALLENGE
Either way, the response will start with ] and be followed by a JSON object which we'll call data."
I´m not sure if the post request response is faulty (and hence the blank line) or if its not faulty and the json parsing is off

You should pass a dictionary object to the post function (the data argument), only in the get method you should pass a query string:
postData = {
#put you post data here
}
r = requests.post("http://play.pokemonshowdown.com/action.php", data=postData)
print(r.text)
print(r.json())